Interface Redesign

[Kloot]

Code: Select all

so the AI can casue desyncs

Not unless they change the FPU control word, which no present AI
does (JVMs don't count as AIs ). There's no technical reason why
AI libs are linked against streflop by the buildsystem, their "callback
input" (commands) looks like a player's from the perspective of the
engine so it can't be synced to begin with.

[hoijui]

yeah i know... thats what i was trying to explain, that they will desync if the FPU controll word gets changed, as does the JVM, and probalby other VMs? as the C interface would allow one to compile and use whatever , however on the AI side (with VMs or compilers changing the FPU), it would be convienient to integrate the FPU controll word reset into the interface, even if it would possibly not be needed.

[Kloot]

Ah right, then it would be best to let AI's decide for themselves whether
the interface should continually reset the FPUCW for them and have the
resetting done behind the scenes on an as-needed basis (instead of for
every AI by default).

[AF]

Perhaps this could be optional so it only does it when the AI indicates that its needed and only for that AI.

[Tobi]

Seriously, before taking any decision first measure performance of fpuldcw and fpustcw instructions and read the intel architecture reference manual (or whatever it was called) about it.

No point tyring to decide on opt-in/opt-out/always on approaches if you don't even know anything about the performance.

If the instruction is effectively as cheap as e.g. add or sub then opt-in/opt-out is just waste of time, could easily reset it after every call then.

If however it has side effects like flushing/invalidating L1/L2 cache or takes a lot of cycles to execute then it's probably best to just have it in the occasional AI that needs it. (And possibly just after loading the AI DLL, since that's a tricky area in which fpu cw resets actually happen in certain circumstances.)

In the end though I think it's AIs responsibility; otherwise we may end up with calling stuff like emms or femms and whatever other obscure instructions to reset (CPU) flags that may sometime ever be messed up by some AI (think about FPU interrupts, signal handlers, anything external to process, processwide winapi/crt settings, crashhandlers, SEH, etc.). Only the AI dev knows what stuff he is messing with so he is the only one who can reset it to original state.

[hoijui]

yeah.. maybe with

Code: Select all

#ifdef RESET_FPU

hmm.. what about the data passed through the interface (through the HandleEvent method)...
will it only be primitive data, like int, float, char*, ...
or also structs?

edit:
if we it as define on the AI side, we can use a compiler flag, right? so the AI dev can decide but does not have to worry about it.
why i am so concerned about this is, because.. i personally wasted a lot of time because of this, and want to make it easyer for others; save them time.

[AF]

I would rather use primitives and arrays where possible. Im not sure how portable structs are between native compiled languages. Using arrays also reduces the overloads needed rather than invoking the great 100 events == 100 structs arguement.

[Tobi]

Structs are quite portable, as long as we set a fixed alignment with #pragmas for them.

(Remember they're being used too to read file headers etc., meaning layout is pretty much fixed as long as you take care of the alignment of the members.)

In my opinion by not using structs in the interface you are making your life a hell of a lot harder

By casting pointers to void* you can also limit overloads.
By reusing event structures for similar events you can also reduce the number of event structures. (Note though that this does make it harder to later add parameters to an event.)

For example (AI side sample code that is called by Spring, and translates the event to a call to IGlobalAI function as we know it now):

Code: Select all

DLL_EXPORT
void HandleSimEvent(void* aihandle, int id, void* data)
{
  IGlobalAI* ai = (IGlobalAI*)aihandle;
  switch (id) {
    case ENEMY_ENTERED_LOS:
      ai->EnemyEnteredLos(
              ((EnemyEnteredLosEventArgs*)data)->enemy_unit_id);
      break; 
    case ENEMY_LEFT_LOS:
      ai->EnemyLeftLos(
          ((EnemyLeftLosEventArgs*)data)->enemy_unit_id);
      break; 
    case ENEMY_ENTERED_RADAR:
      // ...
    case ENEMY_LEFT_RADAR:
       // ...
  }
}

EDIT: Another option could be to make a big union of event structures, so you have a bit more type safety, and just need to know which member of the union you need to access. This is less used for portable code though and I'm not sure how portable unions are.

In above example it would go like:

Code: Select all

      ai->EnemyEnteredLos(
              data.enemy_entered_los.enemy_unit_id);

[AF]

As someone pointed out void* and swig are not happy bunnies, and when put into a relationship tend to generate devil children or so I've heard.

However its nice to know structs are portable.

[hoijui]

:D
thats the truth, also heard that about SWIG and void*
i personally would prefer not to use structs either. though they are supported by SWIG eg, nested structs are not for example, and we have some. and as the whole idea of using C is to make the AI more portable, i would like to use primitive types and arrays only as well, as someone else might get into troubles with structs. i already said i would do the boring work of translating everything, like adding event messages for everything in UnitDef and everything else, and implement the code handling the events. am currently working on my example code, including about 6 sample messages and their handling, will post it here if its ready.

[hoijui]

DOH!
SWIG does not support varargs, and all other ports to languages which dont have varargs themself will surely have problems too with it. so all these ports (eg JAI) will have to rely on an AI side C wrapper of the interface, which consists of individual functions for each event, which are claled by the HandleMessage function(). though thats not too bad, as such a thing is handy anyway.

edit:
then again... wouldn't it be easyer to have most functions as such, and not let them go through HandleMessage()? using Handle Message only for future messages?

[Tobi]

As someone pointed out void* and swig are not happy bunnies, and when put into a relationship tend to generate devil children or so I've heard.

I was under impression SWIG wasn't going to be used for this interface?

[hoijui]

yes, thats true, SWIG is not going to be used for this interface. but its meant only to improve compatibility. it is not only that SWIG and therefore JAI has problems with void*, but in general it makes things more obscure, and harder for other languages.

How i think of it:
if the C interface will be finnished once, we will have 2 interfaces anyway, the C and the C++ one. the C++ interface will be as the current one is, with all the classes and structs and such: Object Oriented.
The C interface is not intended to be used directly by AI devs, as it will never be as comfortable as eg the C++, the Java or the C# one. Thus the C interface is only a kind of bridge, purely for compatibility, on which other interfaces are built upon.

There could possibly also be a C interface using structs, eg a UnitDef struct and others, which then allows to code an AI in pure C, more comfortably. But this interface would be built uppon the basic C interface, which is not using any structs or other "fancy" stuff.

[hoijui]

A first draft of the interface. its quite C like, but some things are still pseudo code. It contains nothing for the LUA stuff, as i have no idea about that.

I am using 2 #defines for one message topic, as it is the most simple and fast way i could think of. this way, one can use the define for the topic index, or the topic name, or the topic index or name (as string) directly. It requires once defined topics to not be changed anymore.

I use 4 files so far, though it may have to be more, and i also mixed a bit of C++ into the implementation of the engine side implementation. This has to be separated better for sure.

what do you think?

[AF]

DLL_EXPORT int HandleMessage_AI(int myTeamId, int messageId, int messageTopic, ...) {

Do not use that, use function overloads instead. The very thought of vargs and their usage makes my stomach churn

Code: Select all

case MSG_TOPIC_INDEX_LONG_TASK_START:

I do not understand where this fits in or what the point is.

However the rest seems okay for now.

I'd also like to point out that we will always have one interface on the AI side. We're not developing a second interface, we're rebuilding the existing interface from the inside out.

As such I would expect classes such as CAICallback and CGlobalAICallback to be split up into 2 classes, a wrapper class and an implementation class, with the implementation class being incrementally shifted from C++ to C with the wrapper class on the AI side also being used to side step the C++ classes in favour of the C API.

Think of it as the C++ interface being a foundation with the AIs standing on top. What we're doing is giving a second foundation and putting one leg on each foundation, and slowly shifting our weight over until we can lift our foot off the c++ foundation and stand completely on the C foundation.

[hoijui]

ok... good you explained it agian.. though.. i understood it this way already. i said that we will have 2 interfaces, as in: an AI dev can choose between 2 interfaces, thoug there will only be one AI interface for real, which will be the C one, and the other interface (C++) is just a wrapper of the same interface. but for the user, it can be seen as 2 interfaces, which he can choose from.

the LONG_TASK was meant as an example for an asynchronous message/event (requesting now, gettign the answer somewhen later, as an other message). you were talkign about this, that it may be used in the future for something.

VarArgs:
hmmm...
function overloads...

Code: Select all

int HandleMessage_AI(int myTeamId, int messageId, int messageTopic, int arg1)
int HandleMessage_AI(int myTeamId, int messageId, int messageTopic, int arg1, int arg2)
int HandleMessage_AI(int myTeamId, int messageId, int messageTopic, int arg1, float arg2)
int HandleMessage_AI(int myTeamId, int messageId, int messageTopic, float arg1, float arg2, int* arg3)
int HandleMessage_AI(int myTeamId, int messageId, int messageTopic, const char* arg1, int arg2, float* arg3, int arg4)
.
.
.

?? :/
i mean...
this way, the interface will change whenever we need a combination that is not yet used. seems very confusing to me, no?
we will end up with 50 HandleMessage_AI() methods or so, each with 1 to 30 events to handle.

what about this:

Code: Select all

int HandleMessage_AI(int myTeamId, int messageId, int messageTopic, int* intArgs, float* floatArgs, char* charArgs)

this way we would need only one method, though the implementation will be ugly as hell -> is not feasable.


i guess overloading still is the best of the three options. :/

[AF]

I would say using a float array rather than multiple float arguements is best. I would also advise adding a parameter indicating the length of the array.

Alternatively those arrays could be changed into a basic C structure:

Code: Select all

struct AIMsgArg {
    char* stringValues;
    int stringValuesLength;

    float* floatValues;
    int floatValuesLength;;

    int* intValues;
    int intValuesLength
/* etc... */
};

int HandleMessage_AI(int myTeamId, int messageId, int messageTopic, int argCount, AIMsgArg** args);

[hoijui]

:D
well...
i would prefer to not use the struct, but anyway, we would rather need this:

Code: Select all

    char* stringValues;
    int* stringValLenghs;
    int numStringVals;

    float* floatValues;
    int* floatValLenghs;
    int numFloatVals;
...

lets say we have 2 floats and 3 float arrays as argument. then we store them all in floatValues, their lengths (eg {1, 1, 3, 40, 3}) in floatValLengths, and numFloatVals will be 5.
but handling this is more ugly then varArgs even, whether in a struct or not.

overloading? :/

[Kloot]

Why not just implement the basic vector ADT for passing a variable
number of arguments of variable types instead of all this hackery?
The vector would maintain its own length and each element would
be a simple container struct around a void* and (say) a type-enum
so casting the void*'s back to their appropriate real types would be
easy and transparent, no need for varargs or unwieldy "overloading"
mechanisms then.

[Tobi]

Function overloading isn't possible in C, you have to give them all different names. (HandleAIMessage1i, HandleAIMessage2i, HandleAIMessage1i1f, etc.)

Also a design where you pass arbitrary data with arbitrary messages is very bad, because it is totally not self documenting: You have to know beforehand that only HandleAIMessage1i will receive the ENEMY_LEFT message for example, and if you typo'ed it to HandleAIMessage2i you wouldn't receive any error at all, the message would just be ignored because it's passed to the other overload. Also suppose a new parameter would be added to a message. Then suddenly a different function gets called and the same thing happens: no warnings, no errors, no whatever, the message is silently ignored.

In other words, make sure the interface makes good use of compile time (type) checking whenever possible, and use runtime checks otherwise. Also make the code as self documenting as possible (also aids because one can use intellisense then), so no one ever has to figure the code out, but everyone can just read it.

Another point in the thread I don't see is, why decide to use arrays instead of structs because structs are presumably less portable (not, if you ask me), and then put the arrays in a struct after all ??? That's combining the disadvantages to get an even bigger disadvantage: less portability and unreadable code.

Just K.I.S.S., and make one function for each message, or make a single HandleAIMessage function that takes a void* that should be casted to a structure with identical name to name of event (plus/minus some prefix/suffix). Then have the event arguments sit in the structure.

The latter, my solution has the following advantages:

• parameters can be added without even breaking ABI compatibility, by just putting a new member in the struct (on the end).
• There is only one "semantic cast"* on the AI side ((void*) -> BlahblahEventArgs) and one on the Spring side. (compare with arrays where every access is a "semantic cast".
• It is well known design, at least to C programmers.

For the AI -> engine direction I'm a bit more in doubt but if I had to choose I'd go for a C style vtable approach. It's very easy to map this to an IAICallback implementation, it's portable ABI (as much as structs and function calls are portable) and C style vtables are also a well known design.

The other approach for AI->engine I see is making a bunch of public functions in spring.exe, and then linking AIs to spring.exe during build (or some library resulting from spring build). Problem with this however is that, AFAIK, it differs a lot per platform on how to do this, and it isn't always as easy to figure out either. Hence I prefer passing function pointers into the AI, so it can then call the functions indirectly.

*semantic cast: figured that's more or less the best term for this: not necessarily type casting, but "casting" the meaning of a variable (meaning defined by it's name). For example, int xposition = yposition; would be a "semantic cast". Same for int unitid = param1;, which is what I'm targetting.

EDIT: oh and indeed AF is right about varargs, varargs calling conventions are even more clumsy then normal calling conventions, so we'd only run into trouble with them. Let alone the incredible high chance for errors when using them (no compile nor runtime checking again..)