GlobalAI ABI Compatibility Layer ("ABIC")

[hughperkins]

ABI Compatibility layer ("ABIC")

What's an ABI compatibility layer and why could we need it?

Currently, the compiler used to compile Spring core dictates the possible compiler to use to compile global AIs.

The ABI compatibility layer aims to allow us to use any compiler to compile global AIs, regardless of the compiler used to compile spring.

Status

Feature complete with the AI .Net interface, except for GiveGroupOrder, AddMapPoint, GetMapPoints.

How does it work?

The ABI compatibility layer converts the C++ GlobalAI interface presented by Spring to a C-interface. C-interface ABIs are standard across different compilers, a few underscores notwithstanding.

What's an ABI?

ABI is "Application Binary Interface". It's what gets exposed by dlls and sos, and is used to link the dll with another application at runtime. There are two parts to an ABI:
*name mangling. This means: how do we convert the name of a function, possibly in a class, to some globally unique string
*argument marshalling. How do arguments get passed to and from the function across the link boundary

C-interface argument marshalling is standardized across compilers. At least, each compiler is capable of certain calling conventions, such as extern "C", and there are calling conventions which are portable across multiple compilers.

C-interface name mangling is almost standard. Some compilers add an underscore at the start, some dont; its possible to use dlltool or similar to tweak this.

C++ ABIs are not standard across compilers, neither for name mangling nor for argument marshalling. C++ ABIs arent guaranteed to be standard across different versions of the same compiler.

Download

ABIC is now integrated into Spring SVN, in the directory rts/ExternalAI/GlobalAICInterface. See AI/Global/TestABICAI for an example of using ABIC directly.

Abic++ C++ wrappers are available within AI/Global/CSAI/AbicWrappers directory n Spring SVN. Just add the following includes to your program:

Code: Select all

#include "AbicAICallbackWrapper.h"
#include "AbicFeatureDefWrapper.h"
#include "AbicMoveDataWrapper.h"
#include "AbicUnitDefWrapper.h"

[hughperkins]

Updated the prototype in the URL above.

*now includes both an msvc-compiled and mingw-compiled helloworld AI

The AIs will greet you, state the mapname, the commander's unitname, and the commander's maximum slope.

This shows that we can use:
*GlobalAI
*IAICallback
*UnitDef
*UnitDef::movedata

How the code in the AIs looks:

Code: Select all

DLL_EXPORT void InitAI(int team)
{
    char buffer[1024];
    
    SendTextMsg( "Hello from abi-called dll!", 0 );
    
    sprintf( buffer, "The map name is: %s", GetMapName() );
    SendTextMsg( buffer, 0 );
}

DLL_EXPORT void UnitCreated(int unit)									//called when a new unit is created on ai team
{
    char buffer[1024];
    
    const UnitDef *unitdef = GetUnitDef( unit );
    sprintf( buffer, "Unit created: %s", UnitDef_get_name( unitdef ) );
    SendTextMsg( buffer, 0 );

    const MoveData *movedata = UnitDef_get_movedata( unitdef );
    sprintf( buffer, "Max Slope: %f", MoveData_get_maxSlope( movedata ) );
    SendTextMsg( buffer, 0 );
}

We are declaring the UnitDef and MoveData as an opaque pointer. This pointer is passed in as the first argument to each accessor function.

[hughperkins]

Well, in case anyone wants to run with the idea anyway, here are the interface headers ported to C.

Ok, you've converted each class into a struct, and each function call into a pointer. The UnitDef is transferred as a pointer to a native UnitDef.

Code: Select all

struct IAICallback_c {
	void (*SendTextMsg) (const char* text, int priority);
	const char* (*GetMapName) (void);
	const UnitDef* (*GetUnitDefByID) (int unitID);
        ...
};

Using a struct to hold the function pointers provides namespace encapsulation. The UnitDef is being transferred as a pointer, which is efficient. The design and usage are very close to the current interface.

How sure can we be that the structure-packing will be invariant across compilers? How sure can we be that UnitDef::buildOptions will be accessible across compilers?

[submarine]

if i understood you correctly we should all use that in our ais, right?

i use vs.net 2003 to build aai binaries - if the next spring release will be compiled with gcc(ming gw) they would not be compatible anymore right?

[Tobi]

GCC and MSVC both have a mechanism to prescribe the packing of structs:
Something like this works on both IIRC:

Code: Select all

#pragma push(pack, 1)
#pragma pop

and GCC also has:

Code: Select all

struct __attribute__((packed)) {
  int blah;
}

As for getting unitdef values, a few functions with an integer which argument (with #defined constants) could work, a bit like OpenGL:

Code: Select all

#define UNITDEF_MAXHEALTH  0
#define UNITDEF_NUMWEAPONS 1
//etc.
extern "C" {
int GetUnitDefInt(UnitDefHandleType handle, int which);
float GetUnitDefFloat(UnitDefHandleType handle, int which);
};

The function itself could map the #defines to offsets by a simple lookup table:

Code: Select all

extern "C" int GetUnitDefInt(UnitDefHandleType handle, int which) {
  static const int offset[10];
  // on first call, fill the offset table
  if (!offset[0]) {
    CUnitDef ud; // just here to calc offsets
    offset[0] = int(&ud.maxHealth - &ud),
    offset[1] = int(&ud.numWeapons - &ud),
    //etc.
  };
  // ud actually points to a CUnitDef
  char* ud = (char*)handle;
  return *(int*)(ud + offset[which]);
}

Only bad thing is that you need to know the type then, but that could be incorporated in the #defines.

And the implementation looks a bit hackish of course, but it is pretty flexible, and doesn't need an enourmous pile of exported functions.

Maybe it could even be implemented using creg, so it doesn't even require maintaining yet another list of member variables. Maintaining it separately could give the advantage that you could hide unused/unusefull items from the AI.

[jcnossen]

Almost;)

Code: Select all

#pragma pack(push, 1)
#pragma pack(pop)

Tobi's method sounds good, it would indeed be easy to use creg for it. But you'd still need a few specialized functions to deal with the STL containers..

[AF]

hugh, your implementing the AI within the DLL EXPORT functions, which puts it at a huge disadvantage in that there can never be more than 1 instance of the AI.

It would need that you make it call a class derived from IGlobalAI and manage the different team n# calls accordingly.

[jcnossen]

hugh, your implementing the AI within the DLL EXPORT functions, which puts it at a huge disadvantage in that there can never be more than 1 instance of the AI.

No, there can be multiple instances of hugh's AI binding, and each can load a different AI implementation DLL.

[hughperkins]

No, there can be multiple instances of hugh's AI binding, and each can load a different AI implementation DLL.

Just to confirm this, it's been tested with 2 AIs simultaneously, and it worked ok. It worked both for 2 msvc AIs and for a combination of an msvc AI with a mingw AI.

[AF]

hmm, whever I share global values they're accessible across AI's, so your saying that this is no longer the case?!

[hughperkins]

if i understood you correctly we should all use that in our ais, right?

i use vs.net 2003 to build aai binaries - if the next spring release will be compiled with gcc(ming gw) they would not be compatible anymore right?

mingw works well when combined with an appropriate IDE, such as Code:Blocks.

The ABI interface is a C interface and will need a wrapper layer to work with AIs presenting a C++ interface.

[hughperkins]

hmm, whever I share global values they're accessible across AI's, so your saying that this is no longer the case?!

You're right, it does sound strange. Going to run a test: run one AI with ARM and one AI with CORE. If they are sharing the same aicallback, they will both state that their commander is an ARMCOM. If the aicallback is localized to each instance, they will reply correctly according to their own faction.

[hughperkins]

Ok, it worked ok:

Code: Select all

Player hughperkins joined as 0
GlobalAI0: Hello from abi-called dll!
GlobalAI0: The map name is: SmallDivide.smf
GlobalAI0: Unit created: ARMCOM
GlobalAI0: Max Slope: 0.330869
GlobalAI1: Hello from abi-called dll!
GlobalAI1: The map name is: SmallDivide.smf
GlobalAI1: Unit created: CORCOM
GlobalAI1: Max Slope: 0.330869
User exited

[hughperkins]

Correction: getting the commandername proves nothing, since the commandername is being obtained using the id passed into the UnitCreated event.

To test separation of AICallback, we need to check something that is directly team specific, such as GetMyTeam().

Here's the result:

Player hughperkins joined as 0
GlobalAI0: Hello from abi-called dll!
GlobalAI0: The map name is: SmallDivide.smf
GlobalAI0: Our ally team is: 0
GlobalAI0: Unit created: ARMCOM
GlobalAI0: Max Slope: 0.330869
GlobalAI1: Hello from abi-called dll!
GlobalAI1: The map name is: SmallDivide.smf
GlobalAI1: Our ally team is: 1
GlobalAI1: Unit created: CORCOM
GlobalAI1: Max Slope: 0.330869
User exited

[hughperkins]

For the record, here's the result with 3 msvc AIs and 3 ming AIs:

Player hughperkins joined as 0
GlobalAI0: Hello from abi-called dll!
GlobalAI0: The map name is: SmallDivide.smf
GlobalAI0: Our ally team is: 0
GlobalAI0: Unit created: ARMCOM
GlobalAI0: Max Slope: 0.330869
GlobalAI1: Hello from abi-called dll!
GlobalAI1: The map name is: SmallDivide.smf
GlobalAI1: Our ally team is: 1
GlobalAI1: Unit created: CORCOM
GlobalAI1: Max Slope: 0.330869
GlobalAI2: Hello from mingw!
GlobalAI2: The map name is: SmallDivide.smf
GlobalAI2: Our ally team is: 2
GlobalAI2: Unit created: ARMCOM
GlobalAI2: Max Slope: 0.330869
GlobalAI3: Hello from mingw!
GlobalAI3: The map name is: SmallDivide.smf
GlobalAI3: Our ally team is: 3
GlobalAI3: Unit created: ARMCOM
GlobalAI3: Max Slope: 0.330869
GlobalAI4: Hello from abi-called dll!
GlobalAI4: The map name is: SmallDivide.smf
GlobalAI4: Our ally team is: 4
GlobalAI4: Unit created: ARMCOM
GlobalAI4: Max Slope: 0.330869
GlobalAI5: Hello from mingw!
GlobalAI5: The map name is: SmallDivide.smf
GlobalAI5: Our ally team is: 5
GlobalAI5: Unit created: ARMCOM
GlobalAI5: Max Slope: 0.330869
User exited

[imbaczek]

Nit: please don't use unguarded sprintf. Use snprintf or dynamically allocate the buffer.

[hughperkins]

New release of ABIC. The file at http://manageddreams.com/csai/ABICompatibilityLayer.zip has been updated.

This release is a big change. We have nearly complete coverage now.

Changes this release:
*generator created
*generator works for: FeatureDef, UnitDef, MoveData, IAICallback
*aicallback is no longer global; there are no global variables in ABIC currently
*calls to IAICallback now follow the ABIC convention of passing an opaque pointer to self as the first parameter
*aicallback re-added as first parameter to InitAI
*UnitDef, FeatureDef, MoveData and IAICallback are all included now
*fairly complete coverage of UnitDef, FeatureDef, MoveData and IAICallback

Example AI:

Code: Select all

struct IAICallback *aicallback;

DLL_EXPORT void InitAI( struct IAICallback *aicallback, int team)
{
    char buffer[1024];
    ::aicallback = aicallback;
    
    SendTextMsg( aicallback, "Hello from abi-called dll!", 0 );
    
    sprintf( buffer, "The map name is: %s", GetMapName( aicallback ) );
    SendTextMsg( aicallback, buffer, 0 );
    
    sprintf( buffer, "Our ally team is: %i", GetMyTeam( aicallback ) );
    SendTextMsg( aicallback, buffer, 0 );
}

DLL_EXPORT void UnitCreated(int unit)									//called when a new unit is created on ai team
{
    char buffer[1024];
    
    const UnitDef *unitdef = GetUnitDef( aicallback, unit );
    sprintf( buffer, "Unit created: %s", UnitDef_get_name( unitdef ) );
    SendTextMsg( aicallback, buffer, 0 );

    const MoveData *movedata = UnitDef_get_movedata( unitdef );
    sprintf( buffer, "Max Slope: %f", MoveData_get_maxSlope( movedata ) );
    SendTextMsg( aicallback, buffer, 0 );
}

[AF]

Could you change ti from

string s = getmapname(aicallback);

to

string s = aicallback->getmapname();

????

Its just that flattening everything out makes for a lot more conversion and porting work, and isnt particularly easy compared to the old style...


And use this instead of sprintf():

Code: Select all

template<class type> inline std::string to_string( const type & value) {
	std::ostringstream streamOut;
	streamOut << value;
	return streamOut.str();
}

or at least use safer versions of it.

[AF]

Code: Select all

AICALLBACK_API bool UnitDef_get_loaded( const UnitDef *self )
{
   return ( ( UnitDef *) self)->loaded;
}

So we could replace

Code: Select all

bool loaded = UnitDef_get_loaded(ud);

with

Code: Select all

bool loaded = ud->loaded;

And avoid a lot of unnecessary code and reduce masses of issues with renaming loads of unitdef calls within existing AIs?

I dont see why we cant just change UnitDef so it has struct not class const char* not string and only have to change the STL containers instead of using all these extra functions......

Nor why we cant just fill in UnitDef classes on the ABI side after crossing the C boundaries

[hughperkins]

Code: Select all

AICALLBACK_API bool UnitDef_get_loaded( const UnitDef *self )
{
   return ( ( UnitDef *) self)->loaded;
}

So we could replace

Code: Select all

bool loaded = UnitDef_get_loaded(ud);

with

Code: Select all

bool loaded = ud->loaded;

There are two reasons:
*passing structs might not be compiler-independent
*passing structs is spring version - dependent

Currently, upgrading the spring version means recompiling the AIs. If we link by function using a def file, old AIs stand a chance of working with a newer Spring release.

printf ...

Note that the printf is only in the test AI. There are no printfs in ABIC.