Interface Redesign

[hughperkins]

Hm, not official supported is bad.

Agreed. The quotation from their admin of "What is MiniGW?" doesnt bode well

What about D-Bus?
But I see a problem with windows. There is a port of D-Bus (winDBus) but I don't know how good and up to date it is.

Yes, does sound interesting. Want to check it out? First thing to check is whether it will build with mingw?

[hughperkins]

Wouldnt a network based layer end up invoking the windows firewall etc? This already happens with access unblock block prompts from innocent looking games.

Yes it would, though it's fairly easy to just accept the prompt that comes up the first time?

Any network layer that short-circuited through pipes would probably avoid this issue?

[Agon]

Hm, not official supported is bad.

Agreed. The quotation from their admin of "What is MiniGW?" doesnt bode well

What about D-Bus?
But I see a problem with windows. There is a port of D-Bus (winDBus) but I don't know how good and up to date it is.

Yes, does sound interesting. Want to check it out? First thing to check is whether it will build with mingw?

Can't I'm a Linux user.

As I understand it, D-Bus is using a daemon to handle communications.

I would prefer a D-Bus implementation instead of an Mono wrapper.
Tobi wrote some pro's and con's about it:

For C# I think making a small C++ AI that allows interproces communication (ie. TCP over loopback) and has a way to start a process is way easier then integrating either mono or the CLR directly into an AI DLL.

EDIT:
Some advantages:

• AI can just use WinForms or WPF or whatever for realtime debugging windows
• Debugging the C# code with express edition of VS and/or MinGW compiled Spring would actually work
• Spring memory corruption can not crash the AI
• AIs compiled for different .NET runtimes can be combined in a single game (it's impossible to load different .NET runtime versions in same native application; that's also the reason windows shell extensions need to be written in C++)
• AI would run in different thread (process) automagically, no need to think about threading issues etc.

Some disadvantage:

• Getting unitdef etc. will be somewhat slower because data actually has to be copied a few times.
• AI may lag a bit behind engine.
• Everything AI does is asynchronous, makes certain things a bit harder.

But I would suggest using D-Bus instead of TCP.
Most Linux user have D-Bus already installed.
And the size is low (~220K) for a the D-Bus library. The Mono wrapper for D-Bus: ~36K

[AF]

Are we suggesting we abandon the C API and go for a network based interface?

I dunno but couldnt we add TCP as an implementation library so we don't end up spending the next year discussing protocols?

[Agon]

Are we suggesting we abandon the C API and go for a network based interface?

I dunno but couldnt we add TCP as an implementation library so we don't end up spending the next year discussing protocols?

I thought only about the third party languages like Java and .Net languages.
Not C/C++.

D-Bus uses objects as I understand it and (de-)serialize them.
http://en.wikipedia.org/wiki/D-Bus
So no protocol is needed. We only need to make the objects public over D-Bus and implement a serialize library in the targeted language.

So it would be like this:
AI-D-Bus loaded->pushes objects public to /games/spring/AI->deserialize objects
extern AI->connects to the virtual D-Bus path /games/spring/AI->serialize objects

No guaranty that this is right

[hughperkins]

Can't I'm a Linux user.

As I understand it, D-Bus is using a daemon to handle communications.

Fine. Let's just run with D-Bus. I've kindof resigned myself to having to reboot into linux to do Spring development anyway.

Can you make a first implementation that just lets us receive Say events, and writes those to console?

[Agon]

Can't I'm a Linux user.

As I understand it, D-Bus is using a daemon to handle communications.

Fine. Let's just run with D-Bus. I've kindof resigned myself to having to reboot into linux to do Spring development anyway.

Can you make a first implementation that just lets us receive Say events, and writes those to console?

Hm, you don't need to use Linux for D-Bus.

I can only implement something like this in C#, because I have near to no knowledge in C++.
Currently installed the new Monodevelop 1.9 alpha release and D-Bus with D-Bus bindings for Mono. So I will give it a try.

Edit:
There are two Mono libraries for D-Bus.
A wrapper for D-Bus (DBus-Sharp) and a implementation in C# available here: http://www.ndesk.org/DBusSharp

[Agon]

Following example are based on NDesk DBus implementation.
All code is written in C#.

Client side example to get DBus object:

Code: Select all

Connection conn = Bus.Session;
			
ObjectPath opath = new ObjectPath ("/org/freedesktop/DBus");
string name = "org.freedesktop.DBus";
	
// Here it serialize the object
IBus bus = conn.GetObject<IBus> (name, opath);	
			
Console.WriteLine ();
foreach (string n in bus.ListNames ())
	Console.WriteLine (n);
	
Console.WriteLine ();
foreach (string n in bus.ListNames ())
	Console.WriteLine ("Name " + n + " has owner: " + bus.NameHasOwner (n));
	
Console.WriteLine ();

An Spring AI example (Client side/AI):

Code: Select all

Connection conn = Bus.Session;
			
ObjectPath opath = new ObjectPath ("/games/spring/ai");
string name = "games.spring.ai";

IAI ai = conn.GetObject<IAI> (name, opath);	

// Here you can implement the AI
// For example:
// Unit unit = ai.GetUnit(200);

[hoijui]

Are we suggesting we abandon the C API and go for a network based interface?

this question seems to be not answered yet.

Agon.. you mean.. we do the C interface as planned, and you will do dbus interface using the C interface or the C++ one (seems more appropriate, as we could have objects like Unit or UnitDef there already). or do you want to abbadon the C interface?

if you want to use D-Bus for a C# (and possibly other languages) implementation of the interface, using the C interface. then this belongs into a thread for a C# interface implementation, and not into this thread.

if you want to replce the c interface with a dbus one...
what are the pro and contras?

btw, i am coding on the C interface (the callback).

[AF]

tbh if I'm forced to use another Operating system to develop my AI then I'm not going to be a very happy bunny at all. This interface is supposed to make life easier not harder.

Implement a DBus AI implementation lib over the C API as we discussed earlier with other bindings. If we go this way then a thread split for the dbus topic is needed.

[Agon]

I meant a D-Bus "interface" over C++.
But I can't do it currently because I can't code in C++ :/ and I don't like it.

D-Bus can be used under Windows, so you are not forced AF

I will do a split.

[AF]

Id just rather not add more and more dependencies. A pure C API with implementation libs for bindings and other interfaces is more appropriate than taking a single library and putting it in as a one shoe fits them all policy.

[zenzike]

Id just rather not add more and more dependencies. A pure C API with implementation libs for bindings and other interfaces is more appropriate than taking a single library and putting it in as a one shoe fits them all policy.

I totally agree with AF. If we want to have a DBus (or other network / messaging) system, then this should be built on top of either the C/C++ interface we've been discussing.

[AF]

That gives those who disagree the option for a TCP or pipe implementation, or native bindings. Either way flexiblity and modularity is key.

[Agon]

That gives those who disagree the option for a TCP or pipe implementation, or native bindings. Either way flexiblity and modularity is key.

Totally support this. As I wrote above I thought about a D-Bus interface above the C++ interface.

Back to the interface redesigning. How is the progress? What is missing to discuss and what needs to be implemented?

[zenzike]

Nothing is missing: I finished the handleEvent side of things, and hoijui is working on the handleCommand.

I had a thought about the function callback system though. I expect it will get rejected on grounds of "too complicated", though it does mean that all AIs are completely forward and backward compatible.

The handleCommand() is designed to set the state of the engine, by doing things like moveUnitCommand, sendMessageCommand. These are wrapped up in structs and setn in handleCommand, just like handleEvent.

The problem comes when we request data from the engine -- the way we have decided to do it is by having a function for each piece of data required. For example,

Code: Select all

int getUnitSpeed(int unit);
SFloat3 getUnitPosition(int unit);
...

and making all these functions available across the interface.

The problem with this is that if a future version of an AI requests a function from an old engine, for example getMorale(), then the engine won't have that function, and the AI will bomb out.

The solution might be to use a mechanism similar to handleEvent and handleCommand, called getFunction:

Code: Select all

void* getFunction(int team, int functionID);

where the return value is a function pointer to the function named in functionID. The good thing here is that if the ID number is implemented, then we can be sure to know what struct or pod to cast to for the return.

This means that if the AI asks for getMorale(), but this does not exist, then the engine can return 0. Consequently, the AI can either disable any calculations involving getMorale() by testing for 0, or by implementing a default function that returns 100% morale.

This way there are 4 points of entry/exit between the engine and the AI:

Code: Select all

int init();
int handleEvent();
int handleCommand();
int getFunction();

Any comments/suggestions?

[hoijui]

Back to the interface redesigning. How is the progress? What is missing to discuss and what needs to be implemented?

yeah, we all agreed on the same for the D-Bus thing about 6 posts ago already, no need to go on with that :D

i am coding right now. the callback has a lot of functions, and as there will be a separate function for each part of UnitDef, WeaponDef, FeatureDef and the structs in there too, it will take some time till i did all this.
a question...
i have this:

Code: Select all

struct SAIFloat3 {
	float x, y, z;
}

how should i convert float3 <-> SAIFloat?

Code: Select all

float3 f3;
SAIFloat3 aif3;
f3 = reinterpret_cast<float3>(aif3);
aif3 = reinterpret_cast<SAIFloat3>(f3);

is this correct? or should it be functions like this:

Code: Select all

float3 SAIFloat3_to_float3(SAIFloat3 aif3) {
	float3 f3;
	f3.x = aif3.x;
	f3.y = aif3.y;
	f3.z = aif3.z;
	return f3;
}
SAIFloat3 SAIFloat3_to_float3(float3 f3) {
	SAIFloat3 aif3;
	aif3.x = f3.x;
	aif3.y = f3.y;
	aif3.z = f3.z;
	return aif3;
}

or what else?


left to discuss, are currently the individual functions of the callback, as i see it. of course, the wrapped C++ function will have to look exactly equal to the current one, but we have the possibility to change the C one to be a bit nicer. i am thinking mainly about these functions:

Code: Select all

int (*getCurrentFrame)(int teamId);
bool (*getProperty)(int teamId, int id, int property, void* dst);
bool (*getValue)(int teamId, int id, void* dst);
int (*getFileSize)(int teamId, const char* fileName);

Code: Select all

bool (*readFile)(int teamId, const char* name, void* buffer, int bufferLen);
void* (*createSharedMemArea)(int teamId, char* name, int size);
void (*releasedSharedMemArea)(int teamId, char* name);

in the C interface, we get the frame through the Update event, so there is practically no need to call back to the engine with getCurrentFrame(), and as the C interface is new, ther is no need to have that function. would you keep it?
getProperty() and getValue() are meant for future callback extensions, but i think are not currently used at all. i think they make no sense at all (in the C interface), as the C callback with functions we currently have, can be extended with normal functions without breaking compatibility with old AIs, right?
getFileSize() and readFile(). i guess there is a reason for this (why it should not be done by the AI directly), but i don't know it. can somebody explain?
and of course the createSharedMemArea() and releasedSharedMemArea(), which we started to discuss before. i was the last one who gave some arguments, an after that, nobody commented anymore. in short: it is an unused low level (and as i think complicated and insecure, and for other languages unpractical) way for AI<->AI interaction (eg, the AIs could keep Metal-Spot-Location Cache in a shared area, to speed up overall AI loading and save memory). as we plan to have an event based AI<->AI communication anyway, i though we could get rid of that now, in the C interface (possibly in the C++ one too, some when).

[hoijui]

we solved the problem you described (the old engine dying because of a new AI) with the version number. engine asks all AIs for the engine/interface version they support, if the version they return is higher the the engines one, the AI will not get loaded. both things have pros and contras. in your way, they AI may still run, but not well, while with the version check, the AI will not even be loaded (eg, the commander will d nothing). the pro of the version check is, that one knows that an older AI version or a newer engine version is needed.

the thing is, i don't remember anymore why we chose not to pass everything through handleCommand(), but use functions. as there is no technical difference between passing everything through handleCommand() or have handleCommand and your new getFunction() way, the same issues should apply. soo.. what was it again?

an other though... couldn't the AI put default empty implementations into all callback function pointers, to achieve the same effect (no engine crash)?

[hoijui]

if you go to page 5 of this thread, the second post (Tobi's first post on that page), you find the reasons why not to use a generic callback function (like your getFunction(), zenzike). they are purely technical, low level.

[zenzike]

if you go to page 5 of this thread, the second post (Tobi's first post on that page), you find the reasons why not to use a generic callback function (like your getFunction(), zenzike). they are purely technical, low level.

Actually no, this is a different solution. Tobi's comments were about passing pointers to the struct/variables as the return of the function. Here I am suggesting we return *a function pointer* this is not the same thing, and avoids all the problems.

We haven't solved this problem, since it would be silly if an AI that would work perfectly well on an old engine doesn't get the choice to do so: for example, if spring has unit morale (like in DOW), then the AI should be able to default to a 100% morale function when the engine can't supply the data. Using the functions that we are currently doing, things would break. If we use function pointers, then the return would be 0 and so we can switch to default behaviour.

We can't pass through handleCommand for getState() type functions, only for setState() ones.