Python AI Interface

[abma]

TheGeorge and me wrote an Python AI Interface for spring.

I think we will start to write an AI when it is integrated into the spring installer, currently there is only an Hello-World AI.

As we got no bug-reports (noone tested it?) it would be useful if someone would really test it and write an ai or so

For short setup instructions look here:

http://springrts.com/wiki/AI:Development:Lang:Python

We are happy about any kind of feedback!

[Neddie]

I'll try to bring this to the attention of a friend who may be interested.

[echoone]

Does the interface make a copy of game state before it passes it off to python or does python directly expose the data structures of the game engine's state?

[thegeorge]

There are two ways the interface interacts with Spring:

Events:
The engine generates events which are translated into a Python data structure. This is an conversion from a C struct into a Python dict.

Callback:
The first event (init) holds in it's C struct a pointer to a callback object, which also is a C struct. The callback object contains all functions, which can be used to get information about the current engine state or to change it (like send a command to a unit or something else). There is a Python Callback Object, which does all the type conversions. You can use this callback direct or via the the classes, which is much nicer.

The interface only holds a reference to the corresponding callback objects. (if this is what you mean with the engine state)

[echoone]

Wanted to give some real quick feedback.

First, let me say thanks for providing this interface. It saves me from having to do it (I had actually already started - but you guys are much farther along) and I can assure you, your efforts are most appreciated.

While I've not done a lot with the API, I have played a little thus far. The network console interface is really, really, helpful. Since this project is currently sans documentation or even pydoc, allowing for introspection is a real life saver. I'm just throwing this out there, but I seem to recall ipython has an interface to allow for wrapping for network/interactive shells. I wouldn't swear to that though. You might want to see if this is a viable upgrade. Having something with the power of ipython as a console would be simply awesome. Then again, you guys might be considering the console as a short term solution for now.

I have noticed the interface in general is extremely fragile. This is already noted in the release notes but it doesn't hurt to reaffirm what you already know. Parameter validation and exception/error handing would be a huge improvement over crashing Spring. In my opinion, this is the single largest obstacle to prevent wider acceptance of the python interface.

Here's what I'm working on. I've created an "AI" called, "RAIMI" (Remote AI Management Interface). This in turn uses Pyro (version 4 for now - for performance/simplification reasons). As the description implies, its really more of an interface for AIs rather than an AI proper. Then, "user AIs" make themselves known via the name server. On event from Spring, RAIMI looks up an AI for the team and passes the event to the AI. At any rate, RAIMI would be fairly static and generally not changed by would be AI authors. In turn, developers make their AI available via the name server whereby RAIMI is able to discover them. This glosses over various details because its both tedious and I'm working out some of the details and coding.

I fully expect RAIMI will do things like statistical analysis of the map, saving each remote AI from having to do so. None of which is remotely close to being coded yet. It of course, also acts as a synchronizing gateway for callbacks, back into the Spring engine; so as to insure single threadedness. This approach should reduce redundant work between multiple AIs as well as drastically improve performance for AI startup (metal mapping/analysis) without having to make a ton of AI calls to obtain such information (I assume).

In doing so I hope this approach allows each team AI to run on one or more CPUs on one or more computers. There are some very real questions yet to be answered which will determine if my approach has wings. If this works it means each AI can run on different machines/CPUs, independent of the game's CPU load.

If my approach pans out, and that's currently a big if, I intent to provide a complimentary library to aid in developing AIs such that the library helps with common tasks, asset tracking, basic strategies, etc. In doing so, I hope to allow people to concentrate on AI implementation rather than becoming mired in the many small details of AI game development.

I'll keep you guys updated as I continue to make progress.

[thegeorge]

That are great news.

I will implement type checking as soon as possible. I just have little time at the moment. But it will definitely be implemented. If done so, an exception would be thrown. The engine however wouldn't be crashing anymore.

There is no documentation of the interface, because... well, there is not much documentation at all. The interface is generated out of the C sources and the little documentation that is there is not used in the interface. However, there is documentation about the types of the arguments that should be used. In some further version, there will be more documentation about what the function does as there is more in the C interface.

But it's really cool to see someone using the interface allready.

[thegeorge]

Update:

To call functions of the interface, you do not need to instantiate the interface classes anymore.

Past:
self.map=Map(self.team)
self.map.get...()

Now:
Map.get...()

If an event occurs, only the AI which receives the event is able to call the interface.

[thegeorge]

Update:

There now is type checking in the interface, so the engine won't crash if wrong types are used.

[echoone]

I have a couple of minor improvements. Where can I submit patches?

[echoone]

I did a quick search and found nothing. Can we get the UnitCommandOptions structure values exposed in the Python interface? If its already exposed, please point the way. These values are required to allow AI's to provide command options - which I'm now in need of using.

http://github.com/spring/spring/blob/ma ... Commands.h

[hoijui]

This is now the last missing thing (of officially supported stuff) left to be ported to the pureint branch.
Lets have a look at it together when you are in the lobby some day.

[echoone]

BTW, in other threads it was made clear that structure definitions are frequently exposed in other interfaces. Is this planned for the python interface?

Right now, obtaining information frequently requires a series of function calls which are fairly expensive in python. In fact, they are so expensive, getters/setters idioms are frequently a source of performance headache for python apps and as such should be avoided. Unfortunately, this is the exact mode of operation which is forced on AIs by the current interface. As such, having shared structures which directly expose attributes would be a significant performance gain.

In my own testing, while these are not currently performance bottlenecks, they are playing a significant role in "bursty" behavior. I've not determined if this is an inherent artifact of my design but I do not believe it to be so. Regardless, more testing is required to actually draw a conclusion.

What happens is, during large games, lots of units require a refresh of current stats. So for example, lets say we have fifty units which require refresh. If we are interested in four attributes for each, we now have 200 synchronously blocking function falls. In large battles fifty units represents a fairly low count and that still ignores that other functions will likely need to be periodically called too.

One thing that will help, at least a little, is to release the python lock for callbacks. This allows pure python threads to continue to run while another thread is busy falling a spring interface function. Increased concurrency is always good. Given the automated nature of the interface, this is likely easy to add.

Having said all that, I have identified several ways my framework have help avoid these issues, including the bursty issue, just the same, I feel there are several items which can be improved to help further alleviate the issue, thusly reducing the work required of python AI developers.