SAIL - A Python AI Framework

SAIL - A Python AI Framework

Here is where ideas can be collected for the skirmish AI in development

Moderators: hoijui, Moderators

Post Reply
echoone
AI Developer
Posts: 150
Joined: 16 Nov 2009, 18:26

SAIL - A Python AI Framework

Post by echoone »

Thought I might give a heads up as to what I've been working on. Thanks to the most excellent python interface, I've been pretty busy. Here's something I've thrown together to give a quick overview of what I've been doing. Obviously it reaches far beyond just another AI.

Current projects:
RAIMIP - Remote AI Management Interface Proxy
This is not so much an AI proper as it is an interface for other AIs to leverage. Ignoring that, RAIMIP is the AI selected within the lobby. This uses the excellent Pyro RPC layer to locate and communicate with AIs created with the SAIL toolkit.

RAIMIP is very much a proxy. It maintains very little state. The state it does maintain is used to intelligently push events to SAIL AIs based on a combination of local unit tracking and event trigged by the game engine. The proxy also processes requests from SAIL AIs at deterministic developer specified intervals.

The proxy does not directly react to any events. Rather it updates internal state and obtains state for SAIL AIs as directed by SAIL AIs.

RAIMI - Remote AI Management Interface
A package which abstracts various values (constants, definitions, Commands, Interests) to facilitate communication between SAIL and RAIMIP. This decouples RAIMIP from SAIL and SAIL from RAIMIP. By decoupling it allows for AI development and limited testing to occur without actually running the spring engine or RAIMIP. As such events can be easily and independently triggered (couple lines of code with Pyro knowledge) in a test environment without requiring developers to constantly start/stop or even run the spring engine. This has a direct result in speeding AI development over and above what is already gained by developing in python.

SAIL - Spring AI Library
Aims to provide a multitude of benefits to would be AI developers by allowing them to concentrate on their AI implementation and behaviors rather than the multitude of common framework which is required before development can even begin.

In addition to accelerating AI development SAIL also completely abstracts and hides Pyro's RPC layer. SAIL "just works" without requiring Pyro, networking, or even RPC specific knowledge. Though to be clear, a properly configured networking stack is required. You might be surprised to learn the number of incorrectly configured windows boxes there are.

Elements I hope to address with SAIL include steerings primitives as well as specific behaviors, concurrency primitives (example, shared [reader/writer] locks), AI Planners (calling controllers), mapping primitives and basic implementations (threat, heat, terrain, metal, height, etc), unit abstractions (with concurrency support), various state caches, various decorators to ease using various primitives (example, locking), and economy primitives.

One major advantage of writing a SAIL AI is that the AI actually runs within its own address space (process). As such, it drastically increases scalability while imposing minimal overhead to the spring engine itself. This means AIs can now scale both horizontally (more CPUs and even more computers (thanks to RPC layer); including more CPUs on more machines) and vertically (faster hardware). This means AIs can continuously run completely independently of the spring engine it self, even without resorting to use of threads. This, of course, opens the door, for as much or as little use, of python's threading and processing modules as desired.

Swarm
Is my own AI. Right now its largely serving as a test bed to drive development for all of the layers on which it sits. Its current goals will remain vague until I'm ready to made it available.

SAIL Concepts

Low Latency and Asynchronousity
The vast majority of spring AI events result in the event being pushed to the SAIL AI via the Pyro RPC layer. Almost all of these events are what Pyro calls "one-way" events. Which is a fancy way of saying, the event is pushed to the SAIL AI and no response is expected, allowed, or even desired. This means the vast majority of spring engine events are extremely low latency in nature. Unlike many AIs which perform a multitude of processing associated with each event, which in turn "holds the frame" for the duration of all events for a given frame, RAIMIP simply notifies the SAIL AI of the event and returns control back to the spring engine. Effectively most events become asynchronous notification events, whereby, once the SAIL AI has been notified of the event, control is immediately returned back to the spring engine. This basically means RAIMIP queues the event within the network layer, control returns to the spring engine.

Ticks
One notable exception to RAIMIP's one-way calls are ticks proper. RAIMIP and SAIL allow AI developers several of ways to manage ticks and developer requested events. In other words, AI developers can configure the ratio of actual ticks received by RAIMIP (directly triggered by game engine) to how many are actually delivered to the SAIL AI (those actually called via RPC by RAIMIP to the SAIL AI). As SAIL AIs can effectively and efficiently run independently of ticks (details up to developer), a lower threshold, and therefore lower overhead on the spring engine, is both possible and preferred.

Timed
As its possible for ticks per second to drop below the maximum 30 ticks per second, the interface allows developers to also specify a timed interface, which is triggered by ticks, to in turn trigger a tick to the SAIL AI. This allows AI developers to contribute in load reduction, if desired. This likely has other uses but this is its primary goal - Command and Interest throttling, etc.

Asynchronous Model
Because SAIL AIs are largely asynchronous in nature, they require the ability to submit Commands and request state (via Interests) from the spring game engine. These tasks are managed via Commands and Interests.

Both Commands and Interests are locally queued within the SAIL framework and dispatched to RAIMIP during ticks. This means the requested SAIL AI tick interval specifies the upper limit at which SAIL AIs can provide feedback (Commands) to the simulation.

Interests
Interests are managed by RAIMIP. In a nutshell, a SAIL AI informs RAIMIP it is interested in monitoring units and/or features at given tick intervals. At the requested interval, RAIMIP collects current values and makes them available to the SAIL AI on that request frame interval.

Commands
Examples of Commands are move and fire. These are received by RAIMIP and processed on behalf of the SAIL AI after a tick notification is pushed to the SAIL AI. In fact, the majority of state obtained by RAIMIP, from a SAIL AI, is the return value obtain from the RPC tick method call.

Obviously I'm working on some concepts I've not previously seen applied to Spring AIs. As far as I know this creates the first extra-process AI implementation. Some concepts still require validation. Some scalability aspects my require additional turning and optimization. Regardless, the above glosses over many technical details and is by no means attempts to be an in depth technical overview.

I still have lots of work to do. RAIMIP is roughly 85% complete and is fairly functional. For example, Unit Interests are supported but Map Interests are currently absent. The SAIL framework proper is barely started. I have roughly 15% completed there, if that. Right now, most of what SAIL does is the RPC abstraction and integration with RAIMI to facilitate easy communication with RAIMIP. It does, however, provide a base AI (SAILAI) from which higher level AIs should be derived.

As for Swarm, it too has very little completed. Just the same, I have been able to do things like initiate hit and runs of multiple coms against another com. Obviously not a functional AI (lacks building, etc) but goes a long way toward test driving all the layers below it as well as validating the concept.

Let's hope this all sees the light of day! :-)
Last edited by echoone on 05 Jul 2010, 06:45, edited 2 times in total.
User avatar
MidKnight
Posts: 2652
Joined: 10 Sep 2008, 03:11

Re: SAIL - A Python AI Framework

Post by MidKnight »

I'm no AI dev, but all that seems pretty awesome!
User avatar
KaiserJ
Community Representative
Posts: 3113
Joined: 08 Sep 2008, 22:59

Re: SAIL - A Python AI Framework

Post by KaiserJ »

ambitious, creative, and useful!
SAIL - Spring AI Library
Aims to provide a multitude of benefits to would be AI developers by allowing them to concentrate on their AI implementation and behaviors rather than the multitude of common framework which is often required before development can even begin.
i like the idea that you're creating a framework to make it easier for other AI programmers, this seems like a great idea to me. like mk im no AI programmer, but im familiar with the concept of simplifying and saving time!
echoone
AI Developer
Posts: 150
Joined: 16 Nov 2009, 18:26

Re: SAIL - A Python AI Framework

Post by echoone »

Thanks for the positive feedback guys!

Thought I would provide an update.

I just finished recovering from total data loss of this project. Python's pip utility, the supposed replacement for distutils (easy_install), which decided to most unexpectedly, and on its own accord, to delete an entire directory hierarchy. That directory contained my project source, among many other things. I detected the data loss immediately before starting a backup. I was forced to recover by pulling from the raw block device. Tedious to say the least. I now have the most important tidbits recovered and am once again moving forward.

As a side note, I emailed Ian Bicking about the catastrophic loss of data and have yet to see a reply. I assume this means he's decided to ignore his most grievous bug in pip. I strongly urge you not to use pip until Ian has both acknowledged the bug and fixed it. Since pip is designed to strongly support virtual python environments, I *assume* the bug stems from some horrible assumption which isn't true when working with code outside of a virtual python environment; as was the case for me. Clearly that's just an assumption on my part.

As for SAIL in general, I now have many more Interests implemented and supported by the proxy (RAIMIP). I also have some basic map classes implemented. I have the basis for some basic pack controllers implemented (AI planners). Also included is a complete, robust, and fast, non-recursive shared (reader/writer) lock implementation. Also is provided is a threading implementation which largely re-uses from python's threading and/or a third party threading2 package; where available. The threading2 package is preferred but not required.

One nice perk of using RAIMIP/SAIL is, since RAIMIP directly implements a tick scheduler, you should no longer see lots of ugly code in AIs which looks like:

Code: Select all

if frame%x == 0:
  do_some_stuff()

elif frame%y == 0:
  do_some_other_stuff()
This is because you have control over both how often ticks are reported to the AI as well as how frequently Interests are evaluated. This means if your AI is notified of an Interest or tick, its because you've already pre-qualified its meets your periodic tick criteria. This means code is more terse, easier to read, easier to maintain, and easier to follow. Hopefully that also translates into fewer bugs, faster AIs, as well as faster development. Yah!

As for map support, I have both a generic and several specific map types implemented. Derived from the generic map is a threat map, los map, and slope map. More map types are coming; including height maps. Jammer and radar maps of types of los maps. Likewise for unit category specific threats (water, ground, air), a unique instance of a threat map is required for each and updated accordingly. Also started are optional map analyzers which allow for easy to use and implement, cpu intensive map analysis, in a thread safe, background thread. These are important for people who wish to do their own pathing and map/path analysis; especially with deformable maps enabled.

I also figured how to allow for easy, dynamic visual debugging, requiring little participation on the AI's behalf. This basically goes the same direction as Interests. I've not implemented anything yet but expect to do so as I continue to work on my unit implementations.

Little has changed on my AI, Swarm. It continues to be used as a testbed driver.

Just over the horizon are additional controllers for economy and construction as well as higher level controllers for overall strategies. These higher level controllers would be responsible for making use of economy, construction, and even packs. Whereby Packs (a collection of units sharing a common goal) are also an implemented controller.

So for example, using a Pack controller, an AI might implement scouting, construction, raiding, defense, etc, as various packs types (derived classes)., which move, attack, defend, and react collectively. Effective use of Packs can also save considerable CPU and improve AI scalability.

Right now the only dependencies are Pyro and NumPy. The threading2 package is not required and is therefore not a dependency, but is effectively leveraged when available. This is also true for Pyro, thanks to my accepted, yet seemingly unattributed patch.

All development is currently taking place on python 2.6 but I fully expect it to be compatible with Python 2.5 and 2.7 (recently released). Support for python 2.4 will likely require some minor code enhancements. But frankly, supporting 2.4 is not at all on my objectives.

As you can tell, I'm pretty thrilled with development so far. IMO, things are progressing nicely. Unfortunately, SAIL has yet to reach an end user usability milestone.

Lastly, I do want to clear the air. In discussion with others, there seems to be some confusion as to whom RAIMIP/SAIL/Swarm is targeting as a user base. The answer is, RAIMIP/SAIL is targeting both casual and hardcore AI developers. Also, python is an excellent prototyping language, so its means rapid prototyping and evaluation becomes easily accessible without a large time investment, even if python is not your primary or preferred development language.

Likewise, once Swarm is able to become more than just a test driver, it is intended both modders and casual users will be able to tweak this AI to suit their needs and/or better target specific mods. So in a nut shell, I'm targeting the broadest possible base; developers, modders, and more technically inclined users.

As more progress is made I'll be sure to provide updates.
User avatar
romulous
Posts: 39
Joined: 31 Jan 2010, 12:36

Re: SAIL - A Python AI Framework

Post by romulous »

This sounds interesting. Are you still working on this project ? I've taken a look at the java interface, but don't know enough about java at this time to do much with it and probly don't have the time either. Your project sounds like a nice way to learn python too. 8)
echoone
AI Developer
Posts: 150
Joined: 16 Nov 2009, 18:26

Re: SAIL - A Python AI Framework

Post by echoone »

I wish I had something ready to share. My framework still has a long ways to go, including concepts yet to be validated. It continues to be actively developed but its also fairly ambitious.

In the meantime, if you want to try out python, just use the python interface. The interface does not depend on my project at all. In fact, its the exact opposite. My project depends on the python interface. The python interface is largely a python wrapping on the C interface.

If you decide you want to give the python interface a try, let me know and I'll happily share some of the issues and workarounds I've encountered. Not that there's many, but there are some, if for no other reason than the interface is brand new.

Besides, I'm sure the python interface developers would love to have another project making use of their efforts.
User avatar
SpliFF
Posts: 1224
Joined: 28 Jul 2008, 06:51

Re: SAIL - A Python AI Framework

Post by SpliFF »

If I wasn't already comfortable with Lua I might have considered the python interface however it seems inferior to Lua in some key areas:

1.) Python is heavy, I don't see my code ever needing a HTTP server, email mime interface or 90% of what Python provides in the main distribution. By contrast everything you need for a Lua AI is already in the Spring engine and therefore requires no end-user dependencies.

2.) The Lua interface has access to OpenGL and drawing primitives. This allows me some key abilities like being able to render AI feedback on the map and implement GUI controls easily.

3.) Performance. A Lua widget is almost guaranteed to be faster at standard computation. Although NumPy does help in this regard I've never really been impressed by python's ability to handle maths in an efficient manner and was actually quite horrified to discover how impractical bitfield and enum operations are.

No intention here to start a religious war. I actually use python a lot in my day-to-day OS and Web scripting and prototyping. Naturally it's a personal preference thing. It's just that python isn't my first preference for game scripting.

Anyway, your framework sounds great. Look forward to seeing it in action.
echoone
AI Developer
Posts: 150
Joined: 16 Nov 2009, 18:26

Re: SAIL - A Python AI Framework

Post by echoone »

SpliFF wrote:If I wasn't already comfortable with Lua I might have considered the python interface however it seems inferior to Lua in some key areas:

1.) Python is heavy, I don't see my code ever needing a HTTP server, email mime interface or 90% of what Python provides in the main distribution. By contrast everything you need for a Lua AI is already in the Spring engine and therefore requires no end-user dependencies.
That means Python requires no end-user dependencies too. Really, LUA follows the good enough mantra. And there is absolutely nothing wrong with that. That's why its so popular for games like this. Its designed from the ground up to be a powerful, sand boxed, embedded scripting language. Python is not. But then again, what you get in return for, "not", it TONS!

But frankly, python has tons of use outside of HTTP and email. Suggesting otherwise is silly. Its designed to be a general purpose computing language with excellent OO capabilities. Most agree it succeeds. There is a good reason why python continues to gain ground in scientific, physics, automation control, and even AI circles. And here's a hint, it has nothing to do with HTTP and email. The fact it has broad adoption in all of these areas, including HTTP and email, is a testament to its general purpose capabilities.

In my own case, python's excellent capabilities, which include networking and network services, and rich third party libraries, are exactly why I'm able to entertain such an advanced, high level framework and design and do so at such a rapid pace. Of course, that also likely explains why the other AIs are written to such a low level.

When python's primary complaint is its white space syntax, I believe that suggests it gets far, far more right than wrong. And frankly, outside of games or niches, LUA isn't exactly on fire. But to be clear, popularity shouldn't, in of itself, be a major factor in my opinion; especially if third party libraries are of no interest or benefit. But for the inverse, consideration of broader adoption should be understood and considered.
SpliFF wrote: 2.) The Lua interface has access to OpenGL and drawing primitives. This allows me some key abilities like being able to render AI feedback on the map and implement GUI controls easily.
Don't confuse hasn't with can't. Python can easily do that too. It just needs the bindings. PyOpenGL has existed for a long, long time now too. Not to mention, the visual debugging and drawing facilities are already exposed in the python interface; though I've not tested them yet.

From what I gather, LUA is one of the oldest and more mature scripting interfaces available for Spring - though the AI use is perhaps fairly new. Correct me as needed. On the other hand, the Python interface is band new. The recent Spring engine release is the first release the python interface is available. Of course, that may be something someone wants to weigh when evaluating their interface of choice.
SpliFF wrote: 3.) Performance. A Lua widget is almost guaranteed to be faster at standard computation. Although NumPy does help in this regard I've never really been impressed by python's ability to handle maths in an efficient manner and was actually quite horrified to discover how impractical bitfield and enum operations are.
Why do you believe that? Every benchmark I've seen (been a while since I've looked - though a half-assed look seems to indicate its still true), python was MUCH faster. It takes the LUA's JIT to become competitive with Python and that's before python starts using tools like numpy or any number of other advanced binding tools for non-native code. And that completely ignores the use of psyco (a pseudo 32-bit python jit), which, while highly code dependent, has shown performance gains well into the hundreds of percents.

From what I know of it, if performance is of considerable weight for your project and your choice boils down to python vs lua, python is the choice to be made. But if you're really performance critical, I'd be wondering why the C/C++ interfaces had been excluded.
SpliFF wrote: No intention here to start a religious war. I actually use python a lot in my day-to-day OS and Web scripting and prototyping. Naturally it's a personal preference thing. It's just that python isn't my first preference for game scripting.
Just the same, Python is commonly used for game scripting and graphics/rendering automation.

Don't get me wrong, I absolutely don't have a problem with people using interfaces other than python, but let's not attempt to create false dogma here.
SpliFF wrote: Anyway, your framework sounds great. Look forward to seeing it in action.
Thanks! I can assure you I'm very much interested in seeing it too! :-) :lol:
User avatar
hoijui
Former Engine Dev
Posts: 4344
Joined: 22 Sep 2007, 09:51

Re: SAIL - A Python AI Framework

Post by hoijui »

the justification of all the different languages supported in the AI interface are there to attract devs that want to use their favourite language (and possibly libs and stuff from there). otherwise, i guess Lua would be better suited indeed.
User avatar
SpliFF
Posts: 1224
Joined: 28 Jul 2008, 06:51

Re: SAIL - A Python AI Framework

Post by SpliFF »

echoone wrote:That means Python requires no end-user dependencies too.
I think you're missing the substance of this complaint. The python binding requires the majority of end-users to download and install python. This is not necessary for a Lua AI as Lua is embedded in Spring.
echoone wrote:But frankly, python has tons of use outside of HTTP and email. Suggesting otherwise is silly.
I never said otherwise, in fact I'm agreeing with this viewpoint. The substance of the complaint is that I think pythons bundled libraries are just pure overkill for the purpose of a Spring AI. It's like buying a mansion for your dog.
SpliFF wrote:The Lua interface has access to OpenGL and drawing primitives.
echoone wrote: Don't confuse hasn't with can't. Python can easily do that too. It just needs the bindings.
It isn't that simple. The Lua GL interface is more than simple OpenGL calls. It involves a system of GL proxies (for multithreading) and specific instructions for rendering to unit or map coords. Not to mention a fairly complete interface to manage shaders.
echoone wrote:python was MUCH faster.
I simply don't accept that. It just isn't consistent with any benchmarks I've seen or my own experiences with both languages.

Anyway. I reiterate. Both languages have strengths and weaknesses. Python's strength has always been its flexibility, not performance or compactness. It's the last two traits I look for in an AI.

Don't think for a minute I'm advocating against your library or python bindings in general. I'm just saying I like Lua better and lets leave it at that.
thegeorge
Posts: 7
Joined: 06 May 2010, 11:22

Re: SAIL - A Python AI Framework

Post by thegeorge »

Writing an AI mostly is more about implementing good concepts and not speed or compactness. I know no other language in which you can write code so fast and effective.

The compare between Python and LUA lacks anyways and this is maybe the wrong thread to discuss this.
echoone
AI Developer
Posts: 150
Joined: 16 Nov 2009, 18:26

Re: SAIL - A Python AI Framework

Post by echoone »

SpliFF wrote:
echoone wrote:That means Python requires no end-user dependencies too.
I think you're missing the substance of this complaint. The python binding requires the majority of end-users to download and install python. This is not necessary for a Lua AI as Lua is embedded in Spring.
I do agree non-Windows users will be required to install Python. I certainly don't argue that. Something that is embedded versus, "has support for", are certainly two different levels of inclusion. Its a decision the developer's made and I can't say I disagree with it. If on the other hand, python becomes a lot more popular for spring developers, then it would be reasonable to re-evaluate the current position. But that's certainly not where we're at now.
SpliFF wrote:The Lua interface has access to OpenGL and drawing primitives.
echoone wrote: Don't confuse hasn't with can't. Python can easily do that too. It just needs the bindings.
It isn't that simple. The Lua GL interface is more than simple OpenGL calls. It involves a system of GL proxies (for multithreading) and specific instructions for rendering to unit or map coords. Not to mention a fairly complete interface to manage shaders.
Well, we're saying the same thing. My point was, just because the bindings have not been created doesn't mean its not possible. My reference to the opengl bindings were to point out opengl bindings in of themselves already exist. We're saying the same thing.

And to be clear, the debugging/visualization interfaces are exposed in the python bindings.
SpliFF wrote:
echoone wrote:python was MUCH faster.
I simply don't accept that. It just isn't consistent with any benchmarks I've seen or my own experiences with both languages.

Anyway. I reiterate. Both languages have strengths and weaknesses. Python's strength has always been its flexibility, not performance or compactness. It's the last two traits I look for in an AI.
Actually, python's strengths has almost always been performance over that of LUA. The Lua guys have been actively working to close the performance gap over the last several years.

As "scripting languages" (which is fairly blurred these days) go, python is pretty speedy. Its "lacking performance" critique has been from a C/C++ and even Java users - not from LUA users or generally any other scripting language camp. In fact, your performance complaint as a LUA user is the first I've EVER read which paints python in negative light. And frankly, I'm unaware of any legitimate benchmark suit which constituently shows LUA will out perform Python. Never even heard it hinted at until now.

Not to say I doubt a micro benchmark can be created for either language which is better for one than the other; which, I suspect is what is giving you, what is widely held as a false impression. Besides, its far more likely developer coding style, implementation details, and language knowledge, will have a larger overall performance impact than the language, when contrasting Python versus Lua.

But frankly, as I originally stated, if critical performance is in the discussion here, its seriously doubtful Python, let alone Lua, should be at the top of the discussion. In turn this really all boils down to, "...but windows users have to also install an extra." I can certainly live with that.

:?: Do you have a micro benchmark which favors Lua over Python? I'm just curious what type of difference and in what use case we're looking at here given your comments.
User avatar
SpliFF
Posts: 1224
Joined: 28 Jul 2008, 06:51

Re: SAIL - A Python AI Framework

Post by SpliFF »

Enough already. I've already said I'm not against the python bindings or your decision to use a language that suits your skills and requirements. You seem to be derailing your own thread to argue against things I didn't say, things that don't matter, and things we both agree on. Try to be less defensive. I'm as big a fan of Python as you are. I've been programming in Python for over 10 years and Lua for 5. I have written commercial code in both. I've worked on the Spring source code. I can program in 15 languages. If that doesn't entitle me to an opinion then nothing does but nobody is forcing you to agree. Just let it go so this thread can return to the topic at hand - which is your AI framework.

Anyway, since you asked:
http://www.timestretch.com/FractalBenchmark.html
http://shootout.alioth.debian.org/u32/w ... python3=on
User avatar
aegis
Posts: 2456
Joined: 11 Jul 2007, 17:47

Re: SAIL - A Python AI Framework

Post by aegis »

there's no guarantee any of those algorithms are optimized in either language

but yes a good implementation of lua's register vm can potentially be faster than python's stack vm disregarding JIT available for both cases
echoone
AI Developer
Posts: 150
Joined: 16 Nov 2009, 18:26

Re: SAIL - A Python AI Framework

Post by echoone »

SpliFF wrote:Enough already.
I'm fully aware there are benchmarks on the web. I asked if you had some since you indicated you had a huge body of experience which indicated otherwise. I'll take your response to mean no.

I don't have a problem with your opinion, or you having one. My point of contention was your assertion which is contrary to the generally accepted facts.
kremmy
Posts: 26
Joined: 12 Jun 2010, 00:15

Re: SAIL - A Python AI Framework

Post by kremmy »

SpliFF wrote:
echoone wrote:That means Python requires no end-user dependencies too.
I think you're missing the substance of this complaint. The python binding requires the majority of end-users to download and install python. This is not necessary for a Lua AI as Lua is embedded in Spring.
Am I incorrect in my understanding that Spring now includes the Python runtime and Python AI interface? I think this was the only relevant point of contention between you guys and it didn't seem to really be addressed.
User avatar
SpliFF
Posts: 1224
Joined: 28 Jul 2008, 06:51

Re: SAIL - A Python AI Framework

Post by SpliFF »

Spring includes python bindings, it doesn't include python itself. Python needs to be installed by the user. This generally only affects Windows users as most linux distributions include a 2.x series python by default.
User avatar
aegis
Posts: 2456
Joined: 11 Jul 2007, 17:47

Re: SAIL - A Python AI Framework

Post by aegis »

and all osx
echoone
AI Developer
Posts: 150
Joined: 16 Nov 2009, 18:26

Re: SAIL - A Python AI Framework

Post by echoone »

Offer a quick follow up here. Development continues...

The pace of development has slowed a bit but I have been taking time to look at performance and other design aspects to ensure my efforts continue to show promise.

From a performance perspective, I have identified several components of the python interface which may create obstacles in the long run. From a scalability perspective, things look incredible. The SAIL AI, running in its own process, rarely even shows in the top list of CPU users.

Potential improvements to the python interface include:
  • Releasing the interpreter lock during callbacks.
    Starting the python interpretor in its own thread.
    Exposing vital data structures rather than requiring getting/setters.
Releasing the interpreter lock means pure python threads can continue to run while other threads are calling into the python AI interface. This increases concurrency; especially while getting/setter interface is required.

This lacking combination requires a fair amount of base AI overhead. By running the interpreter in its own thread, it will help reduce part of the "bursty" nature of the current interface. This is because the AI can continue to run concurrently with engine thusly preventing the need for the AI to perform all of its logic within the confines of a call into the python AI. Furthermore, it means network (or other I/O) activity can take place in the background, outside of the confines of a call into the interpretor. This can be safely supported by adding a single mutex to the interface, which would prevent AI callbacks into the interface when the AI/Interface is not safe to use.

Directly exposing vital data structures, such as unitdef, weapondef, unit, etc., can reduce attribute overhead, effectively reducing AI overhead.

And work continues...
echoone
AI Developer
Posts: 150
Joined: 16 Nov 2009, 18:26

Re: SAIL - A Python AI Framework

Post by echoone »

Guess I'll add a couple more things. If you've read may other AIs, you'll constantly see scaling becoming an issue. Lots of code looks like: "x = y/8", and, "blah = x*16", etc. Even worse is when two different scales are used for two different abstracts which then require comparison and you see a string of multiplies and divides by various scales. SAIL has a Mapper abstraction, which in turn contains various map abstractions. Using this interface, each map and the mapper completely abstracts away the scale arithmetic, so you no longer have to deal with scale at all. Things just work properly.

The mapper abstraction also eases complexity of using various maps. Furthermore, interfaces which require access to a specific map are simply passed the mapper instance. This is one less thing developers need to focus on.

Concurrent map analysis development continues...want some rational to determine land, sea, air, or a combination? This abstraction will help with this in addition to a number of other map domain problems, such as metal location and prioritization, scouting, etc. Allowing for both asynchronous and synchronous evaluation.

An A* pathfinder is available within SAIL. Its very, very slow compared to the engine's A* implementation. Just the same, it does have some uses where the distances are short. Advantages of using this path finder is that it can account for heat maps and threat maps, as well as terrain and debris. In comparison to other python A* implementations, it is very fast. I did some performance comparisons against other readily available python implementations and my implementation is anywhere from equal to 1000x faster depending on a number of variables and the distance of the path involved. Yet, as fast as it is, it can still be several orders of magnitude slower than the engine's implementation. That should be a real eye opener of just how slow most python A* implementations are in comparison to a native code. Its ideal use is for leader/pack handling and construction of defenses whereby only short distances are required from construction site to construction site. Short distances can be calculated in ms to sub ms durations. There is never a requirement to use this implementation and it should always be considered optional. Additional work will likely follow here to allow for fix time constant and abortion due to expiration to allow for bounded operation and orderly fall back.

Memoization is now in place for various calls. This speeds AIs and prevents round trip calls between the AI and RAIMIP. Shared lock implementation is now being used at lower levels to ensure transparent, thread safe, concurrency of AIs and interaction with various other components.

Other tid bits have been worked on...but that will wait for future updates.
Post Reply

Return to “AI”