AI Ladder, 0.5

[hughperkins]

Right, a bunch of updates! The theme for these updates is usability, and hopefully they address many of the things that have been requested.

Some of the biggest changes:
- botrunners now register map, mods and ais themselves on the website
- when you first run a botrunner, it will prompt you for a couple of questions, and set up the config.py itself
- botrunners will now register their account themselves on the website
- calcengines are now known as 'botrunners'

The botrunners being able to register the map, mods and ais themselves is in huge part thanks to the unitsync Python module by aegis, which is *awesome* and makes finding out these informations trivial now. Nice!

Other changes:
- botrunners now ping the website regularly (every 5 minutes) with their current status
- the botrunner page highlights in green those botrunners that appear to be online (with a ping more recent than 10 minutes)
- game timeout should be implemented now (not tested, but should result in a draw)
- ai name and version are grouped together in the submit request form
- removed the test data from the static data
- autodetects spring-headlessstubs if present, and asks if you wish to use
- start positions on maps now determined automatically, using unitsync

breaking changes:
- botrunner names are now assumed to be unique. whereas before it was fine for multiple botrunners to share the same name, it no longer is, and will cause multiple machines to process the same match in parallel

compatibility fixes:
- the database ddl scripts in Python were not compatible with some combination of the python 2.3 or the old mysql on dreamhost. This is now fixed.

The changes are available in git, or you can look at the staging website at:

http://manageddreams.com/ailadderstagin ... runners.py

or download from:

http://github.com/hughperkins/ailadder/archives/master

[hoijui]

woo hoo! sounds nice!
have to try...

[hughperkins]

Quick update in git:
- viewbotrunner form now shows maps and mods supported by that botrunner
- botrunners are only provided requests to run matches on maps and mods that they already have installed

[hughperkins]

By the way, since github seems a bit not entirely up all the time at the moment, there is a clone of the ailadder git repository at:

Code: Select all

git clone http://manageddreams.com/ailaddergit

Both repositories are synchronized as at this post, but I think that maybe the public clone on github is inaccessible at the moment?

[hoijui]

maybe "Your" government? :D
.. ouh.. pssst!

i did some changes too for once.
i tested them on my main machine, the headless one and a windows machine.
spring executable and unitsync are now configurable through a full path (have to be configured this way). that adds some flexibility and makes it easier to get right (eg to ensure that spring and unitsync fit to each other. its the same mechanism used in SpringLobby and QTLobby.
with these changes, i was again able to run a game on the headless machine, with two KAIKs. it again timed out, then upload failed. hope it is not cause of my changes, but could be so. should look at the demo file...
when trying the AI Ladder on windows, it failed in unitsync.py at line 47 (where it tries to load the native unitsync library) with the message: "The specified module could not be found" (translated by me from german)
i am using the standard Python 2.6.3 installer for Windows.

nice work with the configuration wizard and no need to register on the site anymore, pus.. yeah i see lots of improvements already, nice!

[hughperkins]

nice work with the configuration wizard and no need to register on the site anymore, pus.. yeah i see lots of improvements already, nice!

Cool! Thanks!

i did some changes too for once.

Cool, too! :-D

spring executable and unitsync are now configurable through a full path (have to be configured this way). that adds some flexibility and makes it easier to get right (eg to ensure that spring and unitsync fit to each other. its the same mechanism used in SpringLobby and QTLobby.

Ok. Is it possible to make it default to something standard, and then we can tweak the config file for specific installations?

it again timed out, then upload failed. hope it is not cause of my changes, but could be so. should look at the demo file...

I have to confess, running a full game on my little eeepc is not that tempting a proposition, so I'm kind of waiting on someone providing a botrunner to test stuff like this I'll have a look though... maybe I can make an AI that simply walks for 5 seconds, then self-destructs, and that will make for a very short game :-D

[AF]

Code: Select all

D:\Spring>C:\Python26\python botrunner.py
Traceback (most recent call last):
  File "botrunner.py", line 46, in <module>
    unitsync = unitsyncpkg.Unitsync(location)
  File "D:\Spring\unitsync\unitsync.py", line 47, in __init__
    elif location.endswith('.dll'): self.unitsync = ctypes.windll.LoadLibrary(lo
cation)
  File "C:\Python26\lib\ctypes\__init__.py", line 431, in LoadLibrary
    return self._dlltype(name)
  File "C:\Python26\lib\ctypes\__init__.py", line 353, in __init__
    self._handle = _dlopen(self._name, mode)
WindowsError: [Error 127] The specified procedure could not be found

D:\Spring>pause
Press any key to continue . . .

[hughperkins]

Hmmm, looks like either an issue with the path, or a ctypes issue.

If someone can give me ssh access to a Windows box, I could take a look, though I suspect aegis will notice the solution much faster than I will be able to.

[AF]

Here under Vista sp1 x64 business it fails loading untisync procedure could not be found

Capture5.JPG (22.12 KiB) Viewed 1357 times

[aegis]

how does botrunner.py choose the unitsync path/filename?

[hughperkins]

Ok, so, in the original 0.2, 0.2b, 0.2c tar.bz2 link I supplied, which is before hoijui's modifications, the path is simply 'unitsync.dll', as in example.py. In the latest git, 0.2d download, the full path is specified.

[hoijui]

yep.. you have to give the config file the full path, and the interactive config creator wizard even checks if the file is available. i am very sure i gave it the correct path. it either fails casue the windll module is not installed in a default installation of phyton, or becuase it fails loading the library. i know very little about phyton, so it hope on you to help with this/fix it. i am around in chat, if oyu want to experiment (i have a win machine).
AF used a version from before my changes, so yeah, it was the path that was not correct for him.

[hughperkins]

Yeah, I got AF to try with the new version, and he got the same error though. I'll try to catch you in chat, but I'm in McD's, and they give you 20 minutes internet, per day, it's not a lot

Right, other stuff, new version, fixes the issue with the replay uploading failing hopefully:

http://manageddreams.com/ailadder/downl ... .5.tar.bz2

I reproduced the issue on my own computer, and the issue that was on mine is fixed. Also, on the off-chance that there are two problems I've added a diagnostics page under 'configuration' which will check that the replays directory exists on the webserver, and is writable, so hopefully the replay uploading will work ok now.

Also, the botrunner.py mini-setup sequence presents the available candidate spring paths in a numerical menu, to save typing, otherwise you can just type the full path if it's not in the list.

[hughperkins]

hoijui and I had a look at the devil loading issue. On hoijui's machine, the issue was a dll-conflict with a different version of devil.dll in system32. We've tweaked unitsync.py to pre-load devil.dll, sdl.dll, and ilu.dll, and that seems to work.

You can get from git or from:

http://manageddreams.com/ailadder/downl ... .6.tar.bz2

[hughperkins]

Note that replay uploads seem broken on Windows botrunners. We have a fix prototyped and we're just testing that before publishing that.

replay uploads from ubuntu botrunners seems to work fine.

[hughperkins]

Right, hoijui and I spent ages looking at an issue with replay uploading on Windows. It looks like it's basically cracked now, at least it seems to work with spring-dummy. So:

New release 0.2.7:

http://manageddreams.com/ailadder/downl ... .7.tar.bz2

[hughperkins]

aside: wow, xmlrpc really easy in python in cgi.

hoijui asked if there was some way of accelerating the map registration, since he has 300 maps :-O

There are basically two ways of doing this:
- register the maps in one go
- download a list of already-registered maps, and only register the new ones

Either is possible in plain xml, but as the number of calls to the server of this type grows, it could be useful to migrate to a more maintainable framework than hand-coded xml packets...

xmlrpc is part of the standard python library.

Example server cgi file:

http://pastebin.com/m30b3c8df

Example client to call it:

http://pastebin.com/m6979dd8f

Output:

Code: Select all

user@701:/home2/user/dev/python/webservicetest$ python testclient.py 
['getlist', 'gettuple', 'hello', 'hellofn', 'system.listMethods', 'system.methodHelp', 'system.methodSignature']
hello world you said: foo
hello world
['foo', 'blah', 'blore']
['foo', 'blah', 'blore']

Pretty sweet I think. It can pass lists and tuples easily. If you register a class, you don't even have to remember to register new functions. It supports method introspection from the client side...

[hughperkins]

(by the way, if someone knows some hibernate-like way of using databases on python, something well-used, not some bit of code someone wrote five years ago and is never used , then that could be really useful!)

[hughperkins]

New release 0.3:

http://manageddreams.com/ailadder/downl ... .3.tar.bz2

- maps, mods and ais are only registered with the website if they're not already marked registered and marked supported for that botrunner
-- this should accelerate running botrunner on systems with hundreds of maps

Under the hood:
- lots of connections from botrunner to website have been migrated to xmlrpc

[aegis]

hibernate?