AI Ladder, 0.5

[hughperkins]

Update to git master:

- merged in doublepackager branch:

-- packages can now be created by 'package.py'
--- this will prompt for the version, and create two packages 'packages/botrunner_<version>.tar.bz2' and 'packages/website_<version>.tar.bz2, and upload them to a configured website

-- end-user website setup is now from commandline tool 'setupwebsite.py'
-- setupwebsite.py can do the following:
--- install virtualenv (included in website package)
--- install sqlalchemy (included in website package)
--- install jinja2 (included in website package)
--- redirect the #! line of each python script to point to the virtual env (won't do much on Windows unfortunately, so Windows peeps will have to deal with this themselves for now, or choose 'global installation' when prompted)

-- version.py is no longer included in master branch, but will only be created in special release branches (0.5,0.6, ...), by package.py. At least that is the intention, since it should reduce merge conflicts

[hughperkins]

Right, so do you want to add some botrunners, and then I can look at writing some scheduling algorithms

[aegis]

we've got to fix springheadless before I can add my botrunners, but they're otherwise sitting ready :)

edit:
hoijui had a successful game with latest git! currently experimenting, then I'll put my botrunners online for testing

edit:
things are looking up :)

Code: Select all

Finalizing...
Spring 0.80+.0.0
Build date/time: Oct 20 2009 09:55:33
ShowCursor
Spectator UnnamedPlayer finished loading and is now ingame
<Ghost Site> Spectator mode. Widget removed.
UnnamedPlayer added point: Start 0
UnnamedPlayer added point: Start 1
 -> connection established (given id 0)
Speed set to 3.0 [server]
Speed set to 120.0 [server]
Spectator UnnamedPlayer finished loading and is now ingame
GameID: 2c2fde4ae80ba2ddfa18cbbe79b8f899
SkirmishAI0: land map detected
SkirmishAI0: AAI loaded
SkirmishAI1: AAI loaded
Game started... starting timer.
Skirmish AI Bot1 took over controll of team 0
Skirmish AI Bot2 took over controll of team 1
<DefenseRange> Spectator mode. Widget removed.
Construction Vehicle: Build pos blocked
Team0 (UnnamedPlayer) got their just reward
Skirmish AI Bot1, which controlled team 0 is now dead
Game has ended
Game has ended
<autoquit> Automatically exiting in 12 seconds. Move mouse to cancel.
Game over, realtime: 17 seconds, gametime: 529 seconds

                               Part|        Total Time|Time of the last 0.5s
                           CPU load            16.47s 99.20%
                         Draw world             0.80s  0.00%
                       ExtraTexture             0.02s  0.00%
                    Feature::Update             0.24s  0.20%
                       GroundUpdate             0.04s  0.00%
                              Input             0.01s  0.00%
                     Interface draw             0.01s  0.00%
                                Los             0.73s  1.00%
                         Map damage             0.08s  0.20%
                                PFS             0.32s  0.40%
                         PFS:Update             7.32s 75.45%
              Projectile Collisions             0.21s  0.00%
          ProjectileHandler::Update             0.10s  0.20%
                              Radar             0.03s  0.00%
                            Scripts             0.72s  3.59%
                    Shadows/Reflect             0.01s  0.00%
                           Sim time            13.54s 92.81%
                         SkirmishAI             1.96s  2.00%
               SkirmishAI:AAI-0.9:0             0.86s  0.00%
               SkirmishAI:AAI-0.9:1             0.78s  1.40%
                       Unit handler             4.16s 11.38%
                        Unit loader             0.47s  0.00%
                   Unit slow update             1.37s  4.39%
                              Water             0.04s  0.00%

[aegis]

compiling on windows so I can watch replays... spring.exe compiled but is having trouble watching replays and target headlessstubs threw a bunch of errors and died...

edit: fixed the stubs (tested on both windows and linux)

but...

Code: Select all

Linking CXX executable ..\..\spring-headlessstubs.exe
Error running link command: The parameter is incorrect
mingw32-make[2]: *** [spring-headlessstubs.exe] Error 2
mingw32-make[1]: *** [tools/HeadlessSpring/CMakeFiles/spring-headlessstubs.dir/a
ll] Error 2

edit: holy long pathname - it's apparently impossible to create a process on windows with more than a 32kb string.

[aegis]

ailadderstaging breaks when I add myself as a the owner of my whole cluster

[hughperkins]

edit: fixed the stubs (tested on both windows and linux)

Nice! I'll take a look.

ailadderstaging breaks when I add myself as a the owner of my whole cluster

Ok. Couple of clarifications:
- how did you add yourself as the owner of the cluster?
- what do you mean by 'breaks'? Do you mean the formatting screws up on the viewbotrunners.py page? Like this: http://manageddreams.com/ailadderstagin ... runners.py ?

By the way, created branch 'matchscheduler', to start to look at adding scheduling.

Edit: fixed the display issue with viewbotrunners (the fix is a bit hacky, if you know a better way?). Does this address the issue you were seeing before?

[hughperkins]

edit: fixed the stubs (tested on both windows and linux)

Merged into git, kind of. Lots of changes so hard to be sure without testing it.

[hughperkins]

Merged branch matchscheduler into head.

So.... how this work...

To use:
- run your botrunner once, to register your maps and mods
- a 'league' represents a single combination of one map, one mod, and some options
- login, as admin admin for example
- go to 'configuration'|'view leagues'
- fill in the info for a new league:
-- select a map
-- select a mod (best to use BA7.01, no other mods other than BA are expected to work with botrunner for the moment)
-- select the number of matches you want to run between each pair of AIs (eg '3')
-- click on 'add'

Right, so the matchscheduler will add matches to the request queue until there are enough matches to generate appropriate results for each created league. For now all AIs are included, but we can filter later on, using some kind of options system.

The matchscheduler runs whenever a botrunner makes a request to the website.

So, to fill in the request queue, restart the botrunner, or wait for it to poll the webserver, and then if you go to 'runner' 'view requests', you should see lots of requests, maybe something like http://manageddreams.com/ailadderstagin ... equests.py

Visualization of matchscheduler working data:
- you can see the number of matches either requested or finished, per AI pair, per league, in the page ''Debug' | 'show AI match count pair', which is mostly for debug, so not terribly easy to read for now, though we might maybe merge it into the Runner menu at some point, with some more documentation added perhaps.

This is the base scheduler. Possible future improvements:
- any change to a league drops all matchrequests created for that league, that are not either in progress or finished
-- they will be recreated at next botrunner poll
- make it possible to choose which AIs will participate in each league, either by just adding each AI manually to each league (sounds tedious), or by setting options on each league, and only AIs that match those options can participate (sounds more scalable, and the options system is already pretty much implemented, just not used by the scheduler for now).

[hughperkins]

Quick update: improved the page that shows how many matches requested for each ai pair, for each league, and moved it to under the 'Runner' menu as 'View request counts per league."

Edit: a few more improvements to the page (in git master, and ailadderstaging)

Edit2: shows number matches finished now (in git master, and ailadderstaging)

[hoijui]

i don't know exactly what, but you did some ugly git with springheadless.
what you should have done:
git fetch
.. ahh there are changes done by others

in case you have uncommited local changes, do either of these:
* commit them
* git stash (this puts your changes into a kind of temporary cache)

now get the changes of the others into your local branch:
git rebase origin/master

in case you used git stash before:
git stash apply (applies the changes in the temporary cache back to your working directory)

now start applying your changes. when you are done with them, committed, you try to push them, and rebase again if there were changes of others in the meantime (then rebase would have aborted).

a rule of thumb: only merge different branches, do not merge eg master and origin/master, use rebase there.

[hughperkins]

i don't know exactly what, but you did some ugly git with springheadless.
what you should have done:
git fetch
.. ahh there are changes done by others

in case you have uncommited local changes, do either of these:
* commit them
* git stash (this puts your changes into a kind of temporary cache)

now get the changes of the others into your local branch:
git rebase origin/master

in case you used git stash before:
git stash apply (applies the changes in the temporary cache back to your working directory)

now start applying your changes. when you are done with them, committed, you try to push them, and rebase again if there were changes of others in the meantime (then rebase would have aborted).

a rule of thumb: only merge different branches, do not merge eg master and origin/master, use rebase there.

Thanks! I agree it didn't really work out for me before. Should I somehow undo what I did, and redo it the way you just said?

[hoijui]

yeah.. if you think you can, that would be good :D
if you are done, you would have to force push to origin in the end.

edit:
if you need help, there surely are some git experts in the lobby

[hughperkins]

Is that better?

What I did:
- git clone springheadless springheadless-fixing
- changed into springheadless-fixing
- gitk, reverted back to the last change just before the merge (your glstubs fix)
- meld'd again from springheadless into springheadless-fixing...
- redid the commit
- git rm origin, git add origin git@...
- git push -f origin

(then back in springheadless, I did "git fetch", "git reset --hard origin/master" -> and gitk seems to show something reasonable now).

[hoijui]

perfect!
thanks!

[hughperkins]

perfect!
thanks!

Great!

[aegis]

instead of making this out to specifically be a ladder, could we instead focus on doing a really good job of making it an advanced automated spring headless launcher including support for AI, then add nice statistics to rank the AIs?

last time I checked, our launching options could've been better... I think maybe these should be the main initial focus... possibly including a nice preset system, the ability to redo runs, an easy way to run replays for crash tests, sync testing across multiple bots...

[casperjeff]

Question: I've been reading the thread...what is the current state of the spring headless option? I would love to test my own bot in a headless environment at high speed over multiple iterations to maybe try out some genetic algorithms to tweak a variety of parameters....

(the ladder system doesn't interest me as much as being able to run headless on my own box)

[aegis]

you on windows or linux?

[casperjeff]

Primarily windows...not afraid of linux though

[Error323]

Question: I've been reading the thread...what is the current state of the spring headless option? I would love to test my own bot in a headless environment at high speed over multiple iterations to maybe try out some genetic algorithms to tweak a variety of parameters....

(the ladder system doesn't interest me as much as being able to run headless on my own box)

What parameters would you like to learn using GA's? Just curious.