AI Ladder, 0.5

[Achilla]

Maps should be many, and varied. That way you can divide them into subgroups and list e.g. which AI copes best with pathfinding, which with early land rush/skirmish on flat terrain, which on longer-run ground assault, which on teching, which on making most efficient army composition, which is best on the sea etc.

Not to even mention different mods being used. I would suggest BA, CA, XTA and NOTA, if you think more serious about it.

[Gota]

Since we dont even have a ladder for human players,are we not overextending ourselves here?

[Masure]

And what's the difference between a player ladder and an AI ladder assuming AI's name is AI's name + version ?

A ladder is a ladder

[hoijui]

The difference is, that spring AIs wont come together by themselfs to play a game against each other.



... not YET!

[AF]

Enough discussion, if someone can devise a mechanism for identifying which AIs lost and the AIs that won, huge major roadblock there.

*grows impatient*

[Masure]

The difference is, that spring AIs wont come together by themselfs to play a game against each other.

That's not a ladder matter to make AIs play against each other. Ladder manages payers' win/loss.

[hoijui]

you get the team killed messages, with the player/AI names in infolog. i dont know if it corelates 100% with whether the team won or died.

[hughperkins]

Enough discussion, if someone can devise a mechanism for identifying which AIs lost and the AIs that won, huge major roadblock there.

Random thought: maybe have a third 'AI' participate in the game, but either with his one commander way off the map, or with no units at all (not sure if stays as a spectator if has no units?).

The third 'AI' is not actually meaned to be an AI, but just a monitoring routine.

It monitors the game, for example, number of units, resource production, for each player, and writes these out to a file.

When the game ends, one can read the number of units and resource production of each team from the file. Whichever team has the highest is assumed to have won.

[hoijui]

Skirmish AIs get killed/destrpyed/garbage-collected when their team dies. eg, they can not be spectators. But what you want can be done with a widget very easily, so good idea!

thing is, i dunnu how to write widgets.. it should be relatively easy, and it would be good to have some AI related widgets anyway.. would be nice if we had a person that could somewhat dedicate himself to this task. :D

Besides the widget described above, i am thinking of a GUI that makes the AI commands available through buttons and drop-down menus and such. Then maybe one that prints some AI specific performance relevant stuff (though i do not know how easy it would be to get that info for a widget).

about global learning:
we alreayd discussed it one or two times. it has not been done for a spring AI so far. the main problem is, to make learning really usefull, you will usually need a lot of training data/games. And in practice you will have a hard time ot get that. The psring community is not big, only few of them play with AIs, and your training datas validity will probably be limmited in time through events like new spring versions or changes to the training data structure of your AI.
If you take all this into account though (make training valuable even after little training games, very simple training data structure that you will not ever have to alter), it could be cool! :D
good luck with that!
... ahh yeah, maybe... if the format is really simple, then it could probably be used & collected by other AIs as well, which would be even better! :D

[hughperkins]

Besides the widget described above, i am thinking of a GUI that makes the AI commands available through buttons and drop-down menus and such. Then maybe one that prints some AI specific performance relevant stuff (though i do not know how easy it would be to get that info for a widget).

Yes! I was thinking the same thing. Like, I have my Swing gui, but it only works in windowed mode, whereas some sort of a widget that an AI can communicate with would work in any mode, and not have such difficulties.

At a basic level, such a widget could survive with just labels and buttons, though it'd be nice to be able to input text too. Trying to keep it simple here so it's doable.

So, if somehow an ai could do something like:

Code: Select all

aicallback.getGui().setText( "my text\na new line\nanother line" );

... as an initial proof-of-concept that could be wicked.

The psring community is not big, only few of them play with AIs, and your training datas validity will probably be limmited in time through events like new spring versions or changes to the training data structure of your AI.

Yes that's true. So, something like the automated AI ladder could be good for that. It could be interesting to just play a couplle of AIs together automatically for a while and watch them change!

That said, when I play against AAI, I notice that it does seem stronger after a few games, so it's not like it's totally impossible for an AI to learn from games 'in the wild', though it would have to learn pretty quickly, like AAI seems to.

[DJ]

I like this idea from AI playground point of view but as I just discovered just because an AI beats other AI's doesn't mean its fun to play against. I'm building an AI at the moment and have been testing it against KAI. It can win against KAI but when I played against it myself it was really boring.

Not saying no to the idea just that we shouldn't forget that the AI's for spring are a good way to get players to use the engines games and we shouldn't neglect that side and just start competing with other AI's

[imbaczek]

you can make an assumption that if an AI beats other AIs (note plular) on different maps, it's versatile enough to be interesting to play against for a human.

[hughperkins]

You mean because the AI is fast and hard to beat?

We can change that by using the difficulty setting that AIs can make available.

Or you mean because it's too trivial to do something that makes the AI lose, like say build some really strong defense right in the AI's path, and let it throw itself against that defense futilely?

In the second case, in the specific we can defeat that using a threatmap. In the generic, well, as long as an AI could do something similar to another AI, then warring AIs might do this to each other.

Personally I find playing against my AI annoying because it is so fast, and I just want to chill out and build some defense and let it throw itself against my defnese convincingly, whilst I tech up, and then I can destroy it with my amazingly powerful tech. At least, that's how I play against most AIs that come with actual games.

However, that could be partly mitigated by using the difficulty setting. For example the difficulty setting could be used so that the AI forces itself to waste a certain percentage of metal mined, stuff like that.

Or, the AI could make units wait for 10 seconds between orders, ie somethig like:
- game starts
- wait 10 seconds
- commander starts to build metal extractor
- commander finishes building metal extractor
- wait 10 seconds
- commander starts to build solar panel
- etc...

obviously, the 10 seconds would be configurable according to the difficulty level. Like, say 60 seconds for the easiest level, and then maybe 30, 10, 3, 1, or 0 seconds?

For improving play against humans... maybe it might be worth creating a rating system that lets users rate an AI for certain factors that we might feel worthwhile? "What gets measured gets done". Maybe something like:
- how easy was it to set an appropriate difficulty level? (rating 0-5)
- how fun was this to play against? (rating 0-5)
- what single question do you wish you had been asked to rate? (freeform text string)

[DJ]

There were two things that made my AI dull to play against.

1) I have tried to avoid using map hack as a response to the forum posts received on the topic. This is all very well but in a BA game a Heavy laser tower backed up by a couple of LLT's was enough to secure a choke point pretty successfully. KAI is capable of analysing whether it can defeat an emplacement of this sort and will not engage if it can't. My AI simply continued to blunder units into it in a very un-organised way - something I hope to address today.

2) Static defences; this is just my opinion but I think in a battle of AI vs AI the AI that gets the upper hand first wins, basically if you attack the other AI so that it has to pump resources into defending itself it can never expand as much as it needs to and loses. Very few AI v AI battles get to Level 2. This basically means no static defences - if you get on the offensive quick enough in AI v AI you don't need them. However from a users point of view the fun when playing against someone is breaking down a well defended area of territory. When you get the upper hand against my AI you can pretty much just waltz into any territory you want.

--edit--

code is here http://code.google.com/p/spring-djai/ if anyone is interested. It's nowhere near ready for release yet and is mostly uncommented - Hugh I've used your port of Krogoths mex finder, I will add that to the comments.

[hoijui]

Code: Select all

aicallback.getGui().setText( "my text\na new line\nanother line" );

Unfortunately, it will not work like this. Or better said, this would need a refactor of spring and more importantly, it would require all GUI libs to adhere to a common interface. Thilw this would be good of course, it wont happen in practice.
As said, i know little about Spring Lua(-GUIs), and maybe hte above is even not true, but the way i persieve it:
There are many different Lua GUI libs, and most mods use a different one. I think htere is a default one from the engine, BA is now using LolUI, CA has its own GUI lib, ...
that is kind of a problem for us. Lets say we code an AI for BA, and create a GUI widget wiht LolUI. Then we want to make our AI to play CA as well, and to not force the user to load LolUI just for our AI widget, we would have to write a version of it for CAs GUI lib.
... it would be good if someone really knowing about this stuff could comment on it :D

In practise, we will probably see something like:

Code: Select all

aicallback.sendLuaMsg("xAIwidget:actualizeGUI", {"myName", "myVersion", "Hello World from abcAI"}, {myTeam, 11, -33}, {, 0.1f, 0.2f, 0.3f, 1.4f});

this is a proposal, and there would be a similar function for widgets to send a message to AIs:

Code: Select all

sendMsgToAI(aiTeamId, messageTopic, stringParams, intParams, floatParams);

this should be easy to implement, and would suffice for most things, with its genericness.

[AF]

If someone can write said widget and post it up, I can start making leaps and bounds with this.

As for fun, thats a subjective quality, lets handle the hard facts first.

[hughperkins]

code is here http://code.google.com/p/spring-djai/ if anyone is interested. It's nowhere near ready for release yet and is mostly uncommented - Hugh I've used your port of Krogoths mex finder, I will add that to the comments.

Oh, you're writing in Java. Excellent

2) Static defences; this is just my opinion but I think in a battle of AI vs AI the AI that gets the upper hand first wins, basically if you attack the other AI so that it has to pump resources into defending itself it can never expand as much as it needs to and loses. Very few AI v AI battles get to Level 2. This basically means no static defences - if you get on the offensive quick enough in AI v AI you don't need them. However from a users point of view the fun when playing against someone is breaking down a well defended area of territory. When you get the upper hand against my AI you can pretty much just waltz into any territory you want.

Yeah, after reading your earlier comment, I started to think about what I want in an AI that is fun to play against, and I feel that I agree with your analysis that static defenses are fun to play against.

I think that if I wanted to just get annihilated, I might just play against a human.

If I'm playing against a computer, it's because I just want a relaxing game where there is a credible plausible threat, and maybe I won't always win, but I don't want it to be insanely hard and need lightning reactions either...

I feel that maybe a fun AI can be created by imposing two constraints on an AI:

- it's 'territory' is no longer the entire map, but one half of it
- - it will build static defenses around it's territory and all over it

- it will send across small packs of units to attack the player occasionally, to provide a credible threat
- - however, the packs will be of limited size, depending on difficulty, it might be 10 stumpies, or maybe 10 of a mixture of level 2 units, but not 200 brawlers for example

I feel that perhaps with these two constraints self-imposed on an AI, it can be fun, and also possibly this type of AI could be useful in games that are based on missions?

[aegis]

I'd write the (very simple) widget if I had headless spring.
I'd also write a simple web interface, script writer, launcher, bot, etc, and plug in my stack of servers to run it (literal pile of 1u servers).
I'd be able to run (at least) eight-player games with one processor per player... possibly sixteen-player games if cpu usage isn't too bad... or it could be several 1v1/2v2 games simultaneously.

[1v0ry_k1ng]

there reallly isnt much need for a ladder as it is, the released AIs rank as follows:

E323AI
KAI
RAI
AAI

which incidently is also the order of which requires the most memory

when hughAI is released fully configed then there will be leeway for some games, and I dont know if NTAI is likely to make a return in the near future but the playing ability of NTAI is hugely dependant on the quality of its configs, it can potentially beat most other AIs simply because it can be given build orders that it executes perfectly (although last release its combat behavior was admittedly very sloppy)

[Tobi]

Ladder would be epic to be able to track progress and as a means of regression testing.

Just keep old versions around, and have the AI play against it's own older versions too. Then you know you're doing it wrong when your new version loses more then 50% of the time from an older version.