Skirmish AI default conventions

[hoijui]

This thread is for discussing Skirmish AI cenventions/good practise rules.
They are meant to faciliate life for spring players/users, and faciliate automatic testing of AIs, eg for comparing CPU performance, play strength, memory usage, or simply for automatic bug testing.

This will mostly be about options formats and about data-dir structure.

In the end, this will only work if some AI devs adhere to these conventions, so i encourage them especially, to take part.

Cheating
As a start (extracted from the AI Ladder thread), two proposals:
A common way to allow/disallow the AI to cheat through options (AIOptions.lua):

Code: Select all

	{
		key="cheating",
		name="Supported Cheating Modes",
		desc="What cheats shall be used by this AI",
		type="list",
		def="default",
		section="difficulty",
		items={
			{key="none",      name="None",           desc="Do not use any form of cheating"},
			{key="all",       name="All Supported",  desc="Use all cheats supported by this AI"},
			{key="default",   name="Default",        desc="Use the default set of cheats, as defined by the dev(s) of this AI"},
			{key="events",    name="Cheat-Events",   desc="Perfect intelligence agency: AI knows about events it can not see"},
			{key="resources", name="Resource-Hacks", desc="The AI can give resources to its self, for free"},
			{key="map",       name="Map-Hacks",      desc="AI can always see the whole map"},
		}
	},

AIs would agree on the option key, and use a sub- or super-set of the possible values enlisted here.
Eg, an AI that does not support cheating at all would only have the "none" and "default" keys listed, while others may use exactly this set, or define additional values like "units" -> "the AI may use cheats to randomly give units to itsself".

The benefit here is a uniform image towards the user, and making the AI comparable more easily/accurate.

Logging
a common base structure for log files, relative to the AIs data dir. for RAI, it would look like:

Code: Select all

AI/Skirmish/RAI/0.601/log
AI/Skirmish/RAI/0.601/log/archive

The AI writes all log files into the log dir, and on subsequent runs of the AI, old logs are getting moved to the archive subdir, so only logs of the last game are directly under log.

This would allow lobbies to automatically show the log file when the AI crashed, or the AI Ladder system to pack all logs of an AI for a game into an archive, and make it available for download. the same could be done by an automatic AI bug finder.

[hughperkins]

Hey hoijui,

I was just looking at integrating startscript configs into my AI in a somewhat-generic way, and it occurred to me: could AI global configs themselves not be standardized too?, and managed by the lobby?

So, in the lobby, we can specify for each AI:
- when we create a game, we can specify options per-game,
- but also, for each AI, we can see it's global options, and set those! in the lobby!

Then, AIs no longer have to handle this sh^h^h stuff themselves!

Like, right now, HughAI has a gui tab just to handle modifying the config from the GUI, but that could really be done through the lobby, and save every AI from having to do this stuff?

My own attempt at making a generic config controller for my own AI, so far:

http://pastebin.com/f426c7625

... and then, I couldn't help thinking: all these configs: startscript, global AI config, etc, they're all the same! Maybe hoijui can help :-D

[hughperkins]

Right, I checked for you in irc, but you weren't on, so I have an excuse to post here

Finally got around to getting HughAI to reading the startscript options. Unfortunately I have no way to test the format of the AIOptions.lua file, since there is no AIOptions.lua reader for linux AFAIK?

Two points:

1. One of hte options I'm using is coincidentally 'maphack'. At the moment I have this as a simple on/off. Is there any reason why we need to have all the cheats combined into a single field, which the AI then has to refactor into the actual booleans it will use?

Like, I assume that the cheat options specified in the list are not mutually exclusive but can be combined?

2. Vaguely off-topic to your questoin, but on-topic to the thread subject matter , what is the format in AIOptions.lua for providing a boolean? Do I just do type='boolean'?


FWIW, my current list of options are, approximately, probably doesn't actually compile, but you can see the idea:

Code: Select all

local options = {
        {
                key="maphack",
                name="Activate Maphack",
                desc="does the AI use maphack to find your units?",
                type='list',
                def='yes',
                items='no','yes',
        },
        {
                key="guiactivated",
                name="Activate GUI",
                desc="Activates GUI. Mostly useful for developers.  Make sure to play in windowed mode if this is activated!",
                type='list',
                def='yes',
                items='no','yes',
        },
        {
                key="debug",
                name="Activated debug",
                desc="Shows lots of debug info, lines, ghost units and so on, so you can see what the AI is thinking.  Turn off for normal usage."
                type='list',
                def='no',
                items='no','yes',
        },
        {
                key="defaultworkflowname",
                name="Default workflow name",
                desc="Selects the build workflow to use.  By default there is only the default workflow, but you can create others, and switch between them here.",
                type='list',
                def='default',
                items='default',
        },
}

[hoijui]

i don't know if the format is nicely documented somewhere, but it is the same format that map- and mod-options use.
So what i do when i want to see how it works, i open BA with the archiver, and extract/open the *Options.lua file in there.

I am not sure if i got you right about options beeing specified in the lobby... but that was always the idea, lobbies just did not yet implement it for AIs.
This is also the reason why the format is the same, as (at least) the two main lobbies alreayd have support for mod-options, so they could reuse this code for AI options.
For testing your options file, you could replace the content of the file in BA with yours, and then open a battle with that mod in SL and look at the mod options.
But well.. it is simpe enough to see by eye, if you have a working one for comparison.

about the cheating option in particular:
i agree with you that they each way of cheating should have its own property.
To achieve the same possibilities, the standrd would have to be though, that if all the cheat options are disabled, then the AI really should not do cheating at all. the problem there is, what if an AI has eg. map-hack build in, and it can not be disabled? we had to include an additional property per cheat way in the AIInfo file, eg:

Code: Select all

key    = 'cheat-map',
value  = 'always',
desc   = 'Whether to use map hacks to always know everythign about enemy units; possible values: always, never, choosable(see AIOptions)',

i also propose to have the debug option standarddised.
the problem there is, that some use on off and some have levels.
maybe we can have a double standrd to either use debug[true|false] or debugLevel[debug/info/warn/error/fatal].

[hughperkins]

i don't know if the format is nicely documented somewhere, but it is the same format that map- and mod-options use.
So what i do when i want to see how it works, i open BA with the archiver, and extract/open the *Options.lua file in there.

Ah sweet! Good idea!

[slogic]

The problem of common (standardized) AI options is who should read them actually. Evidently, it is an engine (via AI API interface) who controls cheating & reading standard options, because it gives cheating info to AI libraries. AI should declare which cheats it will use. API functions which provide cheating info should be controlled via access policy depending on options selected by user. Visibility of options are dependent on a registered cheats by AI lib.
Otherwise, AI maker can expose cheating options in .lua file but actually ignore them, thus we come to a situation when a player considers AI is not cheating at all which is a lie actually. Such user could mistakenly admire by AI "intelligence".

What do think?

[hoijui]

no go for stasi tactics, this is open source development

[Petah]

Using the AIOptions.lua has it pitfalls. As far as I know (IÔÇÖm probably wrong) the AI cant access things like the 'type' and 'description' fields without directly reading the file. BAI uses its own OptionManager class that can load and save options to either XML or a binary file at present (Lua could possibly be easily implemented if required).
The advantages that my options manager has are that each Option class uses generics to specify the type. Defaults are hard coded into the source then the options are loaded from a file and overwrite the defaults if they are present.

Cheating should not be specified in a per cheat basis. BAI instead uses the difficulty levels to decide whether or not to cheat. Easy - medium and hard do not cheat at all, very hard uses map cheating and impossible uses all cheats.

Config files I imagine are very AI specific. Different AIs would store different information about units and tactics to employ.

[hughperkins]

BAI

This is very off-topic and tongue-in-cheek, but if rtss were named like Spring AIs, I can't help thinking that their names would be like this:

Age of Empires would become "MRTS" ("Microsoft RTS")
Total Annihilation would become "CRTS" ("Cavedog RTS")
Starcraft would become "BRTS"

etc ....

[hoijui]

i already explained why having a standrd set of settings would be good.
also, having each cheat separately is good for the option of clean comparisons. i agree with your arguemnt that most users will probably not want ot set each cheat manually, but rather select a general difficulty level. i dont know if the current *Options.lua format supports presets (my guess is: no), but maybe we can implement this anyway, with just an additional setting:

Code: Select all

difficulty=easy|normal|hard|custom