Interface Redesign

[hoijui]

bibim set up Java on the buildbot, thanks. the caiinterface compiles now.
The installer generation stil fails though, cause the scripts have to be modifyed first. I am working on it.. but it will possibly take a while, as i have never used NSIS before.
A design question:
In the installer, the user will be able to select the Skirmish AIs he wants to have installed. How will we do it with the AI interface libs?
possibilities:

• The user selects which to install
• the installer decides which to install (if no Java AIs are selected for install, the Java AI Interface will not be installed either)
• We always install all interfaces

I am pro the last solution, as an interface without any AI installed does not cause any harm, plus the interfaces are usually very small in file size, plus the user does not have to worry if he has installed the interface when he wants to install a Java AI later eg.
..plus it is easy to code in nsis :D

[imbaczek]

We always install all interfaces

+1

[hoijui]

it is done now... and with bibims help, i got the installer script wokring agian.
though, spring.exe instantly crashes for me on windows. will try to fix that tomorow.

[hoijui]

edited branch url, should now be the correct one :D
https://spring.clan-sy.com/svn/spring/b ... iinterface

[hoijui]

It is still not working for me under windows.
With the installer, generated by buildbot, i get this error:

Code: Select all

Program received signal SIGILL, Illegal instruction.
__static_initialization_and_destruction_0 (
    __initialize_p=<value optimized out>, __priority=65535) at SMath.cpp:15

corresponding source:

Code: Select all

// SMath.cpp
14:    const Double DoubleZero(0.0f);
15:    const Double DoublePositiveInfinity = Double(1.0f) / DoubleZero;
16:    const Double DoubleNegativeInfinity = Double(-1.0f) / DoubleZero;

someone else pls try that installer too, so i know it has nothign to do with my system:
installer:http://planetspring.free.fr/spring/inst ... vn7170.exe
debug files: http://planetspring.free.fr/spring/debu ... g_R7170.7z

when i compile it locally on windows for windows, and i try to run spring.exe, i get an error message saying it can not find boost_thread-mt.dll. why would it look for that? could that happen if i have this dll in my path/library_path or if the static lib is not supplied when linking?

bibim:
please delete:
rts/lib/streflop/*.o and rts/lib/streflop/*.a
in the caiiinterface branch checkout dir on the buildbot. this way we can be sure that streflop gets recompiled with the new flags (no -msse2).
thanks

[hoijui]

it works!!

bibim did a make clean for streflop, and now spring runs on my machine aswell! fisrt pure SSE spring! no more SSE2

installer: http://planetspring.free.fr/spring/erro ... _R7173.txt
debug symbols: http://planetspring.free.fr/spring/debu ... g_R7173.7z
these are generated with -gstubs, instead of -g. please report if you have problems with them.

[hoijui]

this installer should come with a working test java AI (this is the only mentionable difference to the last one)
http://planetspring.free.fr/spring/inst ... vn7180.exe
debug symbols, if you want:
http://planetspring.free.fr/spring/debu ... g_R7180.7z

[hoijui]

sorry, if i am unclear about things in this post, please just ask.
i spent too much time with all this, to be able to think of things that might be non-obvious to others.

current aproximate state of the interface

working interfaces: C/C++, Java

Skirmish AIs that at least load: KAIK, RAI, AAI, NTai, Null*AI

unitsync should contain everything the lobbies need, including:
methods to fetch possible options for Skirmish AIs (works the same way as mod- and map-options)
methods to fetch infos of Skirmish AIs (works in a comparable way like options)

the format for script.txt can be found in the branch under Documentation/Spring start.txt
example:

Code: Select all

	// players in this will share the same units (start with one commander etc)
	[TEAM0]
	{
		TeamLeader=x;   // player number that is the "leader"
		AllyTeam=number;
		RgbColor=red green blue;  // red green blue in range [0-1]
		Side=Arm/Core;  // other sides possible with user mods i suppose
		Handicap=0-100; // Percent bonus on all resources collected ?
		StartPosX=0;    // Use these in combination with StartPosType=3
		StartPosZ=0;    // range is in map coordinates as returned by unitsync (NOT like StartRectTop et al)
		LuaAI=name;     // name of the LUA AI that controlls this team
		[AI]            // [optional]Skirmish AI section
		{
			Name=RAI;     // name of the Skirmish AI that controlls this team
			              // (see spring.exe --list-skirmish-ais for possible values))
			Version=0.1;  // [optional] version of the Skirmish AI that controlls this team
			[OPTIONS]     // [optional] contains AI implementaition specific option values
			{
				difficultyLevel=1;
			}
		}
		// Either LuaAI is set, the [AI] section is defined or none of the two.
		// The TeamLeader field indicates which computer the Skirmish AI will run on.
	}
	//more teams

How things work

lets first have a look at the source.
like in the past, code that is AI related is under rts/ExternalAI. this code is mainly C++ and is engine internal only.
Code under rts/ExternalAI/Interface is meant to be used by AI Interfaces and Skirmish AIs. It is nearly exclusively C. Only some helper methods for AIs or Interfaces are available in C++, for convenience. So a Skirmish AI or Interface dev should only have to look at this code, and not at rts/ExternalAI.
The dir AI/Wrappers contains subdirs, which can be seen as small projects. the only one in SVN so far is the LegacyCpp. This is code that is used when compiling Skirmish AIs that use a common way of connecting to an AI Interface, which is codewise independent of the interface, and which is built into the individual AI. example: all Legacy AIs (eg. AAI and KAIK) use the code under AI/Wrappers/LegacyCpp, so this code ends up compiled in libAAI.so, libKAIK.so, and so on.
Under AI/Interfaces, we have an other small set of subprojects, currently C and Java. Each of these subprojects lets spring access a set of AIs using a specific technology, this will mainly be programming languages, but if you wanted to make AIs work over the network, you would make an AI Interface aswell.
AI/Skirmish contians skimrish AIs. more about these and interfaces is explained in hte next paragraph.

the end product, and how it is constructed out of the code

AI Interfaces
Every AI Interface will have a native library (.dll, .so or .dylib). If it is an Interface for AIs in a different language, it will likely contain a library in this language aswell (an exception is the C AI Interface, of course). The Java interface for example, consists basically of the following files when installed (eg. under AI/Interfaces/Java/0.1/):
libJava.so
interface.jar (interface-specific)
InterfaceInfo.lua
jlib/*.jar (interface-specific. currently, this includes mainly the JNA jars)
lib/*.so (interface-specific. currently, this dir is empty, but native libraries could be put here, if the interface would require any in addition to the jvm)
The .so is called by the engine, and (through JNI and JNA) wrapps data from and to Java classes and methods found in the .jar. The first and the third file have to be present for all Interfaces. The .lua tells the engine everything it has to know about the interface. It contains info that is needed for loading the interface.
in the source it looks like this:
AI/Interfaces/Java/*.h
AI/Interfaces/Java/*.c
AI/Interfaces/Java/*.cpp
AI/Interfaces/Java/java/src/*.java
AI/Interfaces/Java/data/InterfaceInfo.lua #*# (see further down)
AI/Interfaces/Java/data/jlib/*.jar
AI/Interfaces/Java/data/lib/*.so
AI/Interfaces/Java/VERSION
The .h, .c and .cpp files are compiled into the .so, while the .java end up as .class in interface.jar. The java/src dir is the only interface specific thing here. All Interfaces have the data dir, whichs content is copied directly to the installation directory of the Interface. VERSION is a text file with only one line, containing the current version of the Interface. This file is used by Scons, CMake and NSIS (installer generator), to construct a version dependant installation dir structure. *

Skirmish AIs
They have a comparable structure like the AI Interfaces. The main differences are: they usually have only one library (in the target language), plus htey may have an AIOptions.lua file.
an installed Java Skirmish AI (eg. under AI/Skirmish/NullJavaAI/0.1/):
ai.jar (ai-specific)
AIInfo.lua
AIOptions.lua
the source structure:
AI/Skirmish/NullJavaAI/*.java
AI/Skirmish/NullJavaAI/data/AIInfo.lua #*# (see further down)
AI/Skirmish/NullJavaAI/data/AIOptions.lua
AI/Skirmish/NullJavaAI/data/jlib/*.jar
AI/Skirmish/NullJavaAI/data/lib/*.so
AI/Skirmish/NullJavaAI/VERSION
As with the interfaces, the source (in this case, the *.java files) gets compiled into a language specific library (here .jar), and everything under the data dir is copied to the install dir. Exactly as with the interface, VERSION is a text file with only one line, containing the current version of the Skirmish AI, here for the same purpose.

#*# For both AI Interfaces and Skirmish AIs, a special info key-value pair is added by the engine, the "dataDir".
The value for this info is the absolute path of the Interfaces of AIs root dir at runtime (without a trainling slash), eg "/home/userX/myGames/spring/AI/Skirmish/AAI/0.875"
Interfaces and AIs receive their own infos in the call to thier init() function. The idea is, that they can use log-, cache- and config- files from within their own dir (specifically speaking, this is where the *Info.lua file is located), instead of relying to the CWD that is set by spring.
AIs that i adjusted to use this dir: KAIK, RAI, AAI
Will have a chat with AF about this for NTai.
The Java and the C interface use this aswell.
The problem here is, that this dir may be read only, in some setups under linux (also under windows?). Therefore, i should possibly send an other engine generated absolute info to Interfaces and AIs, called "dataDirWritable". it is now on my todo list. question is, should this dir be created by the engine, if it does not yet exist, or is that job of the Interface/AI?
In addtion to this (in the future: these two) dir, the Interfaces receive a list of all data dirs (eg "~/.spring") in their init() method.

[hoijui]

LoidThanead, which is having a look at/workign with the new Java interface, made a suggestion concerning AI event handling. Currently, when an ai returns an error code from an event it received from the engine (eg, the update event returning -1 instead of 0(=OK)), nothing happens, not even a log entry.
This is of course not very comfortable for AI devs. The suggestion is, to log only if the error code has a certain value, eg. 1, and maybe 1 and 2 if it is a debug build. spring would then call a const char* getError() function on the AI, and print it to infolog. To make sure infolog is not filled with error messages of AIs, we could log only the first time an error occurs for event X on AI Y.

Do you like this?
Other ideas?

[Agon]

Hard to tell.
Handling the errors on the AI side gives detailed logs but more work for each developer.
Best would be optional via the AI options lua file.

Could we delete out commented code which is no longer required?
Or even double code (new and old code commented out)?

[hoijui]

i do not think it would be good ot have it in AI options. it is actually an engine option, so it should rather be in spring settings...
hmm, this reminds me... we already have a verbosity slider for AI error messages or somethign like this. maybe we could simple use this value, and define the following:
a return value of 1 means: very important
and 10 means: only log if verbosity is set to very high
then again, you probably want all errors or none usually.
arg, am too tired already, sorry. just am pretty sure it should be in spring settings somehow.

[hoijui]

am now partly working on improving the documentation of the new AI interface.

made 3 sequence diagrams so far (see attachments). i would like to hear what you think about them, and what else you would like to have explained with diagrams or in an other form.

[AF]

I'm not sure what use they are, sure they have a use, but it requires knowledge of UML 2 to understand, and most AI developers started out here with little or no knowledge of UML beyond class diagrams.

I think a basic flow chart would be easier to understand for the average user/newbie programmers.

[imbaczek]

AF: you're spreading FUD. anybody who doesn't understand those diagrams after a minute of looking shouldn't be writing an AI anyway.

[Agon]

AF: you're spreading FUD. anybody who doesn't understand those diagrams after a minute of looking shouldn't be writing an AI anyway.

True, before I looked at the images I thought: "I will see super complex diagrams now."
With this thought I looked at the images and searched for something I could missed because it looked too simple:
load Interface->(I think the diagram is missing: init interface)->load AI->init AI

And after this the handleEvents will spawn

Ah yeah, I have no question at the moment.

[AF]

AF: you're spreading FUD. anybody who doesn't understand those diagrams after a minute of looking shouldn't be writing an AI anyway.

I didn't understand UML diagrams when I first wrote NTai and it remained one of the most competitive AIs for a long time. Newcomers here shouldn't have to decode the diagram if they're unfamiliar with UML diagrams, and so a basic flowchart would be a far better form factor for these diagrams. They are simple so they should look simple at first glance, afterall diagrams are supposed to help.

[imbaczek]

but those diagrams are little more than flowcharts! i didn't say that you have to know UML to write an AI, but not getting the point of those simple images after a short while means you need to learn a bit about programming before attempting to write an AI. dumbing everything down to the point that non-programmers can understand the documentation means the documentation will be either next to useless or too verbose for programmers.

[hoijui]

for me it is clear that sequence diagramms (the ones i used here) are better for explaining what i want ot show here, whether you know what it is or not. i wont use flow charts for this, if you want them in flow-chart-form, do them yourself.

something usefull:
what else would need explaining?
ideas (mainly for self):

• the file structure of an AI
• how does that with the structs and function pointers work?
  we have the callback and the structs for the Skirmish AI, where the function addresses of loaded SkirmishAI libraries are stored.
  -> show that for a native and an other (eg Java) Skirmish AI

[imbaczek]

i'd like to see simple examples of how to do stuff:

- giving orders (construction, move, firestate/etc)
- get unitdefs
- draw lines and arrows (geometricShapes in the engine iirc)
- log messages to console
- get heightmap
- get pathfinder data for a unit
- enable cheats
- get a list of own units
- get a list of units in sight
- get a list of units in a particular area
- get mod options, map options, engine options
- get info about sides players started with
- process events (log them all, e.g.)
- get hashes of used archives (useful for precomputing stuff and saving to disk)

simple example AIs that do each of those would be perfect.

[hoijui]

You mean, we should have a "Hello, World" Test AI scheme, which is then implemted in each language (eg. C, C++, Java, C#) in a Test AI.
This scheme had to be working with absolutely every kind of mod, an if not everything can be done, it had to not crash at least, and report why it could not do what.

An example for such a scheme:

Code: Select all

myAI {
init() {
   - get unitdefs
   - get info about sides players started with
   - enable cheats
   - get a list of own units
}
unitCreated(unit) {
   - 50% chance: attack a random enemy unit
   - 50% chance: build something
}
update(unit) {
    - if the first unit we received is moving,
      draw lines along its way (eg all 300 frame a new line)
}
}

- draw lines and arrows (geometricShapes in the engine iirc)

it is possible to draw points, lines, paths and figures wiht the AI interface as it is now, i dont know if arrows are in fugures, or if they had to be constructed manually out of 3 lines eg.

- get hashes of used archives (useful for precomputing stuff and saving to disk)

you mean hashes for the mod and the map, to not use cached data for BA6.6 when BA6.8 is loaded eg?
if i am not wrong, this is not possible through the interface at the moment, would have to be added. is this the cleanest solution? (eg, what about mod-name + version?)