Java AI Interface for Spring

[cranphin]

i dont know the critical methods (except the already mentioned ones).

no, a unit object is never fully populated. each property is requested from the engine right when you request it from the Unit java Object.

This is probably part of our problem, it would be faster to just have one set of data to copy, I -think-, instead of many calls :)

So ideally, when someone requests info about a unit, get the whole unit and everything with it, keep that in cache for the frame, so at most one call per inspected unit per frame.

But ofcourse it's nowhere near that easy :)
1. If someone only wants the position of a lot of units for example, getting a lot of data for each unit would waste performance I think, and memory :) But this is something that could be checked with profiling.
2. Things like unit def never change per unit, so it's better to request it once per unit life, but that means you can't get -all- the unit info in one call, you need to split it up in parts, and how do you let the java code know when it should make it null (end of life) ? :)
3. things like a unit pos can be different even in one frame, because of the cheat thing.

It's complicated :)

[hoijui]

yeah you are right.. the cheating/not cheating is an other thing we have to be carefull with.

Out of your JProfiler picture, i figured out hte same, AIFloat3() beeing heavy. I had a short look at that... it seems that every time an instance of a jna Struct is allocated, JNA determines the memory size of the native struct, which is kind of strange, as this can not change during runtime. Maybe we really should contact the JNA guy(s) about that, or generally our performance problem.

[cranphin]

yeah you are right.. the cheating/not cheating is an other thing we have to be carefull with.

Out of your JProfiler picture, i figured out hte same, AIFloat3() beeing heavy. I had a short look at that... it seems that every time an instance of a jna Struct is allocated, JNA determines the memory size of the native struct, which is kind of strange, as this can not change during runtime. Maybe we really should contact the JNA guy(s) about that, or generally our performance problem.

I think we should contact them :)
They should be the uber guys in this field

Mayby we can make one addicted to Spring :D

But, ideally, mayby we can have them get spring and give them some sample code for the performance issues :)

[cranphin]

Btw. I'll try to post another profiling picture soon, now that I've added a few performance fixes (some caching, change some code to be executed less often), and with more details. I thing I noticed for example is in jna there's often a call to Class.isAssignableFrom (I think it was that method) which seems to use a lot of time :)
(It's not the only thing tho, the lower you go, the more it's spread over several calls :) ).

I think it would be good if we could get the performance sample mentioned by someone earlier, and profile that

[hughperkins]

Hi Robin,

Cool conversation the other day.

I've got a couple of bugs for you at some point involving line drawing. Since it looks like you're looking at the performance issues, I don't want to distract you from those I can just hold onto the bugs for now, or maybe you have a bug database where I can log them for now?

Hugh

[hoijui]

to be honest.. i am not working on any of this. i am still working on fixing my branch to be merged into master.
AI bugs go where engine bugs go too, on mantis.
You get there through this site (springrts.com):
Development - Report Bugs/Submit a Patch
Then switch to the AI project, and report the issue.

[hughperkins]

Hi Robin,

HughAI is at least up to a state where it runs and can kill the other person, as long as the other person doesn't do anything too difficult, like build any units

You can download it from this patch here if you want:

http://springrts.com/mantis/view.php?id=1623

I've posted in the development forums to have the patch added to git:

http://springrts.com/phpbb/viewtopic.ph ... 89#p376489

[hughperkins]

For the line drawing bugs, I've logged them in Mantis:

http://springrts.com/mantis/view.php?id=1624 Drawing 8000 lines, then viewing them causes Spring to segfault
http://springrts.com/mantis/view.php?id=1625 Java interface v0.1: CreateLineFigureDrawerAICommand causes Spring to segfault

[cranphin]

Well, I tried to make a few usefull profiling pictures :)
Soo much data tho XD

I think the pictures will speak for themselves, have a look :)

Questions/wanna see more, ask! ^_^
(Gah, only 3 att. max, gonna post the nextone after this :) ).

[cranphin]

And the lastone :)

Note: I'm sure I can decrease the amount of setCheating calls, but this is just to give an idea of how such calls are where performance goes, and what happens inside those calls :)

[hoijui]

can you please explain what the numbers represent in those pictures, eg, give the row headings?

for now, i just cared for the percentages. and yeah.. looks like it could probably profit a lot from faster Java-> native calls.

I am currently doing the last steps on the merge of the airefactor branch to master.

[cranphin]

can you please explain what the numbers represent in those pictures, eg, give the row headings?

for now, i just cared for the percentages. and yeah.. looks like it could probably profit a lot from faster Java-> native calls.

Doh XD

Actually, the percentages are most important, they're % of total time measured.

The left number is time in ms, the right number is own time (so time spent in just the method, not in called methods).

But the % are more interesting, especially since this is recorded with sampling (not very accurate in absolute numbers :) ).

Note that this is a run of my code, and if I call a method often it will register as high %, so don't take it too exact, also profiling is tricky, small changes will show different results :)

Just wanted to show what happens underwater, and under jna, roughly, and show that a lot of time does go there :)

[hughperkins]

Hi Robin,

Line-drawing seems totally borked in git head http://springrts.com/mantis/view.php?id=1634

It'd be nice to get this working again. It's way more fun to make AIs when one can draw maps and stuff!

It's possible that I'm doing something silly in my config files, or failed to build Spring correctly from git somehow, so if line drawing is workign for you, or for anyone else, via the java interface, in git head, I'll figure it's probably something I'm doing wrong, but otherwise... it'd be really nice to get this working again! if you get a moment.

Hugh

Edit: so basically, I'd like to be able to test in an environment where:
- I can use maphack
- I can draw maps on the terrain

In Spring 0.80.4, maphack doesn't seem to work, or, at least, cheating seems to be permanently set to 'enabled' and the enemy units don't seem to be visible (haven't tested rigorously, so I could be wrong, but it's my impression)

in git head, maphack works perfectly, but can't draw any lines on the map

[DJ]

Is anyone actually having real problems with the performance of the Java interface? I just ran a test game between my Java AI and Hugh's at 3x speed and it was flawless.

Great work hoijui! The interface is awesome!

[hughperkins]

1. hoijui's interface is awesome.
2. performance sucked on my AI initially, so, as a workaround, it only runs one out of every 30 ticks.

On C#, it is possible to make the AI run once a tick, but hey, the C# interface didn't really survive the msvc->mingw transition

Nevertheless, it would be nice to be able to make calls into the callback that run a bit quicker than 200 microseconds per call....

[hughperkins]

Well, I finally plucked up courage to play against E323 again, and ... I won :-D

Replay:

http://manageddreams.com/hughaireplays/ ... 23.tar.bz2

Built version of HughAI that was used:

http://manageddreams.com/hughai/HughAI. ... 09.tar.bz2

Playing against E323 v2.10.3 (whatever was in git at about 12am gmt yesterday morning).

Random technical thing for techies: it includes a console now! so you can write Java code directly into the gui, and run it, against all the AI's classes and stuff! is that cool or wot :-O Admittedly I haven't used that functionaltiy very much yet, but it was kind of fun to build :-D

Picture of console:



You can also see in the picture that there is a 'config' tab, which allows dynamic configuration of various values, like, for example, 'MapHack', which needs to be enabled in order to compete against E323, since E323 uses maphack too

[hughperkins]

Heightmaps...

Hi Robin,

I was looking at trying to accelerate heightmap loading this morning.

It seems that the heightmap that one gets by doing aicallback.getmap().getheightmap() provides the center heights, rather than the corner heights, of each square.

This is essentially a one-way transformation: you can go from corner heights (n points) to center heights (n-1) points, but you cannot go accurately the other way around, since the equations to do so are insufficiently specified.

In order to obtain the corner heights, what I'm doing at the moment is to call aicallback.getmap().getelevationat() for each point on the map, which takes ... a while... since there are 512 * 512 points * 200 microseconds per call, which is about ... gets out the calculator.... 52,000 milliseconds, or about 52 seconds. Actually, I think it runs slightly faster than that, maybe 20-30 seconds, but it's still a long time...

I can cache that to disk, that's pretty much what I do, but it might be nice to be able to get the original corner heightmap directly from the spring engine. It is after all how the heightmap in spring engine is stored (readmap->heightmap I think?).

What do you think?

By the way, detailed technical runthrough on heightmaps in the comments at the top of my HeightMap class here:

http://github.com/hughperkins/HughAI/ra ... htMap.java

For context, here is my entire HughAI todo list, just in case you have nothing better to do with your time than look through my todo list but the reason it might be useful is that if you look at the section "to communicate" and "to communicate, done", you can see all the things I'm hoping to push onto your own todo list at some point

http://github.com/hughperkins/HughAI/ra ... i/Todo.txt

Oh, while I remember, don't suppose... could you disactivate logging and debugging on the Java Interface jvm.properties files, so that anyone using a Java AI after the next release will have their AI running at full speed?

Hugh

[hoijui]

for logging, best would be if it would log as it does now for a debug build of spring/the java interface, and less otherwise.
will have a look at it, if i am too bored doing this, then i will put it down in general.

about all the todos/bugs you found these days...
:D
well...
i am in general not really motivated anymore to work on spring. i still do so, but .. as there are quite a few AI devs already that seem very potent themselfs, it would be nice if some of you would help fixing the bugs too. fork spring, commit to your fork, tell me or Kloot or Auswaschbar, and it will be in master.

ouh yeah.. call me hoijui pls

i did half of the writing for the direct mapping change this morning. The C part and the build system changes (most ugly stuff) is done, now it is the Java/JNA part that still needs to be done.

[cranphin]

I did half of the writing for the direct mapping change this morning. The C part and the build system changes (most ugly stuff) is done, now it is the Java/JNA part that still needs to be done.

Sweet! :)

Btw. for testing or those curious:
http://code.google.com/p/crans-java-ai/ XD

Dumped it there just so it has a place and so you/others can look at it/test with it. The idea is GPL2, which should be ok (kaik = spring = GPL2 right?). It needs a thankyou file and such ofcourse (with all kaik contributors), but I'll get to that once it gets decently running

It should be relatively playable, so feel free to give it a try :)

Only one important/annoying thing (besides having to build it and put it in a spring skirmish ai folder which I forgot to add XD ) is that the AIInterface jar needs to be slightly modified (for now), as a hack the two methods are made public:Unit.getInstance and UnitDef.getInstance :)

It's an eclipse project btw., just checkout the trunk folder as the root folder :)

PS: I made it easier to try, just get the two zips from http://code.google.com/p/crans-java-ai/downloads/list and place them in the right place :)

[hughperkins]

i still do so, but .. as there are quite a few AI devs already that seem very potent themselfs, it would be nice if some of you would help fixing the bugs too. fork spring, commit to your fork, tell me or Kloot or Auswaschbar, and it will be in master.

Oh... that sucks I was kind of banking on your being a better maintainer than I ever made! My attitude to being a maintainer tended to be: write lots of stuff in about three days, unload it, and hten move on to something else...

You can see that in the fact that the C# interface is absolutely non-existent at this point.

I feel it might be a shame if the Java interface suffered a similar fate. I'm rather banking on your keeping it working and maintained, and all I have to do is make sure my AI works.

i did half of the writing for the direct mapping change this morning. The C part and the build system changes (most ugly stuff) is done, now it is the Java/JNA part that still needs to be done.

Sweet! Nice one, hoijui.

Hugh