Java AI Interface for Spring

[hoijui]

with only knowing this of you as a maintianer, i would say i am already better, even if i would dropp of totally now

The JNA direct mapping branch is operational (tested with NullOOJavaAI on linux and on windows). It is on the main repo now.
name: javaaispeed

The only thing that changed, interface wise, is that these methods are not available:

Code: Select all

Clb_UnitDef_0MAP1SIZE0getCustomParams
Clb_UnitDef_0MAP1KEYS0getCustomParams
Clb_UnitDef_0MAP1VALS0getCustomParams
Clb_Unit_SupportedCommand_0ARRAY1SIZE0getParams
Clb_Unit_SupportedCommand_0ARRAY1VALS0getParams
Clb_Group_SupportedCommand_0ARRAY1SIZE0getParams
Clb_Group_SupportedCommand_0ARRAY1VALS0getParams
Clb_Map_0ARRAY1SIZE0REF1Resource2resourceId0getResourceMapSpotsPositions
Clb_Map_0ARRAY1VALS0REF1Resource2resourceId0getResourceMapSpotsPositions
Clb_FeatureDef_0MAP1SIZE0getCustomParams
Clb_FeatureDef_0MAP1KEYS0getCustomParams
Clb_FeatureDef_0MAP1VALS0getCustomParams
Clb_WeaponDef_0MAP1SIZE0getCustomParams
Clb_WeaponDef_0MAP1KEYS0getCustomParams
Clb_WeaponDef_0MAP1VALS0getCustomParams

Reason: They al use either String[] or Float3[], which is not supported by JNA direct-mapping.
If the performance gain is notable, i may rewrite the C interface to allow fetching those things one by one, eg. const char* GetString(int i) instead of int GetStrings(const char**). This will make the calling of these functions a bit heavier, but as they are all easily cachable (and in fact are already cached in the Java interface) that should not be a problem. The Java interface would stay the same.

cranphin:
could you do some performance testing pls? :D
i should be online all day, and probably the next few days too, if you are not around.

[cranphin]

cranphin:
could you do some performance testing pls? :D
i should be online all day, and probably the next few days too, if you are not around.

I will do so ASAP, but since I'm not home this weekend it may take a few days :)

Will see tho, I have my computer with me XD But may not have time :) I am very curious though :D

[cranphin]

Reason: They al use either String[] or Float3[], which is not supported by JNA direct-mapping.
If the performance gain is notable, i may rewrite the C interface to allow fetching those things one by one, eg. const char* GetString(int i) instead of int GetStrings(const char**). This will make the calling of these functions a bit heavier, but as they are all easily cachable (and in fact are already cached in the Java interface) that should not be a problem. The Java interface would stay the same.

This seems strange, is it really so that arrays (or any kind of multple items) is not supported? They should add that to JNA XD
Then again, i have no idea how diret mapping works :)
Performance wise I can see this being important :)

How does this work with the map.getHeightMap anyways? Did it already work one by one or so ?

Cheers!
Cran

[hoijui]

nice, thanks!

the HeightMap is an array of numbers (int, short, byte, .. something like that). That is no problem, only thing we use that is a problem is: String[] and Pointer[], and AIFloat3 extends Pointer. There are two other things that are not supported... i think NativeMapped and WideString or so... nothing the AI Interface uses.

[ato]

Since I'm referring to it all the time, I've generated the HTML javadoc for the AI Interface and put it online here:
http://ato.github.com/spring-ai-javadoc/

[cranphin]

cranphin:
could you do some performance testing pls? :D
i should be online all day, and probably the next few days too, if you are not around.

I will do so ASAP, but since I'm not home this weekend it may take a few days :)

Doh!

Erm, is there a easy way to get a (windows) build of this? I can't build anything but Java on here XD

[hoijui]

http://buildbot.eat-peet.net/spring/ins ... e9996e.exe

[cranphin]

http://buildbot.eat-peet.net/spring/ins ... e9996e.exe

Doh again XD
I've been trying to profile it, but for some reason when I add the profiler lib it keeps crashing :)
(as in:jvm.option.x=-agentlib:c:\Program Files\YourKit Java Profiler 8.0.15\bin\win32\yjpagent)

No idea why, bah :)

[hughperkins]

Hi Hoijui,

That development was fast!

I gave it a whirl this morning, by running the tests in hughai/test/Tester.java, but I got the same results as before! About 6000 for getpos.

I don't really trust my build, I suspect I'm probably just running the old version somehow. Do you want to try running the testgetpos() test in http://github.com/hughperkins/HughAI/bl ... ester.java ? You could just copy and paste the class into your own java ai tester I guess? I mean, you don't have to, but I feel it's a fair test of how fast the getpos runs?

Hugh

[ato]

Hmm, with just a "dumb" microbenchmark. I'm definitely seeing an improvement but it's not huge. Proper profiling might yield more useful results though.

I'm just doing this in a Clojure REPL:

Code: Select all

(defn speedtest [#^Unit unit]
  (time (dotimes [x 10000] (.getPos unit))))

(speedtest comm) ; where comm is the commander unit object
(speedtest comm) ; several runs to warm the JIT
(speedtest comm)
(speedtest comm)

I get about 250 milliseconds (give or take 5%) for a run with master branch (after rerunning the test several times to warm up the JIT). So that's roughly 25 microseconds per call or 40000 calls per second (as per Hugh's benchmark). With the aispeed branch I get about 220 milliseconds, so 22 microseconds per call or ~45000 calls per second.

Note that I'm running under a 64-bit environment which could conceivably behave quite differently to 32-bit. I tested on a dual-core Athlon 64 X2 3800+ processor with Ubuntu and Java 1.6.0_14. I commented out the logging and debugging stuff in jvm.properties.

Not sure why I'm seeing better results than Hugh. What processor are you testing on Hugh and are you using a 64-bit or 32-bit OS?

Hugh, if you weren't already, try warming up your benchmark by calling your test function several times over (so the JIT has a chance to optimise it). I usually see a huge change in speed in the first couple of runs due to the JIT. I'd also try not calling System.currentTimeMillis() in every loop of your benchmark, I find that call itself to be quite slow. On my system calling System.currentTimeMillis() every loop iteration is slow enough to affect the getPos benchmark by more than 10%. I guess it may be better or worse on other operating systems or processors.

Edit: Just in case it's something to do with getPos returning an AIFloat3 I also checked getWidth() on the Map object. It gets about 8 microseconds per call. So not fast enough that's worth replacing getPos() with 3 seperate calls like getPosX(), getPosY(), getPosZ(). getStartPos() which also returns an AIFloat3 has similar times to getPos(). I didn't look at whether a function that returns an array of 3 floats is any better.

My suspicion is the only way to improve it further over what hoijui's already done is to try returning data in bulk. Like a getTeamUnitPositions() that returns an array of 3*numOfUnits floats. I'm happy enough with the current performance though, it's only likely to be a real problem on really huge games for me (which would probably start lagging my PC anyway).

[hoijui]

thanks you two.
well... the one thing i learned fro mthis attemp to speed it up:
do propper profiling, and then try to speed up the bad stuff, rather then having a guess on some vague stuff.
And testing only the speeed of a single function and not even checking what parts in the call if it are slow, is surely not enough.
you ddi a good thing there, ato, thanks, though what we would really need is propper profiling like cranphin did it.
Best would be, if the person trying to speed up the code would also do the profiling, i guess.

i will integtate the javaaispeed branch into master, somewhen soon, but then i will concentrate on other stuff i guess, eg bug fixing or feature adding.

I had a short google search on java profiling, but the only free stuff i found was Eclipse and Netbeans, wich require the profiling app to be an Eclipse/Netbeans project -> bad for us.
cranphin, i remember you telling me something similar, about your findings. if you or anyone else finds a good way for profiling, please post here.

am very tired, had a hard day (doing PHYSICAL work!! that stuff that can make you sweat! same thing tomorow) .. soo sorry if this is .. missing some.. of anything

[cranphin]

thanks you two.
well... the one thing i learned fro mthis attemp to speed it up:
do propper profiling, and then try to speed up the bad stuff, rather then having a guess on some vague stuff.

A great lesson indeed! ^_^
I always try to explain this to people at work, they never listen :D

I had a short google search on java profiling, but the only free stuff i found was Eclipse and Netbeans, wich require the profiling app to be an Eclipse/Netbeans project -> bad for us.
cranphin, i remember you telling me something similar, about your findings. if you or anyone else finds a good way for profiling, please post here.

Yup, same findings here, and the payed for one I liked crashed against the build you made XD Profling shouldn't be so hard! :)

Anyways, thanks for the effort, it surely is appreciated :)
I do wonder if mayby the problem lays with JNA, it seems to create a lot of overhead (many calls below our methods), guess it may be the price of a easy binding framework :)

Mayby I'll dig into jni and bindings someday..
Not soon I'm sure XD

[cranphin]

Btw. A different approach:
Would it be possible to make a dll that contains just some demo methods and structs? Like a AIFloat3, and mayby something like a Unit/UnitDev class, and a few demo methods that don't do anything but return a single instance, or a list of these things ?

With that we (I? :D) could run that in some jna examples, and try different types of mapping and so, and profile it too.

Mayby find out the bottlenecks, complain to jna people about them XD

This would be good, since we could give them a small example too, instead of having to make them install all of Spring :)

Is it clear what I mean ?
I don't have a C environment, but could someone put a dll like that together ? I'll happily handle the Java side then :)
(Think it needs to come with the source, so I know how to map it XD).

It should be somewhat similar to our trouble methods, so lets say the native code behind:
com.springrts.ai.DefaultAICallback.Clb_Unit_0SINGLE1FETCH2UnitDef0getDef(int, int)
com.springrts.ai.DefaultAICallback.Clb_Unit_getPos(int, int)
com.springrts.ai.DefaultAICallback.Clb_0MULTI1VALS3EnemyUnitsIn0Unit(int, AIFloat3, float, int[], int)
com.springrts.ai.DefaultAICallback.Clb_0MULTI1SIZE3EnemyUnitsIn0Unit(int, AIFloat3, float)

These only need to return demo values/lists, the methods themselves should be dumb and execute quick :)

Just an idea, any takers ? :D

[hoijui]

This does not sound like a good approach to me. We have a system that we want to improve ->we should messure exactly that, and not do some half way clone of it and messure that. If i got you right, for what you want, you could also use any other software that uses JNA.

Maybe we should try other libs, or try to get the one you have running. Did you try to run that build without the prfiling lib? Is it possible to run the profiling lib plus the debugging lib, and see hwere it crashes? or do you have info about where/why it crashes already?

[cranphin]

This does not sound like a good approach to me. We have a system that we want to improve ->we should measure exactly that, and not do some half way clone of it and measure that. If i got you right, for what you want, you could also use any other software that uses JNA.

Well, you're partially right :) Ideally we want to measure the whole system. And that's what I've already done too with profiling! :)
Based on that it seems the main performance 'leak' seems to be the JNA to native code bridge.
But because a game +AI is a hard to control system, I now want to focus on a repeatable and controllable test case around this part.
This has benefits:
- It's a lot easier to rerun often with different parameters/versions/settings/etc.
- Different runs can be compared.
- I can gather profiling data on just the JNA bridge, which I hope will show me more clearly the 'slow' JNA parts.
- I hope the profiler won't crash like this XD
- We have a contained example that we can show to the JNA people.

There are some risks:
- The native code itself could be slow, instead of the jna bridge. But based on my profiling I don't believe this, time is spent in Java-JNA methods mainly.
- Running like this changes the environment too much, so something will be faster/slower in this test. This is a risk, but I'm willing to take it, since I can't think of something better :D

Again, choosing this part is a result from profiling on the whole system :)

Also, I could use any dll, but there may be a big difference between a lib that returns just a int, or one that returns a AIFloat3 or UnitDef :) Also I want minimal native code in the lib, since I want to look at the bridge, not the native code, since that is where I think is the interesting part, I'm pretty sure spring native code performs fine :) (And some random lib would have random native code not related to our things at all :) ).

Maybe we should try other libs, or try to get the one you have running. Did you try to run that build without the prfiling lib? Is it possible to run the profiling lib plus the debugging lib, and see hwere it crashes? or do you have info about where/why it crashes already?

Ok, severeal things mixed :)
- I can't try the other profiler anymore, the trial expired XD Could ask for a OS license, but they only do that for well known projects, if you put their logo on the site (wonder if a wiki page counts? XD)
- I tried that build without the profiling lib (it's only a jvm.properties difference), and it seems to run fine.
- What is the debugging lib ? :)
- I have no idea where it crashes, it just goes poof, I guess some interference between spring native code and the profiler native code or so :)

Ahwell, mayby I can build a dll myself, I think I've got visual c++ express somewhere still, how hard can it be ? XD

Cheers! :)

[hoijui]

k... well i am still not convinced
but yeah, if you do it all by yourself, i dont care of course

the debug lib i meant:
in jvm.properties, the agentlib in the debugging related sectiong/part of the config file.
just.. you know.. the lib that lets you debug the JVM, eg with JSwat, Eclipse, or Netbeans. Doing stuff like halt, continue, move to next instructions, execute till pointer, set breakpoints, set watches, ...
maybe there is info in one of the conifg files abotu the crash (infolog, or the Java AI Interface log files).

Well.. for today, i am way too tired, maybe i will try to find a usable & free profiling lib again tomorow.

edit:
maybe this, it is free:
http://www.ibm.com/developerworks/java/ ... lthcenter/

edit2:
or this. free too, but needs compiling a part in C++, and last news are from 2005, still worth a look maybe, if the above fails:
http://ejp.sourceforge.net/

[cranphin]

the debug lib i meant:
in jvm.properties, the agentlib in the debugging related sectiong/part of the config file.
just.. you know.. the lib that lets you debug the JVM, eg with JSwat, Eclipse, or Netbeans. Doing stuff like halt, continue, move to next instructions, execute till pointer, set breakpoints, set watches, ...
maybe there is info in one of the conifg files abotu the crash (infolog, or the Java AI Interface log files).

Ahh, right :)
Haven't tried it, but since the crash is hard I don't expect much from it, since it's mainly java code debugging, not native :)

Anyway, busy figuring out how to build a dll now :D

[cranphin]

Hmm, well.. XD
I got a dll (it's a wreck, but it works :) ).
Funny: a 'fake' getPos takes 1 second (1.14 really), for 40000 calls of getPos, which creates a AIFloat3 for each call I think.
That's good performance! :)

So mayby you're gonna be right, this will not be good for testing XD


But, something did come from it, I use a bit of jna direct mapping, and I got a nice big error from the profiler on the console when I tried to use it :D So that seems to be why it crashes :)

Look:

Code: Select all

[YourKit Java Profiler 8.0.15] JVMTI version 3001016d; 11.3-b02; Sun Microsystems Inc.; mixed mode, sharing; Windows XP; 32-bit JVM
[YourKit Java Profiler 8.0.15] Profiler agent is listening on port 10001...
[YourKit Java Profiler 8.0.15] *** HINT ***: To get profiling results, connect to the application from the profiler UI
java.lang.UnsatisfiedLinkError: Error looking up function '$$YJP$$skirmishAiCallback_getInstanceFor': The specified procedure could not be found.

	at com.sun.jna.Function.<init>(Function.java:134)
	at com.sun.jna.NativeLibrary.getFunction(NativeLibrary.java:336)
	at com.sun.jna.NativeLibrary.getFunction(NativeLibrary.java:316)
	at com.sun.jna.Native.register(Native.java:1308)
	at com.sun.jna.Native.register(Native.java:1003)
	at HelloWorld.<clinit>(HelloWorld.java:11)
Exception in thread "main" 

No idea why it does that, far too late at night :D
But it's something for me to investigate, mayby it just hates jna direct mapping :)

Tomorrow I should try without direct mapping, so I can use the profiler, see why it 'seems' so much faster like this :)

Edit: note to self, try the same loop in my AI, see how it performs, mayby it is similar :) (Pick a method with low content/fast execution :) ).

Edit2: Also note to self, very similar performance in same loop in AI, around 1.3 seconds for 40000 calls.
Hmmmmm.. I guess there is a lot of native calls, so mayby that's not so great performance. Not bad either tho :) Mayby we should look more in just streamlining the AI with caching, less often updates, etc.
But atleast my sample dll is representative then :D

Edit3: Actually, using the same mapping, and with the profiler connected, it is -identical- performance, also 1.4s for the test :) So atleast the test resembles the game very very well for that part :)

[hoijui]

hmmm... now i am confused.
So you say, direct mapping does make it a lot more faster?

about the crash: maybe you have to make sure to have the mapt to the Java AI Interfaces native library in the native libs path. i forgot the java property for this, but maybe LD_LIBRARY_PATH works too. if that does not help, maybe load the library with System.loadLibrary() in a static block somewhere, though this may very well interfere with the JNA direct mapping lib loading thing.

about the IBM profiling tool:
it is a pain to setup. You need to be registered at the IBM webpage eg. I am on my way there, thoug the linux package is an RPM, and i have Ubuntu. It can not find some dependencies, eg /bin/sh, which i have of course...
well.. maybe ill try on windows.

edit:
good work, and good luck!

[cranphin]

hmmm... now i am confused.
So you say, direct mapping does make it a lot more faster?

No! :)
Well, -I don't know- :D So that's not what I say :) I still need to create a sample dll which allows me to do direct mapping of getPos and such :) (I had some issues with that cause of --stdcall things :) ).
Sorry, I'll try to give more clarity when I'm out of work, and slept away the several too late nights XD

about the crash: maybe you have to make sure to have the mapt to the Java AI Interfaces native library in the native libs path. i forgot the java property for this, but maybe LD_LIBRARY_PATH works too. if that does not help, maybe load the library with System.loadLibrary() in a static block somewhere, though this may very well interfere with the JNA direct mapping lib loading thing.

Looking at the error message, I think it's just interference with the way jna direct binding, and the profiler work (notice the weird prefix on the method name), they both use tricks to make their stuff work, and I think they interfer :) I think it won't be solved also, ahwell Posted on the jna user list about it, don't think they'll have an idea either :)

about the IBM profiling tool:
it is a pain to setup. You need to be registered at the IBM webpage eg. I am on my way there, thoug the linux package is an RPM, and i have Ubuntu. It can not find some dependencies, eg /bin/sh, which i have of course...
well.. maybe ill try on windows.

Argh, sounds great XD
And it probably doesn't support sampling, so it'll slow down execution without lots of carefully set up filters, mayby, but it will be interesting to see, *looks forward to updates* :)