Java AI Interface for Spring

[hoijui]

hmm k :/
well. still god luck with that.

The IBM thing...
my SUN JVM is unable to load the native part of IBMs profiling library...
guess it would ned IBMS JVM... well!!!!

edit: just checke, and it IS for IBMs JVM only. DOH!

[hoijui]

Ok.. tried Netbeans Profiling.
Worked well and was easy.. until it started profiling. This worked fine too, but it just never spit out any data; no error or warning or something (it tracked the application fine, eg showed used memory and stuff, but no profiling data (eg. caled methods)).

[cranphin]

Oh, about my __stdcall thing :)

https://jna.dev.java.net/servlets/ReadM ... msgNo=3210
Usefull to know :)

[hoijui]

ahhh...
i dont get it. So.. it should with buildbot build of javaaispeed, it should fail, cause i dont use that... why would it only fail if you use hte profiling library? um.. i feel i fogot something.. too little sleep.

[cranphin]

Never mind me, it has no bearing on your build :)
You link against AIInterface.dll or so, which seems to not use __stdcall in the dll, so all is fine there :)

It's just an issue I had with my messy tests :)

Also, it has nothing to do with profiling :D
I don't know why my profiler fails, but it's not fixable I think, so don't worry about it

[hoijui]

i do use stdcall on 32bit windows builds, through the EXPORT macro in rts/System/exportdefines.h. Thats why i wonder why it does not fail, but.. i'll just believe you for now, as you tell me i dont have to care about it

[cranphin]

Hehe! ^_^

Well, using __stdcall would result in methodname@## kind of names in the dll, that's why it needs special treatment.

In javaaispeed_spring_0.80.4-132-gee9996e.exe there's the file AIInterface.dll, And it has neat tidy method names like: "Clb_0MULTI1SIZE3FriendlyUnits0Unit".
So it looks like you don't use __stdcall, I think :)


My methods on the other hand look like this: "_skirmishAiCallback_Unit_getPos@8", notice the '@8' :)
I think that's the result of __stdcall :)

I knew nothing about this stuff before yesterday though, so I'm not a good person to talk about this

P.s. you can look at your dll with this :D
http://www.ucware.com/apev/index.htm#jaws

[Kloot]

Well, using __stdcall would result in methodname@## kind of names in the dll, that's why it needs special treatment.

The funny names are due to name mangling, which is necessary in C++ regardless of the calling convention used. But the interface is in plain C, where mangling isn't required.

[cranphin]

Well, using __stdcall would result in methodname@## kind of names in the dll, that's why it needs special treatment.

The funny names are due to name mangling, which is necessary in C++ regardless of the calling convention used. But the interface is in plain C, where mangling isn't required.

If that's that's the case, then yes, that makes perfect sense to me :D
Where does that AIInterface.dll come from anyways? (the one in the java interface folder), is that generated code by the awk script?

P.s. Ahh, right, I based my 'test' dll off the spring.exe code XD
I guess there's a generated wrapper .c code with all the methods in c for the Java interface :)
Still wonder where that code comes from and how it's built :D

[hoijui]

:D
well.. its all in the repo :D if you want to find out.
the interfaces are plugins, and since some weeks, they are really are modular, so the whole Java interface is under AI/Interface/Java, there is no special code outside of this, that does anything for the java interface (except the SCons build files, which are not interstign for you).
in the javaaispeed branch, a tiny part of the native sourcce of the interface is generated by an AWK script, and can only be found in the build directory, the main part of it is at src/native/, which uses the headers under rts/ExternalAI/Interface/*.h.

maybe you want to have a look at these gc flags for the linker:

Code: Select all

-Wl,--kill-at -Wl,--add-stdcall-alias

[cranphin]

:D
well.. its all in the repo :D if you want to find out.

Ahh, right, the generated C code is a new thing to support the direct jna stuff XD
And it would help if I'd actually -look at the branched source- XD
Ok, I see how it works now :)

[cranphin]

Finally a little more time to try with my 'test dll' :)

I found a way to still run the YourKit profiler with the direct mapping too, by disabling several things in the profiler options :)


Right now simple loops are more interesting though.
I'm still trying things, but one simple thing I found is that direct mapping can give -awesome- speed results. But only in certain cases :)

For example, Unit_getDef, which only gives a integer id as result:
- Callback method (this is what the current code uses I think, function pointers in a struct):
216 miliseconds for 40.000 calls.
- Old Mapping but no callback:
136 miliseconds for 40.000 calls.
- Direct mapping:
7 miliseconds for 40.000 calls.. Wow! :)

For methods like Unit_getPos the performance gain is much smaller, but this is because it has to create AIFloat3 objects, which is very costly. I'm gonna see if there's some ways to speed that up, mayby I should ask on the JNA forum now I've got some example code :)

*goes back to tinkering* :)

[hughperkins]

getpos is one of the most critical functions for AI time, so if you can speed that up cranphin that would be awesome.

Random thought: what if the pos was passed as an array of floats insstead of a struct? Presumably one could prototype that quite quickly, not even using spring, just a simple 3 file, 50 line prototype to compare the different ways of passing data?

[cranphin]

getpos is one of the most critical functions for AI time, so if you can speed that up cranphin that would be awesome.

https://jna.dev.java.net/servlets/ReadM ... &msgNo=737
I've got some hopes for this, in 'call the method lots' tests the calculating of this size seems to hit pretty hard, so caching makes a lot of sense. Should see if we can't compile a jna ourselves really which does a prototype of that :)
Accidently posted it to dev isntead of users tho, doh XD
Don't expect magic tho! It's not gonna be amazing fast, but it should help :)

Random thought: what if the pos was passed as an array of floats insstead of a struct? Presumably one could prototype that quite quickly, not even using spring, just a simple 3 file, 50 line prototype to compare the different ways of passing data?

Yeah, I was thinking along those lines too, but not sure how much it will gain, worth trying though. Also was thinking if we could pass stuff by reference instead of by value. All kinds of ideas :)
Say, did you do windows or linux? You can have my little test projects :D

[hughperkins]

Say, did you do windows or linux? You can have my little test projects :D

linux. You?

[cranphin]

Say, did you do windows or linux? You can have my little test projects :D

linux. You?

Doh! Yeah that'd be usefull info :)
I work on windows atm.

I can still give my test project tho, just means you'll have to do some porting :)
Mayby easier to write your own XD

[hoijui]

cool cranphin, nice you go on wiht it!
well.. i guess these findings are enough to justify merging the javaaispeed branch into master already. and.. good post on the JNA mailing list!
all good work!

[cranphin]

Did some more posts on the JNA list :D

I've got the feeling JNA Structure is still somewhat immature, it needs cleaningup/fixing and optimizing/profiling. It could gain a lot of speed by caching information per Class I think :)

So, I've got hope the Java interface will still get faster, but I'm afraid we will have to be patient for the JNA guys :D

[cranphin]

Ohh, a suggestion btw.
Right now it looks like we run without the -server option:
";jvm.option.x=-server"

I think performance will benefit from using the -server option :)
Not 100% sure, but looking at descriptions of it I think it's better suited to our AI's.
In my very contrived tests I do so quite an increase in calls per second using '-server', but ofcourse that's not the best example, kinda hard ot measure it on a full AI though :)

Also I think it's a good idea to disable all the debugging options in jvm.properties for the next release, since people are starting to release proper AI's now and 'just players' will benefit from performance :)
(Well, the debug options that cost performance, but not sure which ones that are :D).

[hoijui]

Ohh, a suggestion btw.
Right now it looks like we run without the -server option:
";jvm.option.x=-server"

the default for 64bit is server already.
yeah... this might be a good point. I believe ot remember that SUN generally recommends using client on 32bit, but as we have a rather uncommon use of the JVM, you might be right there. Also, i agree wiht you that we should probably base this decission on a more realistic test then the ones you take here. For example, if it speeds up a non GUI AI by 1% but makes HughAI with its SWING GUI 50% slower, we might think again.

Also I think it's a good idea to disable all the debugging options in jvm.properties for the next release, since people are starting to release proper AI's now and 'just players' will benefit from performance :)
(Well, the debug options that cost performance, but not sure which ones that are :D).

This has been dealth with already, if you have a closer look at the current jvm.properties file.

about the JNA guys:
maybe we could supply a patch. As i imagine it, this would be a very simple one, osmehow like:

Code: Select all

private Map<Class, int> classSizes = ...;

if (classSizeCaching && classSizes.containsKey(theClass)) {
   return classSizes.get(theClass);
} else {
   // calculate size as usual
   classSizes.put(theClass, theSize);
   return theSize;
}

optionally, the classSizes Map could be a Weak implementaiton, so it will not fill up memory in other projects, that might have a lot of classes which are seldomly used....
then again.. i dont this this could ever be a problem, and surely is not for us (we would only have about 3 entries or so in this map).
checking how easy it isto get JNa source code...