Java AI Interface for Spring

[cranphin]

Oh!

I've got a hunch :)

I think there was a lot of AIFloat3 in this 'reading' map.

-mayby- the new hashcode/equals is messing with the internals of jna!!

I'll need to try that :)

Cheers!

P.s. this is the file:
https://jna.dev.java.net/source/browse/ ... iew=markup

[hoijui]

thanks for the hint!

bumped JNA to version 3.2.0
instead of the main java lib + all the platform specific libs, they have a single jar now which includes everything, and the overall size went down from 1.2MB to 855KB. plus .. as cranphin said, there may be improvments in performance.

Structure.java for this version is here:
https://jna.dev.java.net/source/browse/ ... iew=markup

the part you expected problems in (the reading field) seems to be gone.

If you still seem to have a memory leak, and you think i might be of any use, contact me in the lobby (eg in channel #ai (for AI devs, btw ), where i am most often).

[hoijui]

About performance improvments in JNA 3.2.0:

As i got it, theis is the only (noteworthy) performance improvement method:
https://jna.dev.java.net/#direct

This is not applicable for the Java interface though, as we use function pointers for the method mapping, not global C functions.

[LoidThanead]

The version I'm using is the latest I grabbed from the buildbot: the installer spring_0.79.0-626-gb1d7ec1.exe.

It seems to have a package com.clan_sy.spring.ai as well as com.springrts.ai.

[cranphin]

the part you expected problems in (the reading field) seems to be gone.

If you still seem to have a memory leak, and you think i might be of any use, contact me in the lobby (eg in channel #ai (for AI devs, btw ), where i am most often).

I'll add the irc channel to pidgin soon
If I don't forget XD

About the memory leak:
The reading field has been replaced by something else, I think it will still leak.

I still haven't tested this, but I've thought a lot about it :)
The 'reading' field is a temporary storage to keep track of which 'Structure's are already read, to prevent recursion. It adds before reading, and removes after reading.
Add and remove into a hashmap/set depends on the .equals/.hashcode
By default Structure has a equals/hash based on the memory pointer, which will not change during reading.
But because I asked you to change AIFloat3 (which extends Structure) equals/hash, it now looks at the fields. And the fields will change during reading. This means that the hashcode changes after the items are added to the hashset, which probably messes it up badly, so that remove doesn't work correctly.
This is why lots of objects get lost in it :)

I need to test, but I'm pretty sure this is the case.
So sadly we'll need to remove the nice hashcode and equals for AIFloat3 again I think. It was a good java choice, but a bad jna choice :)

Cheers!

[cranphin]

About performance improvments in JNA 3.2.0:

As i got it, theis is the only (noteworthy) performance improvement method:
https://jna.dev.java.net/#direct

This is not applicable for the Java interface though, as we use function pointers for the method mapping, not global C functions.

Yes, that's what I noticed. Shame we can't use it :)
Still, it's nice to have the latest version, and it's nice to have just one jar :)

Cheers!

[hoijui]

Loid:
thank you!
i will have ot aks bibim to remove it then. till then, just forget about clan_sy.

cranphin:
nice thinkering!
sounds like you hit it...
i will make a comit to remove the equals and hash methods again, though to me this seems to be a bug in JNA; they should not use HashSet for this, but rather a Set impl. that uses ==.
Sadly, JNA uses mailinglists, which i hate. i have registered once already, but then deregistered again.
i may try to report it somehow, but if you want to do so (as you found it actually) it would be ok for me

[cranphin]

Loid:
thank you!
i will have ot aks bibim to remove it then. till then, just forget about clan_sy.

cranphin:
nice thinkering!
sounds like you hit it...

Yup, finally managed to test, now I use 10 mb instead of 64++ XD
Well, on a short test run, but it's clearly solves the issue :)

i will make a comit to remove the equals and hash methods again, though to me this seems to be a bug in JNA; they should not use HashSet for this, but rather a Set impl. that uses ==.

I think it's more complicated then that, the use the C memory pointer, not the object instance id, so the == wouldn't work for them (probably :) ).
But mayby they could use Comparator of their own making or so.

Sadly, JNA uses mailinglists, which i hate. i have registered once already, but then deregistered again.
i may try to report it somehow, but if you want to do so (as you found it actually) it would be ok for me

HEhe! ^_^
I feel some innuendo there :)
I'll see, this weekend I'll be busy, but mayby after :)

Atleast now I can go back to testing the partial hack of a port ai I built XD

[cranphin]

Btw, tiny suggestion, getGame().getCurrentFrame() should probably be marked deprecated :)

[hoijui]

done, though .. as it turned out in our discussion, the whole idea of deprecating this method was probably not so good

i filled an issue on the JNA project page, it cna be foudn here:
https://jna.dev.java.net/issues/show_bug.cgi?id=123

a guy of the JNA team said both the equals nad the hashCode function we used are wrong
maybe has to do with the use of super, i dont think it should be used there. he didnt say more then that it is wrong.

about the issue in JNA, he said that it is a tricky one, but... well they might try to solve it now

[cranphin]

Well, they say they fixed it in svn, which is great :D
They do use object identity now, as you suggested :)

Also, their complaint with our equals is not that our equals is wrong, but that it's redundant, since their equals checks as a memory comparision, so x,y,z is already checked :)

(Atleast, this is what I think they wrote :) ).

Anyway, its all good :D
Make sure to use the next release version of jna when it's out tho

So did the spring dev's ever say anything about the use/details of the frame field ? :)
Hmm.. I still need to add a check for that XD

[cranphin]

So did the spring dev's ever say anything about the use/details of the frame field ? :)
Hmm.. I still need to add a check for that XD

I added a check :D

It -looks- like all events after a 'update' have the same frame as the last update :)

So depricated still seems good

[cranphin]

Hmm, something weird, not sure where it comes from :)

There's a

Code: Select all

.getMap().getResourceMapSpotsPositions 

method, which for fun I've compared to the ported KAIK algorithm
For some maps, the points are identical, but for some maps, KAIK gives good spots, and the method from the ai interface gives bad spots.
Example from 'Icy Run v2', note that the KAIK spots are good:

Code: Select all

KAIKY:( 472.0, 1977.8639, 56.0)
KAIKY:(2472.0, 1977.8639, 376.0)
KAIKY:(3320.0, 1977.8639, 376.0)
KAIKY:(4680.0, 1977.8639, 440.0)
KAIKY:(5832.0, 1977.8639, 648.0)
KAIKY:(1688.0, 1977.8639, 968.0)
KAIKY:(2392.0, 1977.8639, 1496.0)
KAIKY:(5576.0, 1977.8639, 1816.0)
KAIKY:(4008.0, 1970.1075, 792.0)
KAIKY:( 232.0, 1970.1075, 1288.0)
KAIKY:(3336.0, 1970.1075, 1512.0)
KAIKY:(1416.0, 1970.1075, 1720.0)

SPRING:( 472.0, 1977.8639, 312.0)
SPRING:( 424.0, 1977.8639, 1288.0)
SPRING:(1272.0, 1977.8639, 1288.0)
SPRING:( 584.0, 1977.8639, 1496.0)
SPRING:(1736.0, 1977.8639, 2120.0)
SPRING:(1688.0, 1977.8639, 3048.0)
SPRING:( 344.0, 1977.8639, 4648.0)
SPRING:(1480.0, 1977.8639, 5624.0)
SPRING:(1960.0, 1970.1075, 2536.0)
SPRING:( 232.0, 1970.1075, 4008.0)
SPRING:(1288.0, 1970.1075, 4696.0)
SPRING:(1416.0, 1970.1075, 5304.0)

But on the map 'Altored Divide', all positions match 100%, and are all good.

Weird eh ? :)

Note: I removed all cache files for metal maps from Spring\cache\analyzedResourceMaps, so it's not a caching issue :)

[LoidThanead]

I notice that game.getCurrentFrame() has been marked deprecated. What're we supposed to use instead?

[LoidThanead]

During some of my own tests, I noticed this in the outlog file:

Code: Select all

User exited
java.lang.IllegalArgumentException: Structure class com.springrts.ai.event.ReleaseAIEvent has unknown size (ensure all fields are public)
	at com.sun.jna.Structure.calculateSize(Structure.java:718)
	at com.sun.jna.Structure.allocateMemory(Structure.java:237)
	at com.sun.jna.Structure.ensureAllocated(Structure.java:229)
	at com.sun.jna.Structure.size(Structure.java:267)
	at com.sun.jna.Structure.useMemory(Structure.java:220)
	at com.sun.jna.Structure.useMemory(Structure.java:208)
	at com.sun.jna.Structure.<init>(Structure.java:148)
	at com.sun.jna.Structure.<init>(Structure.java:141)
	at com.sun.jna.Structure.<init>(Structure.java:137)
	at com.springrts.ai.AIEvent.<init>(AIEvent.java:34)
	at com.springrts.ai.event.ReleaseAIEvent.<init>(ReleaseAIEvent.java:41)
	at com.springrts.ai.oo.OOAIFactory.handleEvent(OOAIFactory.java:86)

[cranphin]

I notice that game.getCurrentFrame() has been marked deprecated. What're we supposed to use instead?

Use the frame that you get in the update call :)
It's valid for all subsequent events, until the next update call.

[cranphin]

During some of my own tests, I noticed this in the outlog file:

Code: Select all

User exited
java.lang.IllegalArgumentException: Structure class com.springrts.ai.event.ReleaseAIEvent has unknown size (ensure all fields are public)
	at com.sun.jna.Structure.calculateSize(Structure.java:718)
	at com.sun.jna.Structure.allocateMemory(Structure.java:237)
	at com.sun.jna.Structure.ensureAllocated(Structure.java:229)
	at com.sun.jna.Structure.size(Structure.java:267)
	at com.sun.jna.Structure.useMemory(Structure.java:220)
	at com.sun.jna.Structure.useMemory(Structure.java:208)
	at com.sun.jna.Structure.<init>(Structure.java:148)
	at com.sun.jna.Structure.<init>(Structure.java:141)
	at com.sun.jna.Structure.<init>(Structure.java:137)
	at com.springrts.ai.AIEvent.<init>(AIEvent.java:34)
	at com.springrts.ai.event.ReleaseAIEvent.<init>(ReleaseAIEvent.java:41)
	at com.springrts.ai.oo.OOAIFactory.handleEvent(OOAIFactory.java:86)

This has something to do with the new JNA I expect, I think hoijui mentioned something in the past about the release AI event being a struct of size 0, and JNA not liking this.

[LoidThanead]

I notice that game.getCurrentFrame() has been marked deprecated. What're we supposed to use instead?

Use the frame that you get in the update call :)
It's valid for all subsequent events, until the next update call.

I have a situation in which I want to know which frame we're at the moment the object is created. (Instances of it can be created at any time during a game.) Technically, I could pass the frame number as an argument, but this feels unnatural.

I don't really see why being able to look up the frame number is bad somehow. Sure, you won't need to very often, but having the option is better than having to keep track of the frame number internally everywhere it is needed It just adds room for error. (Not that it's very complicated code, but it's still unnescesary, the way I see it.)

[hoijui]

Code: Select all

.getMap().getResourceMapSpotsPositions

if i remember correctly, i took the code from KAIK, and factored out all KAIK specific parts, and put this into the engine/AI Interface. KAIK uses an internal thread map, to sent builders to the most secure mexes first and stuff like this. though this method should be quite similar still. i will have a look at it, thanks.

Code: Select all

game.getCurrentFrame()

the reason this was deprecated, is that it is used extensively in legacy AIs, and wiht the new interface, especially with other languages (like java), where the CPU impact of calling this method is quite some, calling this method often is a bad idea.
in general, you will need at least one object in your AI, that is relatively global. instead of making the OOCallback global, you should have your own class, eg. MyGlobalStuff, which allows fetching the OOCallback, and some other things.. eg your internal global unit manager, if you have soemthing like that, or the currentFrame and teamId of your AI.
It is unlikely that this method will be removed in the next 10 years or so
its just deprecated to encourage not using it. it was deprecated before already, it was just that i did not use the propper DoxyGen/javadoc tag for it.

ReleaseAIEvent
will have to have a look at this too.. thanks
and yeah, it is some JNA fail, pretty much as cranphin explained it, i think.

[cranphin]

Code: Select all

game.getCurrentFrame()

the reason this was deprecated, is that it is used extensively in legacy AIs, and wiht the new interface, especially with other languages (like java), where the CPU impact of calling this method is quite some, calling this method often is a bad idea.
in general, you will need at least one object in your AI, that is relatively global. instead of making the OOCallback global, you should have your own class, eg. MyGlobalStuff, which allows fetching the OOCallback, and some other things.. eg your internal global unit manager, if you have soemthing like that, or the currentFrame and teamId of your AI.

Alternatively, I guess the OOAI layer could request the frame once on every update, store it, and then you rewire the getCurrentFrame to return that value ? :) No native calls anymore, and same result :)
It feels right to do caching in the OOAI layer anyways, to prevent too many native calls :) (If I remember there's caching in other places too).

I don't have a strong opinion here tho :D