Engine Testing - 97.0.1-425-g3d34d8b (23. Sept 2014)

[abma]

release-candidate #5 for 98.0!

some performance issues were fixed with the pathfinder, so pathing very likely behaves slightly different

Changes

noteable changes since RC3:

• fixed some ATI related crashbugs
• add DrawInMiniMapBackground event
• fix #4506: Some features don't render
• fixed some plane behavior issues

To make the next release post more interesting i need your help. i want to add screenshots & maybe a video to the release post, current screenshots/text snippets are:
http://springrts.com/wiki/ReleaseNotes:98.0


Changes in Detail:
changelog.txt (is currently the same as for rc2)
git diff last rc to HEAD

no breaking changes known atm since 96.0


Download
See the download page for how to get it.


Bugs

If you find a bugs, please report to Mantis.

Please attach infolog.txt as file, if you crash!

current release (known) blocking bugs no major bug.

[Silentwings]

\o/

[Beherith]

Yay indeed! Thanks for all the work!

[raaar]

nice!

NOTE: didn't find reference in the changelog for the fixing of the fighter aircraft combat behaviour.

They no longer seem to fly into the ground when targeting ground units, but they seem to only try to regain altitude when they pass directly over their target, which can make them fly too low and potentially bump into buildings.

[Silentwings]

I've added a few minor things to the "breaking changes" wiki page.

[abma]

i've heard no big complains about major bugs. only issue i know about is that current development version of spring seems to be slightly slower for Zero-K.

last change to speak to delay 98.0 :) else i'll try to create a release tomorrow.

thanks all for your great help!

[Jools]

Thanks yourself for allowing us to test this.

[Google_Frog]

i've heard no big complains about major bugs. only issue i know about is that current development version of spring seems to be slightly slower for Zero-K.

last change to speak to delay 98.0 :) else i'll try to create a release tomorrow.

thanks all for your great help!

It is not slightly slower. It is significantly slower. I have tested team games on the new engines and almost always receive performance complaints. These complaints are backed up by benchmarking.

I do not have the worse reported problems with dev engine. For me the max speed game frame has 150% the time taken of the 91.0 max speed game frame. The update also takes longer but that has varied more. These values are significantly worse for Licho.

Here is an explanation of the benchmark and some data:
https://docs.google.com/spreadsheets/d/ ... =624811569

[abma]

It is not slightly slower. It is significantly slower. I have tested team games on the new engines and almost always receive performance complaints. These complaints are backed up by benchmarking.

some changes make it slower by design: for example the lua-allocator was changed, now it counts memory usage and time for allocate & garbage-collector in the frame-grapher. idk how much the impact is.

[Jools]

They have 91 as baseline, would be interesting to see similar comparisons with 96 as baseline.

[Jools]

Writing in here instead of in the News section, as this is technical:

Code: Select all

- LuaRules & LuaGaia are now splitted into their synced & unsynced parts, and each one got its own luaState
  ! this way unsynced parts of gadgets now get `synced` callins like 'UnitCreated' etc. w/o  SendToUnsynced from their synced part

Is there any point in rewriting existing gadgets that use SendToUnsynced, or doesn't it matter which way to use for speed?

Of course many dual state gadgets can now be made solely unsynced ones.

[Google_Frog]

It is not slightly slower. It is significantly slower. I have tested team games on the new engines and almost always receive performance complaints. These complaints are backed up by benchmarking.

some changes make it slower by design: for example the lua-allocator was changed, now it counts memory usage and time for allocate & garbage-collector in the frame-grapher. idk how much the impact is.

The benchmarks do not use frame grapher.

[abma]

The benchmarks do not use frame grapher.

most data is always collected.

https://github.com/spring/spring/blob/d ... r.cpp#L166

but this is just an example what was changed which decreases performance. this is theoretical, i don't think if it can be measured / makes a notable difference.

[Google_Frog]

Most data is always collected where? Do you mean that collecting this data within the engine has reduced performance? It seems unlikely that you would say that.

[abma]

Most data is always collected where? Do you mean that collecting this data within the engine has reduced performance?

i'm pretty sure collecting the data doesn't improve the performance

but time taken there should be like 0.000001%, but idk it wasn't tested. no clue how fast spring_gettime() is. it should take only a few cpu cycles, but internally it does a system call, so not sure.

but still i think with this measurements it will basicly improve performance, as the lua code can be written in a more efficient way when time taken / mem usage is known to devs.

[Google_Frog]

So it sounds like the discussion about the timing and frame grapher is irrelevant.

Do you have anything to say about what I actually said? That the dev engine has much worse performance than 91.0.

[abma]

Do you have anything to say about what I actually said? That the dev engine has much worse performance than 91.0.

my personal impression is that 98.0 is slightly faster than 91.0... in short: no, can't confirm.

zero-k was two years optimized for spring 91.0...

i don't see any constructive in this discussion atm. it would be helpful to point out what specific part of the engine is slower. "everything!" isn't helpful

[Google_Frog]

Were you using 91.0 with Linux and OMP? I have reports from Linux players that 91.0 is unusable for them. Anarchid did some digging around and found a way to disable OMP which fixed the performance.

There is very little documentation on how to optimize the interface between lua and Spring (as opposed to information on how to optimize pure lua code, that exists elsewhere). Any optimization we've done has been through experimentation so in a sense ZK is optimized for 91.0. What is fast and what is slow has been mostly unchanged for many years. A massive change would be needed to explain the performance difference. If there is was a massive intentional change to how games need to optimize their lua then it is important that you announce it. Game devs have almost no way of discovering this by themselves. If there was an unintentional massive change then it is simply a bug.

In fact there has been a significant change to some optimization within gadgets. Access to SYNCED and SendToUnsynced is apparently much slower. I have already been informed of this by jK and I updated the gadgets accordingly. There was also a minor Spring.GetUnitsCommand change which has also been taken into account.

I have bad data which says that the lua of ZK takes less total time with the dev engine than it does in 91.0. This is bad data because it consists of looking at the debug view and various lua profilers. Current lua-side widget or gadget profilers have a significant performance impact so I am unable to gather better data.

How would I point out which parts are slower? Give me a detailed method for doing so and I will do that. As I said, both gameframe and update are slower. I know no way of getting good data on finer structures.

I think an effective testing method would use an engine branch with drastic changes to candidates for poor performance. Replace bits of the engine with placeholders. If that is too hard take large segments of the engine and have them run 10x more than usual. Repeat for many segments to try to find one which has an unreasonable effect.

They have 91 as baseline, would be interesting to see similar comparisons with 96 as baseline.

I recall 96.0 being even worse than dev engine. I have no good data for that though.There are compatibility issues with ZK on 96.0 so it would take effort to set up a test.

[abma]

How would I point out which parts are slower? Give me a detailed method for doing so and I will do that. As I said, both gameframe and update are slower. I know no way of getting good data on finer structures.

- create a deterministic test. sth. like: create 100 units and destroy 100 units, get that time.
- create / query a command queue

basicly test times for create/query/change/delete functions. not sure if this could be easily added to all lua functions at engine side. thats my understanding how you could compare performance between non-syncing engines.

still: comparing is very difficult as engines behave different! even config settings are quiet different, as some config defaults (afaik) have changed. lower gfx details and it will be faster!

still then its questionable if this command can be speeded up at all. imo it would be more efficient to find the bottlenecks and speed up these. To find bottlenecks, bisect. Also to make results more compareable it would make sense to use spring-headless.

maybe just wait for some feedback of BA players for current 98.0.

[MetalSucker]

Not sure it's actually related to the engine changes, but while playing http://zero-k.info/Missions/Detail/208 there's a certain point where the AI has a lot of units (if you don't destroy it fast enough) and the performance goes to 5fps on a decent computer, pretty sure it's not because of the graphics. Granted, you have to wait 20 minutes + on that particular map, but it does happen, the difference on the new engine (97.something git test ) being that it does appear to be a bit slower but the tanks don't get stuck on slopes anymore - the old engine did that a lot. I can't see if it's because of the AI, pathfinding or whatever else happens there. 64 bit linux.

+ /me willing to try and give any additional info you would need, but I need some guidance on how to do that.

HTH