Dev meeting minutes 2012-06-18

Dev meeting minutes 2012-06-18

Minutes of the meetings between Spring developers are archived here.
Post Reply
zerver
Spring Developer
Posts: 1358
Joined: 16 Dec 2006, 20:59

Dev meeting minutes 2012-06-18

Post by zerver »

Present: <abma>, <jk>, <zerver>


<abma> hey!



=== Current release branch ===
<abma> whats missing for a release?
<abma> zerver: is mt fixed / can a release be made?
<abma> or ..uhm, whats with rectangleoptimizer?
<abma> zerver, jK: DONGDONGDONG! wake up! ^^
<zerver> abma I think MT desync is fixed but a 1-2 days to test it would not hurt

==> Release soon maybe



=== Rectangle optimizer for unsynced heightmap updates ===
<jk> ding
<jk> claiming of rectangle optimizer is WRONG
<jk> and i angry to be forced to prove it - instead the bug reporter (note it wasn't reported here!) has to prove it with an example test case
<jk> +am
<zerver> hi
<jk> at the end zerver might thought it is slower cause he tested a DEBUG build versus a RELEASE one
<jk> (you rarelly run DEBUG builds iirc)
<zerver> debug builds are not very useful
<jk> see
<zerver> The claim is not wrong, what happens is that during big action 10000 rectangles end up in the optimizer
<zerver> And the optimizer is O(N^2) 10000*10000 = huge
<jk> I didn't put the SCOPED_TIMER for nothing around Optimize()
<jk> it uses 0 - yeah zero - time
<zerver> Maybe you turned off map deformation :P
<jk> no, the opposite
<jk> wrote a gadget that deforms whole map
<zerver> Use a real-world test instead. 2 x give all close by each other, share half to enemy
<jk> I EVEN RUN A DEMO OF 2 CAIS FIGHTING
<jk> I KNOW THAT IT IS FASTER!
<jk> YOU have to PROVE that it doesn't!
<jk> and you have to make a mantis ticket as anyone else (or a git comment ...)!
<jk> I didn't reverted you LuaHandle stuff either (and yeah it cost me much more time than you could ever imagine to work with it ...)
<jk> *your
<abma> communication is really required in such cases...
<zerver> caps lock is not required
<abma> zerver: can you be a bit more specific whats possible slower with the rectangleoptimizer?
<zerver> Yes, it hangs for a second or more when there is big action, large explosions
<jk> check the profilergraph
<zerver> while I'm building it let me explain one other issue. In UpdateDraw() You Optimize() the set of rectangles, then pick some rectangles (up to a certain max area) and send them to UpdateHeightMapUnsynced()
<zerver> The next UpdateDraw you optimize the remaining rectangles again, possible together with some newly added ones.
<jk> test shown that it is not worth to check if it got changed
<jk> *tests
<jk> as said Optimize() uses no time
<zerver> Okay, I'm inserting a printout of the unsyncedHeightMapUpdates.size() to show you.
<jk> that's before an Optimize()
<jk> cause the heightmapuodate is splitted into LOS resolution
<zerver> Before and after if u wish
<zerver> Although before is more interesting, since you algorithm is O(N^2)
<zerver> zerver.net/optimize.jpg
<zerver> 1.5 seconds spent optimizing already after a few big explosions
<jk> run a buildbot build
<zerver> [f=0001014] Warning: BEFORE unsyncedHeightMapUpdates.size() = 7748
<zerver> [f=0001015] Warning: AFTER unsyncedHeightMapUpdates.size() = 68
<zerver> Nice reduction, but it takes time
<abma> conclusion: rectangleoptimpizer optimizes speed in general but causes spikes / short lags?
<abma> (as it is)
<zerver> possibly
<zerver> I did not test the performance for a "low" action load
<zerver> I rand the buildbot build, 2.39 seconds spent optimizing
<zerver> *ran
<zerver> Just give all 2 times and selfd all units
<jk> as said try buildbot build
<zerver> this is
<zerver> 4 seconds spent optimizing, just selfd 2 give alls
<zerver> BA 7.68
<zerver> Spring already has issues with CPU spikes during big explosions, and we don't need any more of that IMO
<zerver> If whole team dies (last comm killed), your algorithm may take 30 seconds to optimize
<zerver> 4 give alls --> 20 seconds
<jk> you told yourself to use real world examples
<jk> give all isn't a realworld example
<zerver> THis is real world example, last comm killed
<zerver> Commander ends game...
<jk> last comm killed updates whole map (Optimize() isn't even called then)
<zerver> If so, still unacceptable. Some player rages, and the others get a 20 seconds lag.
<zerver> selfd 4 give alls --> 25 seconds updateheightmapoptimize and 23 seconds updateheightmapsynced
<zerver> Now I will compare to the old version
<jk> http://home.arcor.de/jkei/screen00363.jpg http://home.arcor.de/jkei/screen00364.jpg
<abma> whats unsyncedheigtmapunsynced2 ?
<jk> a counter at the bottom of UpdateHeightMapSynced around the #ifdef stuff
<abma> aah, ok
<jk> wanted to check if the LOSHeightMapUpdateLOSCheck etc. are slow cause of lvl2 cache misses
<jk> but they aren't :)
<zerver> Selfd 4 give alls, old version: http://zerver.net/optimize2.jpg
<zerver> Selfd 4 give alls, rectangleoptimizer version: http://zerver.net/optimize3.jpg
<zerver> 18+18 seconds vs 1 + 1 second
<jk> k a test with a DEBUG2 build gave a increased cpu usage
<jk> a RELWITHDEBINFO with a minor tweak in rectangle optimizer & 4 give all + selfd just used 1.9s (nothing special for such a selfd)
<jk> now testing w/o the tweak
<abma> jK: are you testing with BA or ZK?
<zerver> Btw reason that UpdateHeightmapSynced goes through the roof as well, is that you perform Optimize() inside the mutex, which is not desirable
<jk> BA currently (the cpu usage is MUCH less in ZK)
<jk> k test without the tweak only showed 3s (still dar away from your 30sec)
<abma> yep, zk seems to be less cpu intensive with ctrl+a / ctrl + d
<jk> I say you either use a DEBUG2 build or you are using MT and have mutex problems
<abma> related to heightmap updates
<jk> (still the tweak is usefull to have, cutting the cputime by 50% in extreme situations is always good)
<jk> but we are far away from the 30sec
<jk> and in normal situations (max 100 heightmap updates in a gameframe), it is ~50% faster than the old code
<zerver> Tried now with single threaded, 28 seconds to selfd 4 give alls
<zerver> You want demo?
<jk> yes
<zerver> http://zerver.net/bigboom.zip (exe file included)
<zerver> rofl it triggers hang detection
<zerver> abma plz test also
<abma> uhm, i'm on linux currently
<abma> let's see what wine + spring says...
<zerver> lol
<zerver> version is 7f0e736
<zerver> from the buildbot
<jk> with wine you won't got compareable results
<jk> just run the demo with linux build
<jk> (there is less to desync)
<jk> and if there is a desync it won't matter much
<jk> k got a 20 lag (didn't got it on pacman map)
<jk> *20s
<jk> possibly BA just sucks and has HUUUUUUGE explosion radii with less change on the terrain
<jk> while ZK has small radii but huge change
<jk> testing now with tweak
<jk> tweak redudced it to 1.6s
<jk> *reduced
<zerver> okay, can I test the tweak?
<jk> (tweak is just a `redundant` merge stage before overlap checking, to reduce the number of rectangles)
<zerver> nice
<jk> just copy the section under "//! Merge when possible"
<jk> and paste it a 2nd time above "//! Fix Overlap"
<zerver> k
<abma> hmm, ~19 sec lag here with rectangleoptimize...
<abma> ~1sec with the tweak
<abma> imo it feels more responsive like spring 88... (rectangleoptimize + tweak)
<abma> but thats very subjective
<zerver> yes,much better, but needs profiling to verify that it outperforms the previous solution
<abma> ~1sec in spring 88 with 4*give all + ctrl-a/d
<abma> so very similar :-/
<abma> just to understand it: the rectangle optimizer merges terrain changes?
<abma> this means, if it merges two terrain changes into one, only one lua event is triggered? (afaik seismincping)
<abma> ...only one instead of two
<zerver> yes something like that
<zerver> the total time spent optimize + unsyncedheightmapupdate with my demo is 2.5 + 1.5 = 4 seconds
<zerver> with the old version (no optimize +) unsyncedheightmapupdate = 2.5 seconds
<zerver> so although unsyncedheightmapupdate indeed is faster, the optimize takes away all performance benefit
<abma> hmmm, just for the curious
<abma> how is exposion-damage handled? is it like: explosion -> all things in range get damage (+ event), if something else explodes the same is done again?
<zerver> yeah
<abma> just thought about a similar optimization here...
<abma> afaik seismicping lua events aren't really used... damage is...
<abma> but no idea if thats possible, because of unit-ids
** <jk> left the channel( Quit ).
<abma> meh, seems its late
<zerver> he gave up :)
<zerver> imo we need to run a few regular ba games through the old + optimizer versions and if the optimizer is faster then I vote to add it, otherwise not. I can live with end-game explosion being slightly slower.
<abma> imo such optimazion really makes sense if it then triggers fewer lua events
<abma> maybe the optimazation itself can be optimized...
<zerver> the optimizer adds quite a bit of code complexity, so it MUST be noticably faster to be justified imo
<abma> yeah, thats true
<zerver> 10-20% faster, then no prob, go ahead and add it
<abma> i've to go to bed now... i'm afk till wednesday i guess
<abma> can you make minutes? if not i'll make them thursday or so
<zerver> yeah, sure
<zerver> gn
<abma> thanks! good night! :)

==> The tweaked rectangle optimizer can be re-added if it is faster in real-world games, need some compatible demos to test with
Google_Frog
Moderator
Posts: 2464
Joined: 12 Oct 2007, 09:24

Re: Dev meeting minutes 2012-06-18

Post by Google_Frog »

Is there a lua callin for terrain changed? Also explosions have speed which adds a per unit delay to explosion-damage.
zerver
Spring Developer
Posts: 1358
Joined: 16 Dec 2006, 20:59

Re: Dev meeting minutes 2012-06-18

Post by zerver »

I made a demo for testing the rectangleoptimizer in something that resembles a real game. It has some jeffy spam and berthas.
http://zerver.net/rotest.zip

These are my results (25 minutes game):

Without optimizer: 5.4 sec updateheightmapunsynced

With optimizer: 4.5 sec updateheightmapunsynced + 0.3 sec updateheightmapoptimize

So 4.8 sec vs 5.4 sec is something like 11% faster.

Considering that the optimizer is proven to be slower (by one second or more) with huge explosions, this 0.6 seconds gain is not very impressive. I'm not going to revert it, but I question if it is worth the added code complexity.
User avatar
jK
Spring Developer
Posts: 2299
Joined: 28 Jun 2007, 07:30

Re: Dev meeting minutes 2012-06-18

Post by jK »

Optimizer now uses even in your stress tests ~zero usage, cause of https://github.com/spring/spring/commit ... f49696b644

another tip: run it as non-spectator and you will see a HUGE performance increase
Last edited by jK on 21 Jun 2012, 02:03, edited 1 time in total.
User avatar
jK
Spring Developer
Posts: 2299
Joined: 28 Jun 2007, 07:30

Re: Dev meeting minutes 2012-06-18

Post by jK »

Google_Frog wrote:Is there a lua callin for terrain changed? Also explosions have speed which adds a per unit delay to explosion-damage.
yes, UnsyncedHeightMapUpdate()
User avatar
Anarchid
Posts: 1384
Joined: 30 Nov 2008, 04:31

Re: Dev meeting minutes 2012-06-18

Post by Anarchid »

MT is fixed? yay? :0
Google_Frog
Moderator
Posts: 2464
Joined: 12 Oct 2007, 09:24

Re: Dev meeting minutes 2012-06-18

Post by Google_Frog »

89.0 soon? I heard that murmurs of release a few weeks ago.
User avatar
Forboding Angel
Evolution RTS Developer
Posts: 14673
Joined: 17 Nov 2005, 02:43

Re: Dev meeting minutes 2012-06-18

Post by Forboding Angel »

Anarchid wrote:MT is fixed? yay? :0
Define "Fixed".
Post Reply

Return to “Meeting Minutes”