<abma> hey!
=== Current release branch ===
<abma> whats missing for a release?
<abma> zerver: is mt fixed / can a release be made?
<abma> or ..uhm, whats with rectangleoptimizer?
<abma> zerver, jK: DONGDONGDONG! wake up! ^^
<zerver> abma I think MT desync is fixed but a 1-2 days to test it would not hurt
==> Release soon maybe
=== Rectangle optimizer for unsynced heightmap updates ===
<jk> ding
<jk> claiming of rectangle optimizer is WRONG
<jk> and i angry to be forced to prove it - instead the bug reporter (note it wasn't reported here!) has to prove it with an example test case
<jk> +am
<zerver> hi
<jk> at the end zerver might thought it is slower cause he tested a DEBUG build versus a RELEASE one
<jk> (you rarelly run DEBUG builds iirc)
<zerver> debug builds are not very useful
<jk> see
<zerver> The claim is not wrong, what happens is that during big action 10000 rectangles end up in the optimizer
<zerver> And the optimizer is O(N^2) 10000*10000 = huge
<jk> I didn't put the SCOPED_TIMER for nothing around Optimize()
<jk> it uses 0 - yeah zero - time
<zerver> Maybe you turned off map deformation
<jk> no, the opposite
<jk> wrote a gadget that deforms whole map
<zerver> Use a real-world test instead. 2 x give all close by each other, share half to enemy
<jk> I EVEN RUN A DEMO OF 2 CAIS FIGHTING
<jk> I KNOW THAT IT IS FASTER!
<jk> YOU have to PROVE that it doesn't!
<jk> and you have to make a mantis ticket as anyone else (or a git comment ...)!
<jk> I didn't reverted you LuaHandle stuff either (and yeah it cost me much more time than you could ever imagine to work with it ...)
<jk> *your
<abma> communication is really required in such cases...
<zerver> caps lock is not required
<abma> zerver: can you be a bit more specific whats possible slower with the rectangleoptimizer?
<zerver> Yes, it hangs for a second or more when there is big action, large explosions
<jk> check the profilergraph
<zerver> while I'm building it let me explain one other issue. In UpdateDraw() You Optimize() the set of rectangles, then pick some rectangles (up to a certain max area) and send them to UpdateHeightMapUnsynced()
<zerver> The next UpdateDraw you optimize the remaining rectangles again, possible together with some newly added ones.
<jk> test shown that it is not worth to check if it got changed
<jk> *tests
<jk> as said Optimize() uses no time
<zerver> Okay, I'm inserting a printout of the unsyncedHeightMapUpdates.size() to show you.
<jk> that's before an Optimize()
<jk> cause the heightmapuodate is splitted into LOS resolution
<zerver> Before and after if u wish
<zerver> Although before is more interesting, since you algorithm is O(N^2)
<zerver> zerver.net/optimize.jpg
<zerver> 1.5 seconds spent optimizing already after a few big explosions
<jk> run a buildbot build
<zerver> [f=0001014] Warning: BEFORE unsyncedHeightMapUpdates.size() = 7748
<zerver> [f=0001015] Warning: AFTER unsyncedHeightMapUpdates.size() = 68
<zerver> Nice reduction, but it takes time
<abma> conclusion: rectangleoptimpizer optimizes speed in general but causes spikes / short lags?
<abma> (as it is)
<zerver> possibly
<zerver> I did not test the performance for a "low" action load
<zerver> I rand the buildbot build, 2.39 seconds spent optimizing
<zerver> *ran
<zerver> Just give all 2 times and selfd all units
<jk> as said try buildbot build
<zerver> this is
<zerver> 4 seconds spent optimizing, just selfd 2 give alls
<zerver> BA 7.68
<zerver> Spring already has issues with CPU spikes during big explosions, and we don't need any more of that IMO
<zerver> If whole team dies (last comm killed), your algorithm may take 30 seconds to optimize
<zerver> 4 give alls --> 20 seconds
<jk> you told yourself to use real world examples
<jk> give all isn't a realworld example
<zerver> THis is real world example, last comm killed
<zerver> Commander ends game...
<jk> last comm killed updates whole map (Optimize() isn't even called then)
<zerver> If so, still unacceptable. Some player rages, and the others get a 20 seconds lag.
<zerver> selfd 4 give alls --> 25 seconds updateheightmapoptimize and 23 seconds updateheightmapsynced
<zerver> Now I will compare to the old version
<jk> http://home.arcor.de/jkei/screen00363.jpg http://home.arcor.de/jkei/screen00364.jpg
<abma> whats unsyncedheigtmapunsynced2 ?
<jk> a counter at the bottom of UpdateHeightMapSynced around the #ifdef stuff
<abma> aah, ok
<jk> wanted to check if the LOSHeightMapUpdateLOSCheck etc. are slow cause of lvl2 cache misses
<jk> but they aren't :)
<zerver> Selfd 4 give alls, old version: http://zerver.net/optimize2.jpg
<zerver> Selfd 4 give alls, rectangleoptimizer version: http://zerver.net/optimize3.jpg
<zerver> 18+18 seconds vs 1 + 1 second
<jk> k a test with a DEBUG2 build gave a increased cpu usage
<jk> a RELWITHDEBINFO with a minor tweak in rectangle optimizer & 4 give all + selfd just used 1.9s (nothing special for such a selfd)
<jk> now testing w/o the tweak
<abma> jK: are you testing with BA or ZK?
<zerver> Btw reason that UpdateHeightmapSynced goes through the roof as well, is that you perform Optimize() inside the mutex, which is not desirable
<jk> BA currently (the cpu usage is MUCH less in ZK)
<jk> k test without the tweak only showed 3s (still dar away from your 30sec)
<abma> yep, zk seems to be less cpu intensive with ctrl+a / ctrl + d
<jk> I say you either use a DEBUG2 build or you are using MT and have mutex problems
<abma> related to heightmap updates
<jk> (still the tweak is usefull to have, cutting the cputime by 50% in extreme situations is always good)
<jk> but we are far away from the 30sec
<jk> and in normal situations (max 100 heightmap updates in a gameframe), it is ~50% faster than the old code
<zerver> Tried now with single threaded, 28 seconds to selfd 4 give alls
<zerver> You want demo?
<jk> yes
<zerver> http://zerver.net/bigboom.zip (exe file included)
<zerver> rofl it triggers hang detection
<zerver> abma plz test also
<abma> uhm, i'm on linux currently
<abma> let's see what wine + spring says...
<zerver> lol
<zerver> version is 7f0e736
<zerver> from the buildbot
<jk> with wine you won't got compareable results
<jk> just run the demo with linux build
<jk> (there is less to desync)
<jk> and if there is a desync it won't matter much
<jk> k got a 20 lag (didn't got it on pacman map)
<jk> *20s
<jk> possibly BA just sucks and has HUUUUUUGE explosion radii with less change on the terrain
<jk> while ZK has small radii but huge change
<jk> testing now with tweak
<jk> tweak redudced it to 1.6s
<jk> *reduced
<zerver> okay, can I test the tweak?
<jk> (tweak is just a `redundant` merge stage before overlap checking, to reduce the number of rectangles)
<zerver> nice
<jk> just copy the section under "//! Merge when possible"
<jk> and paste it a 2nd time above "//! Fix Overlap"
<zerver> k
<abma> hmm, ~19 sec lag here with rectangleoptimize...
<abma> ~1sec with the tweak
<abma> imo it feels more responsive like spring 88... (rectangleoptimize + tweak)
<abma> but thats very subjective
<zerver> yes,much better, but needs profiling to verify that it outperforms the previous solution
<abma> ~1sec in spring 88 with 4*give all + ctrl-a/d
<abma> so very similar :-/
<abma> just to understand it: the rectangle optimizer merges terrain changes?
<abma> this means, if it merges two terrain changes into one, only one lua event is triggered? (afaik seismincping)
<abma> ...only one instead of two
<zerver> yes something like that
<zerver> the total time spent optimize + unsyncedheightmapupdate with my demo is 2.5 + 1.5 = 4 seconds
<zerver> with the old version (no optimize +) unsyncedheightmapupdate = 2.5 seconds
<zerver> so although unsyncedheightmapupdate indeed is faster, the optimize takes away all performance benefit
<abma> hmmm, just for the curious
<abma> how is exposion-damage handled? is it like: explosion -> all things in range get damage (+ event), if something else explodes the same is done again?
<zerver> yeah
<abma> just thought about a similar optimization here...
<abma> afaik seismicping lua events aren't really used... damage is...
<abma> but no idea if thats possible, because of unit-ids
** <jk> left the channel( Quit ).
<abma> meh, seems its late
<zerver> he gave up :)
<zerver> imo we need to run a few regular ba games through the old + optimizer versions and if the optimizer is faster then I vote to add it, otherwise not. I can live with end-game explosion being slightly slower.
<abma> imo such optimazion really makes sense if it then triggers fewer lua events
<abma> maybe the optimazation itself can be optimized...
<zerver> the optimizer adds quite a bit of code complexity, so it MUST be noticably faster to be justified imo
<abma> yeah, thats true
<zerver> 10-20% faster, then no prob, go ahead and add it
<abma> i've to go to bed now... i'm afk till wednesday i guess
<abma> can you make minutes? if not i'll make them thursday or so
<zerver> yeah, sure
<zerver> gn
<abma> thanks! good night! :)
==> The tweaked rectangle optimizer can be re-added if it is faster in real-world games, need some compatible demos to test with
