View Issue Details
| ID | Project | Category | View Status | Date Submitted | Last Update |
|---|---|---|---|---|---|
| 0006340 | Spring engine | General | public | 2019-11-15 15:09 | 2020-04-10 23:18 |
| Reporter | Krogoth | Assigned To | hokomoko | ||
| Priority | normal | Severity | major | Reproducibility | always |
| Status | resolved | Resolution | fixed | ||
| Product Version | 104.0 +git | ||||
| Target Version | 105.0 | ||||
| Summary | 0006340: Significant reduction in FPS with scroll/zoom on maps with rough terrain | ||||
| Description | Suspecting some changes in ROAM, that appeared shortly after version 1250 (not sure which version exactly). While on powerful PC it might not be very noticeable, on a laptop it renders game unbearable. | ||||
| Steps To Reproduce | Select a map with significant roughness, e.g. Stronghold. Scroll/zoom. | ||||
| Additional Information | Same issue was addressed by Zero-K players. | ||||
| Tags | No tags attached. | ||||
| Attached Files | |||||
| Checked infolog.txt for Errors | Irrelevant | ||||
| has duplicate | 0006348 | closed | There's camera stuttering on larger maps |
|
|
To avoid misunderstanding: v 1250 works fine, no lag. |
|
|
There was some investigation WRT this around ZK. This is quite definitely ROAM. Moving the camera while viewing the cliffy borders on the map Onyx Cauldron with the ROAM mesh drawer ON results in a massive FPS drop. Panning with the legacy mesh drawer results in a very slightly lower FPS when camera is static, but completely smooth panning - an overall massive increase in quality. The ZK investigation has not resulted in an engine issue because there's a user-side workaround in the form of the "/mapmeshdrawer" command. If the issues are truly unbearable, a game may include a widget to automatically switch to the legacy setting when on a map that is known to perform badly with ROAM. |
|
|
Hmm... From my experience, all maps perform badly with new ROAM. Switching to legacy could be a last resort, but I really like how ROAM looks, more than legacy. I didn't investigate the difference between new and old, but the old one worked well, correct? |
|
|
*between new and old ENGINE code |
|
|
I don't know if 1250 works correct. These statements are making me confused: > v 1250 works fine, no lag. > I didn't investigate the difference between new and old, but the old one worked well, correct? |
|
|
it does. meant difference in code. bug maybe anywhere, theoretically not in ROAM code, so I can't 100% say that ROAM code is bad. |
|
|
I got tired of seeing 0006220 pop up over and over again and disabled threaded mesh tessellation in ROAM to get rid of it. Anyone here is free to volunteer to fix the actual bug and restore performance. |
|
|
Ill try |
|
|
While trying to reproduce the crash.. How normal is "cannot recreate persistent storage buffer", VBO error? |
|
|
nvm |
|
|
Found it! ------ |X|X| ------ Look at the middle diamond (right quarter of left patch, left quarter of right patch) If left triangle of it is getting split and when it tries to split base neighbour and goes out of nodes, it leaves childs hanging with unset neighbours to the right side. Then, if in the next iteration right triangle (of the center diamond) goes out of nodes right away, it doesn't connect hanging pointers. Finally, if tesselation from bottom or top causes left triangle to split (via multiple splits such that ultimately cause Patch:line 240 executed), then splitting left triangle's child nodes will use unset neighbours, thus access violation. Obviously, it doesn't happen like this if single thread. |
|
|
Principally: the problem is leaving pointers hanging between splits of triangle and its base neighbour. |
|
|
Great! |
|
|
thanks for the legwork, I'll see what I can do soon. |
|
|
Any news on this bug and or possible fix? |
|
|
Just in case if Kloot was busy and to ease things up for him, position in code: https://github.com/spring/spring/blob/a17a11a58b06da8bc8510d1eb3cd92b559ea694c/rts/Map/SMF/ROAM/Patch.cpp#L293 Setting tlc->RightNeighbour and trc->RightNeighbour only when branch, thus leaving them hanging if both branch this and next iter go out of nodes. One of solutions could be eliminating childs of empty base neighbour (in addition to own), but there can be many solutions. |
|
|
*trc->LEftNeighbour |
|
|
*not empty base neighbour, but one with not valid childs Where is My EDIT Button? :X |
|
|
Latest 1455 release made this problem even worse... |
|
|
I don't know what caused it, but there is a performance regression between 104.0.1-1435-g79d77ca and 104.0.1-1455-g39f8fbd. My benchmark shows a difference of 19% between the average update time. The benchmark does not move the camera, however there is some terrain deformation. I'll bisect. |
|
|
Awesome! I hope you can find it. I'm currently working on a new map and i'm already with-holding myself from adding too much detail to increase the chance it doesn't stutter. which.. could be both good and bad :) (good = more optimized cleaner maps / bad = i'm constrained in use of creative/detailed maps) |
|
|
I've tried to pin down what might be the exact cause in my TMA20 map. I've been reducing (micro)details, smoothened rough areas, removed almost 90% of rocky/bumpy area's and rounded hard cliff edges. I've done this in 7 steps, everytime removing more and more. Despite it having effect, even after the 7th lowering of details, the stuttering is still not gone, though like 75%-ish less. I would like to put this as a good test case, where you can see/test both maps (high quality one vs very tuned down version) - but both still have the stuttering. I think it's a great loss that this stuttering happens, because it severely limits mappers. You can only get decent framerates/no stutter if you have very simple maps. When you want some more organic stuff, and more microdetails (which greatly enhance the more closeup battles - which i think Spring definately could use) then you quickly go into stutter-fest-mode. Currently rounding up my 7th revision into a separate map that i'll upload as an 'optimized' version. But i hope you can really look into this, because this is just not something you'd want. |
|
|
PS. you can test the 'optimized' TMA20XR 2.1 here: https://springfiles.com/spring/spring-maps/tma-20-xr And also my map DSDR (Delta Siege Dry Remake) does have some stutter issues as well. Is it possible that ROAM has problems with erosion (thinner deeper lines in the surface)? Download lastest DSDR 3.5 here: https://springfiles.com/spring/spring-maps/dsdr I'll try a test where i blur/smooth out only these erosion-lines and see what happens... |
|
|
Update: Smoothing out the erosion-lines and overall smoothing edges did improve fps/stutter issues, but this varies between systems, and still is noticeable, especially on lower-end systems. So (if the above from Krogoth is correct) ROAM has some issues. There's another issue about cratereffect (the dynamic cratering/terraindeforming) that results in large decreases of fps/smoothness, especially noticeable in fast succession like mortar volleys, missile barrages or D-guns. Might these perf-issues also be related to ROAM not performing optimal? I can remember slightly that these deforms worked pretty well and fast/smooth in older Spring RTS. I hope someone of you can dive into this and try to fix this, because i think it's a pretty fundamental (and unique) game-aspect/feature that should work as good as possible. |
|
|
I can confirm v 1250 to be practically flawless and stutter free. All maps mentioned above work very well with the 1250 release in both w32 and w64 versions. @Kloot i heard you are the one who can most probably fix this? Could you please look into this? |
|
|
I don't actively dev Spring much anymore, but try the following: download the last available engine build (1463-gb19d6f5) enter "/maptesselusethreads 0 0" into the console; compare performance against both 1250 and 1435-g79d77ca enter "/maptesselusethreads 1 1" into the console and repeat |
|
|
in latest 1463 and 1250 these things don't make any difference. Is that possible? |
|
|
only if you included the ""'s when entering /maptesselusethreads 0 0; there should be a very detectable change in CPU usage afterward. fyi this command does not exist in 1250, I added it to 1455+ when fixing 0006220. |
|
|
I'll ask Floris. I've tried every combination wiht/without "" and with 0, 0 0, with ;, with cheats, but i don't see any difference/thing happen. |
|
|
i didnt notice anythign either, but i made the game now also set /mapmeshdrawer 1 for best performance in the meantime |
|
|
Downside is that without cam-movement - fps is like 40% lower with meshdrawer 1 (basic mesh) set on quality bias 5 (comparable with grounddetail of 130 with meshdrawer 2 = ROAM) |
|
|
And this only is an issue with larger maps, or maps that have more height variation - so setting globally to meshdrawer 1 would affect everyone that plays other or smaller maps too much. |
|
|
@Kloot - did you read Korogths notes? He said he found the problem, probably with the multithreading. Can't you use this for a solution? Or perhaps tell us how to exactly get this fixed in BAR? |
|
|
btw, even when using the lowest grounddetail of 4 on roam, the fps drop when moving camera is nearly the same as with highest detail |
|
|
yeah - but in 1250 this is no issue at all - so something went wrong/broke between then and now. |
|
|
all is known, just needs someone's time investment to write actual fix i am busy myself for now as for maptesselfhleawufhie - no visual difference, confirm |
|
|
*noticeable at least |
|
|
The speeds on 1250 are pretty insanely good compared to 1453-55-63... @Krogoth do you think you could bring this back with this fix? |
|
|
1) 0006220 should be fixed in 1455 2) the regression between 104.0.1-1435-g79d77ca and 104.0.1-1455-g39f8fbd (0006362) should be fixed in 1463 3) "even when using the lowest grounddetail of 4 on roam, the fps drop when moving camera is nearly the same as with highest detail" and "As for the ROAM/Basic map mesh discussion - both are severely impacted by this. So as well for ROAM as for the other Mapmesh techniques this decrease happens." (from 0006362) imply there is a separate issue unrelated to ROAM which still exists in 1463 4) I have no time to investigate this soon |
|
|
Hi @Kloot, Thanks for your time and investigations & fixes. I hope you can find time somewhere because i think the map-mesh situation isn't good. |
|
|
Here is a video comparing 1463 and 1453: https://www.youtube.com/watch?v=pkOmFqXgdkE maptesselusethreads does not appear to have any effect, and 1453 is noticeably worse than 1463 when moving the camera. |
|
|
104.0.1-1454-g3c0e4e1 has good performance and has bad performance 104.0.1-1455-g39f8fbd. |
|
|
Nevermind, I was mistaken. I ran some benchmarks on various engine versions. This benchmark zooms the map in and out rapidly: https://www.youtube.com/watch?v=lNhcRp13lto |
|
|
I compared the interpretation of the terrain detail setting with the changes. Both versions appear to result in the same level of detail, so this is not a cause of the issue. |
|
|
I made PRs https://github.com/spring/spring/pull/494 https://github.com/spring/spring/pull/495 |
|
|
@Google_Frog: 1. do you agree that https://github.com/spring/spring/pull/504 makes https://github.com/spring/spring/pull/495 obsolete? (for me it looked so...) 2. if so: can the same change applied to maintenance? (also, sorry, i should have squashed the pull request to a single commit) |
|
|
They are probably obsolete but I won't be able to do the proper test until the same change is allied to maintenance. |
|
|
*applied. And obsolescence is looking very likely from what I have seen on Discord. |
|
|
I've run the benchmark. https://github.com/spring/spring/commit/69b06d03411fd97f0042a348d839e88d16bd6cb1 is an improvement over the worst of this bug, but it is still worse than 1435 and https://github.com/spring/spring/pull/497. The "1474 maptes00" run is 1474 with the addition of Spring.SendCommands("maptesselusethreads 0 0"). |
|
|
I have closed https://github.com/spring/spring/pull/497 because it conflicts with the changes. https://github.com/spring/spring/pull/494 and https://github.com/spring/spring/pull/495 do not conflict and provide the fastest version of ROAM. Beherith has shown interest in modifying current ROAM so I think the best approach is to pull the single thread ROAM PRs. This makes it much easier to compare future work (including ivand's potential tessellation attempts) to a known performant version. Even now there is the possibility that a performance regression in a different part of Spring snuck in between 1463 and 1474l. This seems unlikely from the commit log, but holding off on this will only make comparing performance harder. |
|
|
Fixed in LOAM |
| Date Modified | Username | Field | Change |
|---|---|---|---|
| 2019-11-15 15:09 | Krogoth | New Issue | |
| 2019-11-15 15:13 | Krogoth | Note Added: 0020220 | |
| 2019-11-15 15:17 | Anarchid | Note Added: 0020221 | |
| 2019-11-15 16:04 | Krogoth | Note Added: 0020222 | |
| 2019-11-15 16:05 | Krogoth | Note Added: 0020223 | |
| 2019-11-15 16:09 | Anarchid | Note Added: 0020224 | |
| 2019-11-15 16:12 | Krogoth | Note Added: 0020225 | |
| 2019-11-15 16:36 | Kloot | Note Added: 0020226 | |
| 2019-11-15 16:51 | Krogoth | Note Added: 0020227 | |
| 2019-11-18 11:04 | Krogoth | Note Added: 0020240 | |
| 2019-11-18 11:15 | Krogoth | Note Added: 0020241 | |
| 2019-11-20 08:48 | Kloot | Relationship added | has duplicate 0006348 |
| 2019-11-20 15:19 | Krogoth | Note Added: 0020251 | |
| 2019-11-20 15:20 | Krogoth | Note Added: 0020252 | |
| 2019-11-20 17:32 | IceXuick | Note Added: 0020253 | |
| 2019-11-20 22:51 | Kloot | Note Added: 0020254 | |
| 2019-11-28 15:23 | IceXuick | Note Added: 0020263 | |
| 2019-11-29 06:29 | Krogoth | Note Added: 0020264 | |
| 2019-11-29 06:30 | Krogoth | Note Added: 0020265 | |
| 2019-11-29 06:37 | Krogoth | Note Added: 0020266 | |
| 2019-12-03 22:54 | IceXuick | Note Added: 0020272 | |
| 2019-12-30 14:48 | abma | Target Version | => 105.0 |
| 2019-12-30 14:48 | abma | Checked infolog.txt for Errors | Irrelevant => Irrelevant |
| 2020-01-21 03:45 | Google_Frog | File Added: 1435_1455.zip | |
| 2020-01-21 03:45 | Google_Frog | Note Added: 0020306 | |
| 2020-01-21 10:43 | IceXuick | Note Added: 0020310 | |
| 2020-01-27 10:30 | IceXuick | Note Added: 0020311 | |
| 2020-01-27 23:20 | IceXuick | Note Added: 0020312 | |
| 2020-01-28 16:07 | IceXuick | Note Added: 0020313 | |
| 2020-02-03 08:02 | IceXuick | Note Added: 0020318 | |
| 2020-02-03 11:50 | Kloot | Note Added: 0020319 | |
| 2020-02-03 11:50 | Kloot | Note Edited: 0020319 | |
| 2020-02-03 15:59 | IceXuick | Note Added: 0020320 | |
| 2020-02-03 17:19 | Kloot | Note Added: 0020321 | |
| 2020-02-03 17:40 | Kloot | Note Edited: 0020321 | |
| 2020-02-04 10:43 | IceXuick | Note Added: 0020323 | |
| 2020-02-04 13:42 | Floris | Note Added: 0020324 | |
| 2020-02-04 14:13 | IceXuick | Note Added: 0020326 | |
| 2020-02-04 14:14 | IceXuick | Note Added: 0020327 | |
| 2020-02-04 14:16 | IceXuick | Note Added: 0020328 | |
| 2020-02-04 14:36 | Floris | Note Added: 0020329 | |
| 2020-02-04 14:48 | IceXuick | Note Added: 0020330 | |
| 2020-02-04 20:04 | Krogoth | Note Added: 0020333 | |
| 2020-02-04 20:05 | Krogoth | Note Added: 0020334 | |
| 2020-02-04 21:40 | IceXuick | Note Added: 0020335 | |
| 2020-02-04 22:09 | Kloot | Note Added: 0020337 | |
| 2020-02-04 22:12 | Kloot | Note Edited: 0020337 | |
| 2020-02-04 22:14 | Kloot | Note Edited: 0020337 | |
| 2020-02-05 01:06 | IceXuick | Note Added: 0020338 | |
| 2020-02-11 12:44 | Google_Frog | Note Added: 0020346 | |
| 2020-02-17 02:58 | Google_Frog | Note Added: 0020348 | |
| 2020-02-17 05:23 | Google_Frog | File Added: 20200217_033223_dataFile.csv | |
| 2020-02-17 05:23 | Google_Frog | Note Added: 0020349 | |
| 2020-02-17 05:33 | Google_Frog | File Added: terrainDetail.jpg | |
| 2020-02-17 05:33 | Google_Frog | Note Added: 0020350 | |
| 2020-02-18 12:48 | Google_Frog | Note Added: 0020351 | |
| 2020-03-26 22:28 | abma | Note Added: 0020367 | |
| 2020-03-26 22:29 | abma | Status | new => feedback |
| 2020-03-27 03:12 | Google_Frog | Note Added: 0020371 | |
| 2020-03-27 07:26 | Google_Frog | Note Added: 0020372 | |
| 2020-03-28 23:41 | Google_Frog | File Added: 20200328_123752_dataFile.csv | |
| 2020-03-28 23:41 | Google_Frog | Note Added: 0020373 | |
| 2020-03-28 23:58 | Google_Frog | Note Added: 0020374 | |
| 2020-04-10 11:45 | Beherith | Note Added: 0020385 | |
| 2020-04-10 23:18 | hokomoko | Assigned To | => hokomoko |
| 2020-04-10 23:18 | hokomoko | Status | feedback => resolved |
| 2020-04-10 23:18 | hokomoko | Resolution | open => fixed |
