2021-04-22 19:24 CEST

View Issue Details Jump to Notes ]
IDProjectCategoryView StatusLast Update
0006340Spring engineGeneralpublic2020-04-10 23:18
Assigned Tohokomoko 
Product Version104.0 +git 
Target Version105.0Fixed in Version 
Summary0006340: Significant reduction in FPS with scroll/zoom on maps with rough terrain
DescriptionSuspecting some changes in ROAM, that appeared shortly after version 1250 (not sure which version exactly).
While on powerful PC it might not be very noticeable, on a laptop it renders game unbearable.
Steps To ReproduceSelect a map with significant roughness, e.g. Stronghold.
Additional InformationSame issue was addressed by Zero-K players.
TagsNo tags attached.
Checked infolog.txt for ErrorsIrrelevant
Attached Files

has duplicate 0006348closed There's camera stuttering on larger maps 



Krogoth (reporter)

To avoid misunderstanding:
v 1250 works fine, no lag.


Anarchid (reporter)

There was some investigation WRT this around ZK.

This is quite definitely ROAM.

Moving the camera while viewing the cliffy borders on the map Onyx Cauldron with the ROAM mesh drawer ON results in a massive FPS drop.

Panning with the legacy mesh drawer results in a very slightly lower FPS when camera is static, but completely smooth panning - an overall massive increase in quality.

The ZK investigation has not resulted in an engine issue because there's a user-side workaround in the form of the "/mapmeshdrawer" command. If the issues are truly unbearable, a game may include a widget to automatically switch to the legacy setting when on a map that is known to perform badly with ROAM.


Krogoth (reporter)

From my experience, all maps perform badly with new ROAM.
Switching to legacy could be a last resort, but I really like how ROAM looks, more than legacy.
I didn't investigate the difference between new and old, but the old one worked well, correct?


Krogoth (reporter)

*between new and old ENGINE code


Anarchid (reporter)

I don't know if 1250 works correct.

These statements are making me confused:

> v 1250 works fine, no lag.
> I didn't investigate the difference between new and old, but the old one worked well, correct?


Krogoth (reporter)

it does.
meant difference in code.
bug maybe anywhere, theoretically not in ROAM code, so I can't 100% say that ROAM code is bad.


Kloot (developer)

I got tired of seeing 0006220 pop up over and over again and disabled threaded mesh tessellation in ROAM to get rid of it.

Anyone here is free to volunteer to fix the actual bug and restore performance.


Krogoth (reporter)

Ill try


Krogoth (reporter)

While trying to reproduce the crash..
How normal is "cannot recreate persistent storage buffer", VBO error?


Krogoth (reporter)



Krogoth (reporter)

Found it!
Look at the middle diamond (right quarter of left patch, left quarter of right patch)
If left triangle of it is getting split and when it tries to split base neighbour and goes out of nodes, it leaves childs hanging with unset neighbours to the right side.
Then, if in the next iteration right triangle (of the center diamond) goes out of nodes right away, it doesn't connect hanging pointers.
Finally, if tesselation from bottom or top causes left triangle to split (via multiple splits such that ultimately cause Patch:line 240 executed), then splitting left triangle's child nodes will use unset neighbours,
thus access violation.
Obviously, it doesn't happen like this if single thread.


Krogoth (reporter)

Principally: the problem is leaving pointers hanging between splits of triangle and its base neighbour.


IceXuick (reporter)



Kloot (developer)

thanks for the legwork, I'll see what I can do soon.


IceXuick (reporter)

Any news on this bug and or possible fix?


Krogoth (reporter)

Just in case if Kloot was busy and to ease things up for him,
position in code:
Setting tlc->RightNeighbour and trc->RightNeighbour only when branch, thus leaving them hanging if both branch this and next iter go out of nodes.
One of solutions could be eliminating childs of empty base neighbour (in addition to own), but there can be many solutions.


Krogoth (reporter)



Krogoth (reporter)

*not empty base neighbour, but one with not valid childs
Where is My EDIT Button? :X


IceXuick (reporter)

Latest 1455 release made this problem even worse...


Google_Frog (reporter)

I don't know what caused it, but there is a performance regression between 104.0.1-1435-g79d77ca and 104.0.1-1455-g39f8fbd. My benchmark shows a difference of 19% between the average update time. The benchmark does not move the camera, however there is some terrain deformation. I'll bisect.


IceXuick (reporter)

Awesome! I hope you can find it.
I'm currently working on a new map and i'm already with-holding myself from adding too much detail to increase the chance it doesn't stutter.

which.. could be both good and bad :)
(good = more optimized cleaner maps / bad = i'm constrained in use of creative/detailed maps)


IceXuick (reporter)

I've tried to pin down what might be the exact cause in my TMA20 map. I've been reducing (micro)details, smoothened rough areas, removed almost 90% of rocky/bumpy area's and rounded hard cliff edges.

I've done this in 7 steps, everytime removing more and more.
Despite it having effect, even after the 7th lowering of details, the stuttering is still not gone, though like 75%-ish less.

I would like to put this as a good test case, where you can see/test both maps (high quality one vs very tuned down version) - but both still have the stuttering. I think it's a great loss that this stuttering happens, because it severely limits mappers.

You can only get decent framerates/no stutter if you have very simple maps. When you want some more organic stuff, and more microdetails (which greatly enhance the more closeup battles - which i think Spring definately could use) then you quickly go into stutter-fest-mode.

Currently rounding up my 7th revision into a separate map that i'll upload as an 'optimized' version. But i hope you can really look into this, because this is just not something you'd want.


IceXuick (reporter)

PS. you can test the 'optimized' TMA20XR 2.1 here: https://springfiles.com/spring/spring-maps/tma-20-xr

And also my map DSDR (Delta Siege Dry Remake) does have some stutter issues as well. Is it possible that ROAM has problems with erosion (thinner deeper lines in the surface)?

Download lastest DSDR 3.5 here: https://springfiles.com/spring/spring-maps/dsdr

I'll try a test where i blur/smooth out only these erosion-lines and see what happens...


IceXuick (reporter)

Update: Smoothing out the erosion-lines and overall smoothing edges did improve fps/stutter issues, but this varies between systems, and still is noticeable, especially on lower-end systems.

So (if the above from Krogoth is correct) ROAM has some issues.

There's another issue about cratereffect (the dynamic cratering/terraindeforming) that results in large decreases of fps/smoothness, especially noticeable in fast succession like mortar volleys, missile barrages or D-guns.

Might these perf-issues also be related to ROAM not performing optimal? I can remember slightly that these deforms worked pretty well and fast/smooth in older Spring RTS.

I hope someone of you can dive into this and try to fix this, because i think it's a pretty fundamental (and unique) game-aspect/feature that should work as good as possible.


IceXuick (reporter)

I can confirm v 1250 to be practically flawless and stutter free.
All maps mentioned above work very well with the 1250 release in both w32 and w64 versions.

@Kloot i heard you are the one who can most probably fix this? Could you please look into this?


Kloot (developer)

Last edited: 2020-02-03 11:50

View 2 revisions

I don't actively dev Spring much anymore, but try the following:

  download the last available engine build (1463-gb19d6f5)
  enter "/maptesselusethreads 0 0" into the console; compare performance against both 1250 and 1435-g79d77ca
  enter "/maptesselusethreads 1 1" into the console and repeat


IceXuick (reporter)

in latest 1463 and 1250 these things don't make any difference.
Is that possible?


Kloot (developer)

Last edited: 2020-02-03 17:40

View 2 revisions

only if you included the ""'s when entering /maptesselusethreads 0 0; there should be a very detectable change in CPU usage afterward.

fyi this command does not exist in 1250, I added it to 1455+ when fixing 0006220.


IceXuick (reporter)

I'll ask Floris. I've tried every combination wiht/without "" and with 0, 0 0, with ;, with cheats, but i don't see any difference/thing happen.


Floris (reporter)

i didnt notice anythign either,
but i made the game now also set /mapmeshdrawer 1 for best performance in the meantime


IceXuick (reporter)

Downside is that without cam-movement - fps is like 40% lower with meshdrawer 1 (basic mesh) set on quality bias 5 (comparable with grounddetail of 130 with meshdrawer 2 = ROAM)


IceXuick (reporter)

And this only is an issue with larger maps, or maps that have more height variation - so setting globally to meshdrawer 1 would affect everyone that plays other or smaller maps too much.


IceXuick (reporter)

@Kloot - did you read Korogths notes? He said he found the problem, probably with the multithreading. Can't you use this for a solution? Or perhaps tell us how to exactly get this fixed in BAR?


Floris (reporter)

btw, even when using the lowest grounddetail of 4 on roam, the fps drop when moving camera is nearly the same as with highest detail


IceXuick (reporter)

yeah - but in 1250 this is no issue at all - so something went wrong/broke between then and now.


Krogoth (reporter)

all is known, just needs someone's time investment to write actual fix
i am busy myself for now
as for maptesselfhleawufhie - no visual difference, confirm


Krogoth (reporter)

*noticeable at least


IceXuick (reporter)

The speeds on 1250 are pretty insanely good compared to 1453-55-63... @Krogoth do you think you could bring this back with this fix?


Kloot (developer)

Last edited: 2020-02-04 22:14

View 3 revisions

1) 0006220 should be fixed in 1455

2) the regression between 104.0.1-1435-g79d77ca and 104.0.1-1455-g39f8fbd (0006362) should be fixed in 1463

3) "even when using the lowest grounddetail of 4 on roam, the fps drop when moving camera is nearly the same as with highest detail" and "As for the ROAM/Basic map mesh discussion - both are severely impacted by this. So as well for ROAM as for the other Mapmesh techniques this decrease happens." (from 0006362) imply there is a separate issue unrelated to ROAM which still exists in 1463

4) I have no time to investigate this soon


IceXuick (reporter)

Hi @Kloot,

Thanks for your time and investigations & fixes.

I hope you can find time somewhere because i think the map-mesh situation isn't good.


Google_Frog (reporter)

Here is a video comparing 1463 and 1453: https://www.youtube.com/watch?v=pkOmFqXgdkE

maptesselusethreads does not appear to have any effect, and 1453 is noticeably worse than 1463 when moving the camera.


Google_Frog (reporter)

104.0.1-1454-g3c0e4e1 has good performance and has bad performance 104.0.1-1455-g39f8fbd.


Google_Frog (reporter)

Nevermind, I was mistaken. I ran some benchmarks on various engine versions. This benchmark zooms the map in and out rapidly: https://www.youtube.com/watch?v=lNhcRp13lto


Google_Frog (reporter)

I compared the interpretation of the terrain detail setting with the changes. Both versions appear to result in the same level of detail, so this is not a cause of the issue.


Google_Frog (reporter)

I made PRs https://github.com/spring/spring/pull/494 https://github.com/spring/spring/pull/495


abma (administrator)


1. do you agree that https://github.com/spring/spring/pull/504 makes https://github.com/spring/spring/pull/495 obsolete? (for me it looked so...)

2. if so: can the same change applied to maintenance?

(also, sorry, i should have squashed the pull request to a single commit)


Google_Frog (reporter)

They are probably obsolete but I won't be able to do the proper test until the same change is allied to maintenance.


Google_Frog (reporter)

*applied. And obsolescence is looking very likely from what I have seen on Discord.


Google_Frog (reporter)

I've run the benchmark. https://github.com/spring/spring/commit/69b06d03411fd97f0042a348d839e88d16bd6cb1 is an improvement over the worst of this bug, but it is still worse than 1435 and https://github.com/spring/spring/pull/497. The "1474 maptes00" run is 1474 with the addition of Spring.SendCommands("maptesselusethreads 0 0").


Google_Frog (reporter)

I have closed https://github.com/spring/spring/pull/497 because it conflicts with the changes.

https://github.com/spring/spring/pull/494 and https://github.com/spring/spring/pull/495 do not conflict and provide the fastest version of ROAM. Beherith has shown interest in modifying current ROAM so I think the best approach is to pull the single thread ROAM PRs. This makes it much easier to compare future work (including ivand's potential tessellation attempts) to a known performant version.

Even now there is the possibility that a performance regression in a different part of Spring snuck in between 1463 and 1474l. This seems unlikely from the commit log, but holding off on this will only make comparing performance harder.


Beherith (reporter)

Fixed in LOAM

-Issue History
Date Modified Username Field Change
2019-11-15 15:09 Krogoth New Issue
2019-11-15 15:13 Krogoth Note Added: 0020220
2019-11-15 15:17 Anarchid Note Added: 0020221
2019-11-15 16:04 Krogoth Note Added: 0020222
2019-11-15 16:05 Krogoth Note Added: 0020223
2019-11-15 16:09 Anarchid Note Added: 0020224
2019-11-15 16:12 Krogoth Note Added: 0020225
2019-11-15 16:36 Kloot Note Added: 0020226
2019-11-15 16:51 Krogoth Note Added: 0020227
2019-11-18 11:04 Krogoth Note Added: 0020240
2019-11-18 11:15 Krogoth Note Added: 0020241
2019-11-20 08:48 Kloot Relationship added has duplicate 0006348
2019-11-20 15:19 Krogoth Note Added: 0020251
2019-11-20 15:20 Krogoth Note Added: 0020252
2019-11-20 17:32 IceXuick Note Added: 0020253
2019-11-20 22:51 Kloot Note Added: 0020254
2019-11-28 15:23 IceXuick Note Added: 0020263
2019-11-29 06:29 Krogoth Note Added: 0020264
2019-11-29 06:30 Krogoth Note Added: 0020265
2019-11-29 06:37 Krogoth Note Added: 0020266
2019-12-03 22:54 IceXuick Note Added: 0020272
2019-12-30 14:48 abma Target Version => 105.0
2019-12-30 14:48 abma Checked infolog.txt for Errors Irrelevant => Irrelevant
2020-01-21 03:45 Google_Frog File Added: 1435_1455.zip
2020-01-21 03:45 Google_Frog Note Added: 0020306
2020-01-21 10:43 IceXuick Note Added: 0020310
2020-01-27 10:30 IceXuick Note Added: 0020311
2020-01-27 23:20 IceXuick Note Added: 0020312
2020-01-28 16:07 IceXuick Note Added: 0020313
2020-02-03 08:02 IceXuick Note Added: 0020318
2020-02-03 11:50 Kloot Note Added: 0020319
2020-02-03 11:50 Kloot Note Edited: 0020319 View Revisions
2020-02-03 15:59 IceXuick Note Added: 0020320
2020-02-03 17:19 Kloot Note Added: 0020321
2020-02-03 17:40 Kloot Note Edited: 0020321 View Revisions
2020-02-04 10:43 IceXuick Note Added: 0020323
2020-02-04 13:42 Floris Note Added: 0020324
2020-02-04 14:13 IceXuick Note Added: 0020326
2020-02-04 14:14 IceXuick Note Added: 0020327
2020-02-04 14:16 IceXuick Note Added: 0020328
2020-02-04 14:36 Floris Note Added: 0020329
2020-02-04 14:48 IceXuick Note Added: 0020330
2020-02-04 20:04 Krogoth Note Added: 0020333
2020-02-04 20:05 Krogoth Note Added: 0020334
2020-02-04 21:40 IceXuick Note Added: 0020335
2020-02-04 22:09 Kloot Note Added: 0020337
2020-02-04 22:12 Kloot Note Edited: 0020337 View Revisions
2020-02-04 22:14 Kloot Note Edited: 0020337 View Revisions
2020-02-05 01:06 IceXuick Note Added: 0020338
2020-02-11 12:44 Google_Frog Note Added: 0020346
2020-02-17 02:58 Google_Frog Note Added: 0020348
2020-02-17 05:23 Google_Frog File Added: 20200217_033223_dataFile.csv
2020-02-17 05:23 Google_Frog Note Added: 0020349
2020-02-17 05:33 Google_Frog File Added: terrainDetail.jpg
2020-02-17 05:33 Google_Frog Note Added: 0020350
2020-02-18 12:48 Google_Frog Note Added: 0020351
2020-03-26 22:28 abma Note Added: 0020367
2020-03-26 22:29 abma Status new => feedback
2020-03-27 03:12 Google_Frog Note Added: 0020371
2020-03-27 07:26 Google_Frog Note Added: 0020372
2020-03-28 23:41 Google_Frog File Added: 20200328_123752_dataFile.csv
2020-03-28 23:41 Google_Frog Note Added: 0020373
2020-03-28 23:58 Google_Frog Note Added: 0020374
2020-04-10 11:45 Beherith Note Added: 0020385
2020-04-10 23:18 hokomoko Assigned To => hokomoko
2020-04-10 23:18 hokomoko Status feedback => resolved
2020-04-10 23:18 hokomoko Resolution open => fixed
+Issue History