Notes about performance, 105.0 vs 106.0 vs bar105 engines

[raaar]

I've done some basic testing to compare performance of 105.0 with the latest official engine release and the latest BAR105 engine release.

MF v1.80
comet catcher redux v2
disable luaui, disable luarules
(use tab hotkey for zoom out top view)
vsync off


--- spring_105.0_windows-64
spawn 5000 aven_magnum a bit north from the center (they take up most of the map)
zoomed out top view (icons) : ~100 fps, ~0.9 speed
zoomed in a bit until the models appear (but with as many models visible as possible) : 4 fps, ~0.3 speed
zoom out top view (icons) then order them to move south : freezes for a second then drops to ~10 fps for a few seconds then to ~90 fps, ~0.15 speed
~3.05 GB ram usage

--- BAR105.105.1.1-784_windows-64
spawn 5000 aven_magnum a bit north from the center (they take up most of the map)
zoomed out top view (icons) : ~100 fps, 1.0 speed
zoomed in a bit until the models appear (but with as many models visible as possible) : 22 fps, 1.0 speed
zoom out top view (icons) then order them to move south : freezes for a second then drops to ~10 fps for a few seconds then to ~90fps, ~0.15 speed
~3.75 GB ram usage

NOTE: a few days ago I had tested BAR105.105.1.1-769 and it had about 2/3 as much fps when zoomed out and showing icons, that issue has been fixed by Ivand and now it has about the same fps in my pc

--- spring_106.0_windows-64
spawn 5000 aven_magnum a bit north from the center (they take up most of the map)
zoomed out top view (icons) : ~160 fps, ~0.9 speed
zoomed in a bit until the models appear (but with as many models visible as possible) : 30 fps, ~0.82 speed
zoom out top view (icons) then order them to move south : freezes for a second then drops to ~10 fps for a few seconds then to ~120fps, ~0.15 speed
~3.35 GB ram usage

NOTE: i wrote "freeze" when i try to move the units, but it's more like a ~1s delay between issuing the order and them moving



There are some differences and the 106.0 seems faster than the BAR105, but breaks compatibility. The performance differences between BAR105 and 106.0 may be due to feature differences.

A key difference is that I get 5x FPS or more from either 106.0 or BAR105 relative to 105.0 when just looking at a scene with hundreds of units/features, nice!

[Beherith]

Did you run it with /luaui disable, out of curiosity?

[raaar]

yes, otherwise i'd be unable to compare with 106.0 as it breaks with my lua code.

disable luaui, disable luarules

[Shruggoth]

I ran a similar test.
As an extra note in 106 I did /grounddetail 140 as MF auto-sets that upon initialization via widget.

--- spring_105.0_windows-64
zoomed out tab view (icons) : 45 fps
zoomed in until models just filled the screen: 3 fps
zoom out tab view (icons) then order them to move south : 1s freeze, few seconds of ~1fps, then ~10fps

--- spring_bar_.BAR105.105.1.1-769-g56f63fc_windows-64
zoomed out tab view (icons) : 32 fps
zoomed in until models just filled the screen: 15 fps
zoom out tab view (icons) then order them to move south : ~10fps (not jumpy like 105)

--- spring_106.0_windows-64
zoomed out tab view (icons) : 43 fps
zoomed in until models just filled the screen: ~23 fps
zoom out tab view (icons) then order them to move south : ~9 fps (not jumpy like 105)

Similar looking performance, but bar105 and 106 lack the hitching in movement like raaar experienced.

[raaar]

Another test, checking ram usage

MF v1.80
comet catcher redux v2
disable luaui, disable luarules

--- 105.0
2.75 GB

--- BAR105-769
3.42 GB

--- 106.0
3.06 GB


the new engines seem to consume more ram than 105.0, at least when the game starts, and BAR105 consumes the most.

EDIT: another player tested and got different results, but still showed more memory usage on the newer engines:

[19:20:07] <Shruggoth> ima check what my RAM does
[19:20:58] <Shruggoth> 2.5GiB on CCR, spring 105
[19:21:21] <Shruggoth> how much RAM do you have, ximes?
[19:21:37] <Shruggoth> ooh
[19:21:39] <Shruggoth> 3.1GiB on 769
[19:21:50] <Shruggoth> so it demands an extra 600MiB for me
[19:22:35] <Shruggoth> Somehow
[19:22:41] <Shruggoth> 106 uses 4.2GiB

[Beherith]

The new engines are x64, that naturally consumes more ram :)

[raaar]

hmm, but isn't 105.0 x64 as well?

Another thing i failed to notice on my first test is how much spring slowed down the game. I recorded fps, but not game speed.

[SeanHeron]

Thanks for doing this and describing your results!

Given what you describe, at least for me it'll prob be things other than performance that I'd mostly be considering (as I've said to some people in private already) if I need to make the choice (for example compatibility issues as you mentioned; or my guestimate of support and future development, etc. - though thats obviously a whole different can of worms).

Still, definitely nice to have some semi-objective numbers out there :).

[raaar]

Thanks, these are basic tests though. It'd be nice to do proper benchmarking.

I've updated the tests to use "vsync off", show the speed multipliers and ram usage and to use the latest BAR105 release which has a fix for a performance issue with rendering icons.