[Request] Feedback on multithreaded simulation

[zerver]

https://github.com/spring/spring/commits/MTsim

What I did:
1. The pathfinder runs in an asynchronous thread in parallel with the rest of the simulation. This means that when units request paths, it just gives them path IDs, but no path yet. Less than 1 game frame later the path thread has computed the path, and the unit starts moving.

2. The movetype update and slowupdate is multithreaded. It updates all the movetypes in a random order, but it still syncs, because all the operations that would desync are delayed and run by a single thread afterwards.

3. Some additional (minor) multithreading of QuadField, Radar and LOS update.

1 and 2 depend on a system of "stable data" to make sure that unit data read by the worker threads is not changed suddenly by another thread. Any volatile data read would naturally desync the game.

The good:
Performance! (some examples below)
Random idling units: >100% faster
Patrolling fighters: >40% faster
Lots of collisions and pathfinding in large unit squads: >200% faster
Idling fighters: >20% faster

So basically this means twice as big battles if you have a quad core. You can "give all" twice as much units before the CPU is maxed out.

The bad:
It makes the code more complex
Units are a bit more likely to get stuck (fixable)
Some remaining bugs, and most likely desyncs (fixable)

If you have a quad core, please test and give feedback.
http://springrts.com/dl/buildbot/defaul ... -gad8971f/

Recommended settings for 8 logical CPUs:

Code: Select all

MultiThreadCount = 6
SetCoreAffinity = 1
SetCoreAffinitySim = 16
SetCoreAffinitySimMT = 204
SetCoreAffinityRenderMT = 204
SetCoreAffinityPath = 204
SimThreadCount = 5

[Silentwings]

Looks fantastic - I've downloaded and will test over next few days. Do you want bugs reported here or on mantis?

[Forboding Angel]

Does it work on non-ba?

[abma]

- some ai's seems to not work (at least RAI spams warnings)
- rts/System/Sync/DesyncDetector.cpp/.h should be removed and the demowriter should have a config setting that allows to write checksum data to the demofile (or does it already do that?), this would be useful for other stuff, too.
- it is slower on a single-core i guess

can't say much about the design, as i'm not familiar with the sim/pathfinder.

[zerver]

Does it work on non-ba?

If you are referring to the current MT-sim executable that may require you to tweak the lua a bit, this pathing/movetype threading is unrelated to that. So it should work. You can compile a non MT-sim executable but still have multiple sim threads and a pathing thread. But I did not spend a lot of time testing that configuration, and this is where you enter the picture.

- it is slower on a single-core i guess

Yeah, just did some testing: 2-4%

[Forboding Angel]

Does it work on non-ba?

If you are referring to the current MT-sim executable that may require you to tweak the lua a bit, this pathing/movetype threading is unrelated to that. So it should work. You can compile a non MT-sim executable but still have multiple sim threads and a pathing thread. But I did not spend a lot of time testing that configuration, and this is where you enter the picture.

Thanks. Umm, can you rephrase that in caveman speak?

[zerver]

Thanks. Umm, can you rephrase that in caveman speak?

If you download the archive from the link above, the situation is as follows:

Compatibility is unchanged. spring.exe is 100% compatible, spring-multithreaded.exe is faster but requires some tweaking of the lua code. They both contain the new "multithreaded simulation" stuff.

For testing purposes it is fine to just take the exe files(s) and drop them in your 91.0 installation folder, but it will not sync with the official 91.0 release.

[Beherith]

Some test results with the build in your link:

System:
i5-750 (4 cores, 4 threads) @ 3ghz
nvidia 560ti

BA 7.72, map DSD, f5 used to hide gui

Vanilla 91.0
2x give all: 43 fps, 20.5 ms avgdrawframe,3ms avgsimframe, 55% gpu load
1500 corvamp patrolling in view: 43.1 fps, 26% cpu load, 49% gpu load, 16.3ms draw, 4.5 ms sim

1000 armflash patrol to a line: avgsimframe peaks at 28 ms, 1 core used.

91.0 MT-SIM
2x give all: 57 fps, 14.5 ms avgdrawframe,2.5ms avgsimframe, 77% gpu load

1500 corvamp patrolling in view: 44.5 fps, 27% cpu load (only 1 core?) 45% gpu load, 15.5ms draw, 3.4 ms sim

1000 armflash patrol to a line: avgsimframe peaks at 18 ms, ~2 cores used (45% of all 4 cores).

Looks great so far, will do more tests later.

edit: tested more, seems like a win on all fronts!
Why do you ask we test with a quad? In my tests it didnt use more than 2 cores, can I make it use more? I didnt use any extra threading settings (not even the ones you provided in first post)

[zerver]

Absolutely, it uses the two remaining cores as well, but they will be loaded to less than 50%. This is because only parts of the sim have been threaded. You need to load it with a lot more than 2 "give all" to see something interesting in task manager.

For a quad without hyperthreading like yours, try this:

Code: Select all

MultiThreadCount = 4
SetCoreAffinity = 1
SetCoreAffinitySim = 2
SetCoreAffinitySimMT = 12
SetCoreAffinityRenderMT = 12
SetCoreAffinityPath = 12
SimThreadCount = 3

There will be an auto config mode for the above when I'm finished.

[Forboding Angel]

Windows xp /-_-

[Beherith]

How does this work compare to the normal MT build? Does it have the same draw-sim thread separation, or is just the sim part of this multithreaded?

Is the spring-multithreaded.exe included in the download link even faster?

[zerver]

spring-multithreaded has a separate sim thread like before, and is therefore even faster. If you do testing, focus on this one.

A newer build here with automatic CPU affinity setup:
http://springrts.com/dl/buildbot/defaul ... -ga07559a/
(you have to delete all manual *affinity* settings from springsettings.cfg to use the automatic setup)

[Beherith]

I tried benchmarking it by running it through some really heavy replays, but it desynced:

Attached demo on ba 7.72

[Beherith]

Windows xp /-_-

Nah, my win7 looks like that too, classic skin ftw. You can tell its win7 from the services tab in task manager.

[zerver]

I assume that is a 91.0 demo, and that will definitely desync. This release is a modded version of the develop branch.

For sync testing online play works well. All players must use the same exe.

[Beherith]

I was just trying to make an objective test, say we play some heavy games, and I want to paralell test some scenario.

Is there a build who's replays I could compare vanilla-mt-mt-sim?

Btw, spring-multithreaded.exe in your package crashes, infolog attached.

[Licho]

Crashes/quits within first 20s of game on Zero-K multiplayer.

Nothing related to fail in infolog (buffering ftw?)

[knorke]

I only have a sad dual core. Still worth/helpful to test?

[Google_Frog]

Multiple crashes or freezes in ZK.

http://zero-k.info/Battles/Detail/116804

[Beherith]

I only have a sad dual core. Still worth/helpful to test?

Imo yes, It rarely went above 2 cores for me, only the most taxing tests got it there.

We tested it with Bluestone in multiplayer. Pretty taxing test, no desyncs! Smooth as silk, yet drawframe always took way more than sim frame. Didnt go above 2 cores, but performed well!

Great stuff!