2019-08-25 03:26 CEST

View Issue Details Jump to Notes ]
IDProjectCategoryView StatusLast Update
0005151Spring engineGeneralpublic2016-03-11 02:22
Reporterjamerlan 
Assigned ToKloot 
PriorityhighSeveritymajorReproducibilityalways
StatusresolvedResolutionno change required 
Product Version101.0 
Target Version102.0Fixed in Version 
Summary0005151: huge CPU load for spring-dedicated
Description"BlackHoleHost" hosts were reverted to spring 100 because spring-dedicated is eating 100% CPU and game is lagging even when 2000 units is not reached yet.

we tried static builds from release post and compiled from sources version - same result
Steps To Reproduce1) start battle with some AI
2) /cheat
3) /nocost
4) spam units
5) take a look at cpu load of spring-dedicated
Additional InformationBA 9.35 + spring 101
TagsNo tags attached.
Checked infolog.txt for lua Errors
Attached Files

-Relationships
+Relationships

-Notes

~0016007

Kloot (developer)

and without AI's?

server code barely changed between 100 and 101 and it still sleeps 5 millisecs per tick, so the only reason CPU use would be higher is if clients are sending more messages.

~0016008

jamerlan (reporter)

no ai. ShockWave have an idea: maybe library was upgraded and now 5 millisec is transfered to 5 nano sec or smth?

~0016009

jamerlan (reporter)

attached demo + ShockWave sent me this: nanosleep({1, 0}, NULL

~0016010

Kloot (developer)

if you can compile the DS you can test that idea quickly by adding a printf here https://github.com/spring/spring/blob/develop/rts/builds/dedicated/main.cpp#L245 or here https://github.com/spring/spring/blob/develop/rts/Net/GameServer.cpp#L2427 (change the 5 to say 100) and counting how many lines per second are written.

~0016016

abma (administrator)

Last edited: 2016-03-07 21:54

View 3 revisions

static build of spring-dedicated 101:

ldd spring-dedicated
    linux-vdso.so.1 => (0x00007ffc200b1000)
    librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f28ace47000)
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f28acc3f000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f28ac937000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f28ac567000)
    /lib64/ld-linux-x86-64.so.2 (0x00005612390e2000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f28ac347000)

it links to libpthread, that is an error?!

local build spring-dedicated also links to libboost_thread.so

edit: seems to be pulled in by the crashhandler/headlessstubs

~0016031

Kloot (developer)

Last edited: 2016-03-09 21:45

View 2 revisions

note, as of f188aad8b91aca28a39d6378f675b6b008086c95 the server sleeps can be controlled:

gameserver now has a ServerSleepTime config parameter (milliseconds, default 5)
dedicated now has a --sleeptime commandline parameter (seconds, default 1)

test this DS build and report back please.

~0016033

jamerlan (reporter)

Looks like now issue is solved. Without additional arguments passed :-) (see fixed_cpu.png)
Strange that bug was reproducible only on BlackHoleHost (Debian GNU/Linux 8 (jessie)) but not on ACE hosts (unknown OS)
I was unable to reproduce on my own host (CentOS Linux release 7.2.1511) too.

~0016034

Kloot (developer)

Can you find out which versions of g++ and libboost are installed on BHH?

~0016035

jamerlan (reporter)

blackhole:
gcc http://pastebin.com/MNBkH2P1
libboost: http://pastebin.com/CwhrpDnw

~0016036

jamerlan (reporter)

testing@BlackHoleHost:~$ dpkg-query -l 'g++*' ?······························································································
Desired=Unknown/Install/Remove/Purge/Hold ?······························································································
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend ?······························································································
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad) ?······························································································
||/ Name Version Architecture Description ?······························································································
+++-====================-===============-===============-============================================= ?······························································································
ii g++ 4:4.9.2-2 amd64 GNU C++ compiler ?······························································································
ii g++-4.9 4.9.2-10 amd64 GNU C++ compiler ?······························································································
un g++-4.9-multilib <none> <none> (no description available) ?······························································································
un g++-multilib <none> <none> (no description available)

~0016043

Kloot (developer)

there was a bug in 1.55 boost::chrono that would cause this_thread::sleep_for to return immediately, but spring uses std::chrono on linux so no idea besides "system/kernel issue". thanks for testing anyway.
+Notes

-Issue History
Date Modified Username Field Change
2016-03-07 16:39 jamerlan New Issue
2016-03-07 16:39 jamerlan File Added: spring-dedicated.png
2016-03-07 17:06 Kloot Note Added: 0016007
2016-03-07 17:10 jamerlan Note Added: 0016008
2016-03-07 17:16 jamerlan Note Added: 0016009
2016-03-07 17:17 jamerlan File Added: 20160307_182404_DeltaSiegeDry_101.sdfz
2016-03-07 17:23 Kloot Note Added: 0016010
2016-03-07 21:42 abma Target Version => 102.0
2016-03-07 21:46 abma Note Added: 0016016
2016-03-07 21:53 abma Note Edited: 0016016 View Revisions
2016-03-07 21:54 abma Note Edited: 0016016 View Revisions
2016-03-09 20:15 Kloot Note Added: 0016031
2016-03-09 20:15 Kloot Status new => feedback
2016-03-09 21:25 jamerlan File Added: fixed_cpu.png
2016-03-09 21:36 jamerlan Note Added: 0016033
2016-03-09 21:36 jamerlan Status feedback => new
2016-03-09 21:44 Kloot Note Added: 0016034
2016-03-09 21:45 Kloot Note Edited: 0016031 View Revisions
2016-03-09 22:44 jamerlan Note Added: 0016035
2016-03-09 22:46 jamerlan Note Added: 0016036
2016-03-11 02:22 Kloot Note Added: 0016043
2016-03-11 02:22 Kloot Status new => resolved
2016-03-11 02:22 Kloot Resolution open => no change required
2016-03-11 02:22 Kloot Assigned To => Kloot
+Issue History