View Issue Details

IDProjectCategoryView StatusLast Update
0005151Spring engineGeneralpublic2016-03-11 02:22
Reporterjamerlan Assigned ToKloot  
PriorityhighSeveritymajorReproducibilityalways
Status resolvedResolutionno change required 
Product Version101.0 
Target Version102.0 
Summary0005151: huge CPU load for spring-dedicated
Description"BlackHoleHost" hosts were reverted to spring 100 because spring-dedicated is eating 100% CPU and game is lagging even when 2000 units is not reached yet.

we tried static builds from release post and compiled from sources version - same result
Steps To Reproduce1) start battle with some AI
2) /cheat
3) /nocost
4) spam units
5) take a look at cpu load of spring-dedicated
Additional InformationBA 9.35 + spring 101
TagsNo tags attached.
Attached Files
spring-dedicated.png (Attachment missing)
20160307_182404_DeltaSiegeDry_101.sdfz (Attachment missing)
fixed_cpu.png (Attachment missing)
Checked infolog.txt for Errors

Activities

Kloot

2016-03-07 17:06

developer   ~0016007

and without AI's?

server code barely changed between 100 and 101 and it still sleeps 5 millisecs per tick, so the only reason CPU use would be higher is if clients are sending more messages.

jamerlan

2016-03-07 17:10

reporter   ~0016008

no ai. ShockWave have an idea: maybe library was upgraded and now 5 millisec is transfered to 5 nano sec or smth?

jamerlan

2016-03-07 17:16

reporter   ~0016009

attached demo + ShockWave sent me this: nanosleep({1, 0}, NULL

Kloot

2016-03-07 17:23

developer   ~0016010

if you can compile the DS you can test that idea quickly by adding a printf here https://github.com/spring/spring/blob/develop/rts/builds/dedicated/main.cpp#L245 or here https://github.com/spring/spring/blob/develop/rts/Net/GameServer.cpp#L2427 (change the 5 to say 100) and counting how many lines per second are written.

abma

2016-03-07 21:46

administrator   ~0016016

Last edited: 2016-03-07 21:54

static build of spring-dedicated 101:

ldd spring-dedicated
    linux-vdso.so.1 => (0x00007ffc200b1000)
    librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f28ace47000)
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f28acc3f000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f28ac937000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f28ac567000)
    /lib64/ld-linux-x86-64.so.2 (0x00005612390e2000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f28ac347000)

it links to libpthread, that is an error?!

local build spring-dedicated also links to libboost_thread.so

edit: seems to be pulled in by the crashhandler/headlessstubs

Kloot

2016-03-09 20:15

developer   ~0016031

Last edited: 2016-03-09 21:45

note, as of f188aad8b91aca28a39d6378f675b6b008086c95 the server sleeps can be controlled:

gameserver now has a ServerSleepTime config parameter (milliseconds, default 5)
dedicated now has a --sleeptime commandline parameter (seconds, default 1)

test this DS build and report back please.

jamerlan

2016-03-09 21:36

reporter   ~0016033

Looks like now issue is solved. Without additional arguments passed :-) (see fixed_cpu.png)
Strange that bug was reproducible only on BlackHoleHost (Debian GNU/Linux 8 (jessie)) but not on ACE hosts (unknown OS)
I was unable to reproduce on my own host (CentOS Linux release 7.2.1511) too.

Kloot

2016-03-09 21:44

developer   ~0016034

Can you find out which versions of g++ and libboost are installed on BHH?

jamerlan

2016-03-09 22:44

reporter   ~0016035

blackhole:
gcc http://pastebin.com/MNBkH2P1
libboost: http://pastebin.com/CwhrpDnw

jamerlan

2016-03-09 22:46

reporter   ~0016036

testing@BlackHoleHost:~$ dpkg-query -l 'g++*' ?······························································································
Desired=Unknown/Install/Remove/Purge/Hold ?······························································································
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend ?······························································································
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad) ?······························································································
||/ Name Version Architecture Description ?······························································································
+++-====================-===============-===============-============================================= ?······························································································
ii g++ 4:4.9.2-2 amd64 GNU C++ compiler ?······························································································
ii g++-4.9 4.9.2-10 amd64 GNU C++ compiler ?······························································································
un g++-4.9-multilib <none> <none> (no description available) ?······························································································
un g++-multilib <none> <none> (no description available)

Kloot

2016-03-11 02:22

developer   ~0016043

there was a bug in 1.55 boost::chrono that would cause this_thread::sleep_for to return immediately, but spring uses std::chrono on linux so no idea besides "system/kernel issue". thanks for testing anyway.

Issue History

Date Modified Username Field Change
2016-03-07 16:39 jamerlan New Issue
2016-03-07 16:39 jamerlan File Added: spring-dedicated.png
2016-03-07 17:06 Kloot Note Added: 0016007
2016-03-07 17:10 jamerlan Note Added: 0016008
2016-03-07 17:16 jamerlan Note Added: 0016009
2016-03-07 17:17 jamerlan File Added: 20160307_182404_DeltaSiegeDry_101.sdfz
2016-03-07 17:23 Kloot Note Added: 0016010
2016-03-07 21:42 abma Target Version => 102.0
2016-03-07 21:46 abma Note Added: 0016016
2016-03-07 21:53 abma Note Edited: 0016016
2016-03-07 21:54 abma Note Edited: 0016016
2016-03-09 20:15 Kloot Note Added: 0016031
2016-03-09 20:15 Kloot Status new => feedback
2016-03-09 21:25 jamerlan File Added: fixed_cpu.png
2016-03-09 21:36 jamerlan Note Added: 0016033
2016-03-09 21:36 jamerlan Status feedback => new
2016-03-09 21:44 Kloot Note Added: 0016034
2016-03-09 21:45 Kloot Note Edited: 0016031
2016-03-09 22:44 jamerlan Note Added: 0016035
2016-03-09 22:46 jamerlan Note Added: 0016036
2016-03-11 02:22 Kloot Note Added: 0016043
2016-03-11 02:22 Kloot Status new => resolved
2016-03-11 02:22 Kloot Resolution open => no change required
2016-03-11 02:22 Kloot Assigned To => Kloot