View Issue Details

IDProjectCategoryView StatusLast Update
0002484Spring engineGeneralpublic2011-11-06 18:33
ReporterLicho Assigned Toabma  
PrioritynormalSeveritycrashReproducibilitysometimes
Status resolvedResolutionfixed 
Product Version0.82.7.1 
Target Version83.0Fixed in Version83.0 
Summary0002484: Dedicated server massive memleak (3g+ in few seconds) - also brings down main spring server
DescriptionIt happens randomly http://planet-wars.eu/ss/SS-20110620023950.png mid game.

I have seen it 4 times already.

In all cases RES ram usage jumped to 3GB and server was brought to standstill.

Demo attached (from client). I can also provide demo from server itself if it helps.

TagsNo tags attached.
Attached Files
20110620_020612_Charlie in the Hills v2_0.82.7.zip (Attachment missing)
Checked infolog.txt for Errors

Activities

abma

2011-06-20 21:52

administrator   ~0006773

Last edited: 2011-06-20 21:56

did the dedicated-server "crash" because oom? if so, when stopped the demo?

or better, at which frame did it start to memleak?

tvo

2011-06-20 22:04

reporter   ~0006774

Use ulimit to prevent it from taking down the entire server (I've suggested that some times already...)

ulimit -v N

where N is in KB. Also: man bash

Licho

2011-06-20 23:28

reporter   ~0006780

No server didnt crash, it kept running i had to kill -9 it.
Demo i sent is from my pc, it stops when server stopped responding.

abma

2011-06-21 00:41

administrator   ~0006784

can you make a ulimit? maybe then it "crashes" when tries to malloc... this maybe helps a bit to find the cause.

Licho

2011-06-21 15:11

reporter   ~0006793

How to use ulimit to apply it to spring-dedicated? There is no plimit on ubuntu?
Im afraid about making shell script wrapper for entire spring-dedicated, is that the only way?

abma

2011-06-21 18:37

administrator   ~0006795

Last edited: 2011-06-21 20:02

a wrapper would be very simple:

#!/bin/sh
ulimit -v 500000
exec /usr/bin/spring-dedicated $@

just chmod +x this file...

the other solution would to ulimit the parent process, but then the parent process could be killed by oom, too...

Licho

2011-06-21 18:46

reporter   ~0006796

Will this redirect standard output correctly?
Will it listen to processor affinity and priority?

abma

2011-06-21 20:00

administrator   ~0006797

Last edited: 2011-06-21 20:03

hu? there is no redirection, this script just simple calls ulimit before running spring-dedicated... it should not make a difference for the caller...

i'm only unsure if the path of spring-dedicated is correct and if the memory limit makes sense... 500MB if i did it correctly.

hoijui

2011-06-27 15:28

reporter   ~0006816

as i get it, it is considered to be a memleak in springie, not spring dedicated server?

Licho

2011-06-27 16:06

reporter   ~0006819

No, this memleak is in spring-dedicated.
It has nothing to do with springie.

abma

2011-08-06 04:11

administrator   ~0007226

Last edited: 2011-08-06 04:12

@licho:

did you add the wrapper-script?

can you provide a smaller demo from the server? can you attach also infolog.txt/logs from the server?

hoijui

2011-09-10 15:15

reporter   ~0007359

some say, it is caused by mono.
if it is that, then updating mono or moving the autohosts to the windows VM, where they can be run on proper .Net VM could solve it.

if it is not that, then.. we need a better way to find out what in spring-dedicated it is. or we just trust in it being fixed in master (which probably is not even too unlikely).

Licho

2011-09-16 09:59

reporter   ~0007374

Sorry I didnt provide feedback but I dont know how to test this without changing live servers.

How could mono affect dedicated server which is normal native binary process?

abma

2011-09-16 10:11

administrator   ~0007375

without any input we can't fix anything, so... imo you've to try out or wait for the next release. is there no development-server for that? if so, springie was also a live-test, so this is no reason to not try it...

it could also help, if you can provide smaller demos + infolog.txt

but it looks like, that the error doesn't happen when replaying a demo-file.

Licho

2011-09-16 10:17

reporter   ~0007376

I had development server in the past but it involved me manually copying windows builds of dedicated and due to way spring hashes change with engine version it lead to huge version spam in plasma database.
This + minimal interest in playing there caused me to turn it off.

Licho

2011-11-06 18:29

reporter   ~0007500

It appears to be fixed now this can be closed.

abma

2011-11-06 18:33

administrator   ~0007501

thanks! (closing as its fixed)

Issue History

Date Modified Username Field Change
2011-06-20 02:48 Licho New Issue
2011-06-20 02:48 Licho File Added: 20110620_020612_Charlie in the Hills v2_0.82.7.zip
2011-06-20 02:48 Licho Graphics Card => unknown
2011-06-20 21:39 abma Target Version => 0.83.0
2011-06-20 21:52 abma Note Added: 0006773
2011-06-20 21:56 abma Note Edited: 0006773
2011-06-20 22:04 tvo Note Added: 0006774
2011-06-20 23:28 Licho Note Added: 0006780
2011-06-21 00:41 abma Note Added: 0006784
2011-06-21 15:11 Licho Note Added: 0006793
2011-06-21 18:37 abma Note Added: 0006795
2011-06-21 18:38 abma Note Edited: 0006795
2011-06-21 18:46 Licho Note Added: 0006796
2011-06-21 20:00 abma Note Added: 0006797
2011-06-21 20:01 abma Note Edited: 0006795
2011-06-21 20:02 abma Note Edited: 0006795
2011-06-21 20:03 abma Note Edited: 0006797
2011-06-27 15:28 hoijui Note Added: 0006816
2011-06-27 16:06 Licho Note Added: 0006819
2011-08-06 04:11 abma Note Added: 0007226
2011-08-06 04:11 abma Status new => feedback
2011-08-06 04:12 abma Note Edited: 0007226
2011-09-10 15:15 hoijui Note Added: 0007359
2011-09-16 09:59 Licho Note Added: 0007374
2011-09-16 10:11 abma Note Added: 0007375
2011-09-16 10:17 Licho Note Added: 0007376
2011-11-06 18:29 Licho Note Added: 0007500
2011-11-06 18:33 abma Note Added: 0007501
2011-11-06 18:33 abma Status feedback => resolved
2011-11-06 18:33 abma Fixed in Version => 83.0
2011-11-06 18:33 abma Resolution open => fixed
2011-11-06 18:33 abma Assigned To => abma