View Issue Details

Jump to NotesJump to History
IDProjectCategoryView StatusDate SubmittedLast Update
0004396Spring engineGeneralpublic2014-05-18 04:542014-05-22 00:21
ReporterMajorBoredom Assigned Toabma  
PrioritynormalSeverityminorReproducibilityrandom
Status closedResolutionunable to reproduce 
Product Version96.0 
Summary0004396: Crash at end of game in CNetProtocol::UpdateLoop
DescriptionI produced the following stack trace with gdb. Frames 0000010-#11 are the interesting ones imo:

[... infolog.txt lines omitted ...]
[SpringApp::ShutDown][10]
[SpringApp::ShutDown][11]
[WatchDog::Uninstall][1] hangDetectorThread=(nil)
[SpringApp::ShutDown][12]
[SpringApp::Run] exitCode=0
terminate called after throwing an instance of 'boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<boost::lock_error> >'
  what(): boost: mutex lock failed in pthread_mutex_lock: Invalid argument

Program received signal SIGABRT, Aborted.
[Switching to Thread 0x7fffbf7fe700 (LWP 17476)]
0x0000003dea4359e9 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
56 return INLINE_SYSCALL (tgkill, 3, pid, selftid, sig);
Missing separate debuginfos, use: debuginfo-install DevIL-1.7.8-14.fc19.x86_64 SDL-1.2.15-12.fc19.x86_64 dbus-libs-1.6.12-2.fc19.x86_64 flac-libs-1.3.0-2.fc19.x86_64 freetype-2.4.11-7.fc19.x86_64 gsm-1.0.13-9.fc19.x86_64 jasper-libs-1.900.1-24.fc19.x86_64 jbigkit-libs-2.0-9.fc19.x86_64 json-c-0.11-6.fc19.x86_64 lcms-libs-1.19-9.fc19.x86_64 libGLEW-1.9.0-3.fc19.x86_64 libICE-1.0.8-5.fc19.x86_64 libSM-1.2.1-5.fc19.x86_64 libXau-1.0.8-1.fc19.x86_64 libXcursor-1.1.14-1.fc19.x86_64 libXext-1.3.2-1.fc19.x86_64 libXfixes-5.0.1-1.fc19.x86_64 libXi-1.7.2-1.fc19.x86_64 libXrandr-1.4.1-1.fc19.x86_64 libXrender-0.9.7-6.20130524git786f78fd8.fc19.x86_64 libXtst-1.2.2-1.fc19.x86_64 libasyncns-0.8-5.fc19.x86_64 libattr-2.4.46-10.fc19.x86_64 libcap-2.22-6.fc19.x86_64 libicu-50.1.2-9.fc19.x86_64 libjpeg-turbo-1.2.90-3.fc19.x86_64 libmng-1.0.10-11.fc19.x86_64 libogg-1.3.0-5.fc19.x86_64 libpng-1.5.13-2.fc19.x86_64 libsndfile-1.0.25-7.fc19.x86_64 libtiff-4.0.3-9.fc19.x86_64 libunwind-1.1-2.fc19.x86_64 libuuid-2.23.2-5.fc19.x86_64 libvorbis-1.3.4-1.fc19.x86_64 mesa-libGLU-9.0.0-4.fc19.x86_64 openal-soft-1.15.1-2.fc19.x86_64 pulseaudio-libs-3.0-10.fc19.x86_64 tcp_wrappers-libs-7.6-75.fc19.x86_64 zlib-1.2.7-10.fc19.x86_64
(gdb) bt
#0 0x0000003dea4359e9 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1 0x0000003dea4370f8 in __GI_abort () at abort.c:90
#2 0x000000385f460565 in __gnu_cxx::__verbose_terminate_handler () at ../../../../libstdc++-v3/libsupc++/vterminate.cc:95
0000003 0x000000385f45e6c6 in __cxxabiv1::__terminate (handler=<optimized out>) at ../../../../libstdc++-v3/libsupc++/eh_terminate.cc:38
0000004 0x000000385f45e6f3 in std::terminate () at ../../../../libstdc++-v3/libsupc++/eh_terminate.cc:48
0000005 0x000000385f45e91f in __cxxabiv1::__cxa_throw (obj=0x7fffa8000990, tinfo=<optimized out>, dest=<optimized out>)
    at ../../../../libstdc++-v3/libsupc++/eh_throw.cc:84
#6 0x000000000049474c in boost::throw_exception<boost::lock_error> (e=...) at /usr/include/boost/throw_exception.hpp:67
#7 0x00000000008aab7a in lock (this=<optimized out>) at /usr/include/boost/thread/pthread/mutex.hpp:71
#8 lock (this=<optimized out>) at /usr/include/boost/thread/lock_types.hpp:344
#9 unique_lock (m_=..., this=<optimized out>) at /usr/include/boost/thread/lock_types.hpp:124
0000010 spring_time::sleep (this=this@entry=0x7fffbf7fde00) at /home/joya/dev/spring/rts/System/Misc/SpringTime.cpp:234
#11 0x0000000000552ebf in CNetProtocol::UpdateLoop (this=0x171ea10) at /home/joya/dev/spring/rts/Net/Protocol/NetProtocol.cpp:150
0000012 0x000000000047282e in operator() (p=<optimized out>, this=<optimized out>) at /usr/include/boost/bind/mem_fn_template.hpp:49
0000013 operator()<boost::_mfi::mf0<void, CNetProtocol>, boost::_bi::list0> (a=<synthetic pointer>, f=..., this=<optimized out>)
    at /usr/include/boost/bind/bind.hpp:253
0000014 operator() (this=<optimized out>) at /usr/include/boost/bind/bind_template.hpp:20
#15 boost::detail::function::void_function_obj_invoker0<boost::_bi::bind_t<void, boost::_mfi::mf0<void, CNetProtocol>, boost::_bi::list1<boost::_bi::value<CNetProtocol*> > >, void>::invoke (function_obj_ptr=...) at /usr/include/boost/function/function_template.hpp:153
#16 0x000000000091fb2c in operator() (this=0x7fffbf7fde70) at /usr/include/boost/function/function_template.hpp:767
#17 Threading::ThreadStart(boost::function<void ()>, std::shared_ptr<Threading::ThreadControls>*) (taskFunc=..., ppThreadCtls=0x2ba2780)
    at /home/joya/dev/spring/rts/System/Platform/Linux/ThreadSupport.cpp:110
#18 0x00000000008c31c7 in operator()<void (*)(boost::function<void()>, std::shared_ptr<Threading::ThreadControls>*), boost::_bi::list0> (
    a=<synthetic pointer>, f=<optimized out>, this=<optimized out>) at /usr/include/boost/bind/bind.hpp:313
#19 operator() (this=<optimized out>) at /usr/include/boost/bind/bind_template.hpp:20
0000020 boost::detail::thread_data<boost::_bi::bind_t<void, void (*)(boost::function<void ()>, std::shared_ptr<Threading::ThreadControls>*), boost::_bi::list2<boost::_bi::value<boost::function<void ()> >, boost::_bi::value<std::shared_ptr<Threading::ThreadControls>*> > > >::run() (this=<optimized out>)
    at /usr/include/boost/thread/detail/thread.hpp:117
#21 0x00000034a6c0c0da in boost::(anonymous namespace)::thread_proxy (param=<optimized out>) at libs/thread/src/pthread/thread.cpp:165
#22 0x0000003deac07c53 in start_thread (arg=0x7fffbf7fe700) at pthread_create.c:308
#23 0x0000003dea4f5dbd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:113
(gdb)

Steps To ReproduceJust run 96.0 in debug mode and spectate quite a few games, it happens once in a while.
Additional InformationIt looks like CNetProtocol::UpdateLoop() never finishes! It contains an infinite loop, so it is still running when the game is over. If it enters the spring_msecs(400).sleep() call on NetProtocol.cpp:150, then it tries to acquire a mutex lock inside this function. It seems the mutex has already been destroyed / cleaned up by another thread. So eventually abort() is called and this is how UpdateLoop() finishes.

This thread is eitehr an instance of the pingThread started in Game.cpp or the netHeartbeatThread started in LoadScreen.cpp.
TagsNo tags attached.
Attached Files
udpconnflush-backtrace.1.txt (Attachment missing)
udpconnflush-backtrace.2.txt (Attachment missing)
Checked infolog.txt for Errors

Activities

cleanrock

2014-05-18 08:38

reporter   ~0013108

This may have been fixed since 96.0, e.g.:
https://github.com/spring/spring/commit/86638c5ba3b
https://github.com/spring/spring/commit/fe065d52c89

abma

2014-05-21 13:02

administrator   ~0013128

Which os are you using? Which boost version? Very likely something is bad with your build env as nobody else has/had this problem.

abma

2014-05-21 13:27

administrator   ~0013129

Last edited: 2014-05-21 13:32

 also it would be interesting looking into syslog / system log files if some firewall / security feature breaks stuff here.


does it happen with the official static builds as well?

before investigationg any further, please answer my questions...

MajorBoredom

2014-05-21 18:39

reporter   ~0013131

- Fedora 19 x86-64
- kernel 3.13.9
- boost 1.55
- have not tried static builds
- system is otherwise fine:
  - no messages of concern in /var/log/messages nor dmesg
  - cmake does not complain
  - project libs involving locks (gml) seem to build fine

I am attaching 3 separate backtraces showing different facets of the problem (2 with gdb, 1 with libunwind).

MajorBoredom

2014-05-21 18:57

reporter   ~0013132

Take a look at the variables I printed off while gdb had the program suspended.

MajorBoredom

2014-05-21 19:17

reporter   ~0013133

Note that there were two separate issues with this module. This bug report was originally intended to document a crash at endgame due to net threads still running after SpringApp::Shutdown(), but these traces refer to a crash that happens midgame and seems to be due to corrupt containers in UDPConnection. Perhaps I should file a separate bug and put the traces there.

MajorBoredom

2014-05-21 22:24

reporter   ~0013134

I am going to file a separate bug for the ingame crashes since they seem to be unrelated to the endgame ones which have a clearer cause and are less severe.

Issue History

Date Modified Username Field Change
2014-05-18 04:54 MajorBoredom New Issue
2014-05-18 08:38 cleanrock Note Added: 0013108
2014-05-21 13:02 abma Note Added: 0013128
2014-05-21 13:02 abma Assigned To => abma
2014-05-21 13:02 abma Status new => feedback
2014-05-21 13:27 abma Note Added: 0013129
2014-05-21 13:31 abma Note Edited: 0013129
2014-05-21 13:32 abma Note Edited: 0013129
2014-05-21 18:39 MajorBoredom Note Added: 0013131
2014-05-21 18:39 MajorBoredom Status feedback => assigned
2014-05-21 18:41 MajorBoredom File Added: udpconnflush-backtrace.1.txt
2014-05-21 18:41 MajorBoredom File Added: udpconnflush-backtrace.2.txt
2014-05-21 18:57 MajorBoredom Note Added: 0013132
2014-05-21 19:17 MajorBoredom Note Added: 0013133
2014-05-21 22:24 MajorBoredom Note Added: 0013134
2014-05-22 00:21 abma Status assigned => closed
2014-05-22 00:21 abma Resolution open => unable to reproduce