Present: zerver, jK, hoijui, Tobi, Kloot, abma
Welcome
<hoijui>all here!
<zerver>congratz on a somewhat working release
<hoijui>hehe :-D
<hoijui>yeah.. nice work!
<hoijui>mostly to jk!
<hoijui>doing lots of the ugly work
<jK>hehe
<jK>community is divided >_<
<Kloot>as always
<jK>some say it's a lot faster, some say it's a lot slower and intel users say it doesn't work at all :-D
<zerver>:-D that is a common phenomenon
<hoijui>yeah true. also was wondering why its faster for some and slower for others
<jK>I assume it depends on the GPU
<zerver>probably related to gfx yes, I would say it is slower for virtually all CPU wise
<jK>since terrain is now rendered by default through GLSL, the shader performance is very important
<jK>sim is slower yes, but rendering is a bit less CPU intensive
<abma>yeah, nice release!
<Tobi>yeah
what's needed/wanted for 85.0?
<zerver>less crash is much wanted
<zerver>and working pathing for crowds
<zerver>ROAM?
<hoijui>nearly all crash is intel and advmapshading, right?
<jK>crowd pathing is a longtime feature request
<zerver>and some MT bugs, mostly triggered by specific widgets
<jK>advmapshading doesn't crash, it makes the map black on intel drivers
<Kloot>so much for their glsl "support"
<jK>command queue needs to get fixed (alt+f3 changes camera + metalview)
<jK>*chat cmd
<jK>any+[f3] should be checked at the end of the queue
<jK>hehe yeah kloot
<jK>zerver, ROAM still needs the GML support
<Kloot>and carmack said intel hw was getting decent >>
<jK>my idea for the 85.0 timeline was 1-1.5weeks
<jK>Rage runs on Intel GPUs?
<zerver>yeah, I was waiting for ROAM to be merged in develop and then fix it, but maybe I should do it before that
<jK>if I merge it now, and you don't get the support ready before 85.0 ...
<zerver>wrt the pathing, this with crowds getting completely stuck, IIRC this has not been in spring forever, but is a fairly new thing. I know there was some of it in 82.7 but it seems worse now
<jK>timeline 1-1.5weeks is okay?
<zerver>fine
<hoijui>from now? yeah
<Kloot>I'll merge qtpfs soonish then
<jK>k ROAM will happen in 86.0 then
<zerver>I have never touched the ROAM stuff, do I have commit access there?
<jK>no
<jK>it's on behe's account
<zerver>ah
<abma>create a roam branch in main repo?
<abma>as its likely it will be merged
<zerver>or I can make a patch and send someone
<jK>or you can make the patch in the main repo, or do you think you need to modify the render code itself?
<zerver>i'm expecting to discover issues when I do profiling of mutexes
<jK>I didn't used mutexes
<zerver>okay, I'm expecting crashes then
<zerver>
<jK>I don't think you will find any threading conflict, still when you find some we can discuss that
<jK>oh nvm
<jK>possibly reading from heightmap needs mutexes
<jK>but they are likely in a mutex of Readmap
<jK>k you will see
Conclusion: ROAM will be merged in 86.0, GML with ROAM needs to be fixed first. We try to release 85.0 (mostly a bugfix release) in 1-1.5 weeks.
Incomplete infologs caused by forced exit thread?
<jK>flusing/fclose gets done in the dtor of static var, I don't think it gets called when doing a exit(-1)
<jK>neither does the force exit thread print anything to the infolog
<jK>nor do I think to stdout
<jK>cause of:
<jK> logSinkHandler.SetSinking(fa logSinkHandler.SetSinking(false);
<jK> if (forced) {
<jK> LOG_L(L_ERROR, "failed to shutdown normally, exit forced");
<jK> }
<jK> LOG_L(L_ERROR, "%s %s", caption.c_str(), msg.c_str());lse);
<jK> if (forced) {
<jK> LOG_L(L_ERROR, "failed to shutdown normally, exit forced");
<jK> }
<jK> LOG_L(L_ERROR, "%s %s", caption.c_str(), msg.c_str());
<jK>o_O why did it doubled that
<jK>only first 5 lines matter
<jK>https://github.com/spring/spring/blob/1 ... er.cpp#L39
<zerver>yeah, I think exit(-1) is pretty brute force
<jK>it must be brute force as it force exits spring on hangs
<zerver>yep
<jK>the problem is that infolog isn't flushed in that case
<jK>and that this should get fixed
<zerver>so we need to do it manually
<jK>spring 82 neither flushed infolog afaik, and it still contained the full infolog on crashes but 84 doesn't. So this may be the only cause of the change.
<hoijui>hmmm
<hoijui>yeah it did flush
<hoijui>in 82
<zerver>MSVC TerminateProcess(GetCurrentProcess(), -1); will have the same issue, very brute force, no cleanup at all
<hoijui>thing is...
<hoijui>flushing is somethign of the backend, and thus .. letting client code of the log appi do flushing is ugly
<hoijui>i guess i remvoed it when i turned on auto-flushing (always)
<hoijui>and forgot about that when it was decided to remove that
<zerver>expose the backend ffs
<hoijui>we can have an API macro that forwards a call to the backends
<hoijui>something like: "LOG_FINNISHING"
<hoijui>()
<hoijui>is a bit nicer
<zerver>eventHandler.SpringCrashed(), and the backend subscribes to that
<hoijui>as was said, .. that would not really fit into a forced exit
<zerver>yeah we want to KISS, since spring may malfunction after a crash
<hoijui>though yeah.. soethign like that would also be nicer
<zerver>it was more of a joke...
<hoijui>ok
<hoijui>well..
<hoijui>my suggestion back then was, to auto-flush ERROR messages
<hoijui>that would be most easy and least ugly
<zerver>make another message type CRITICAL, and flush that
<hoijui>but .. well obviously it was nto wanted :-D
<Tobi>with exit destructors will be called, with TerminateProcess not
<jK>the end of a crashed infolog contains even non-error messages
<Tobi>destructors of non-stack variables only though
<hoijui>no need for an additional level for that
<hoijui>its not like flushing all error messages is too expensive
<hoijui>if yo uget so many error messages that flushign is expensive... then exepnsive flushing is not your main problem
<jK>as said flushing on error wouldn't give you a full infolog on crashes
<zerver>no, I guess if you have a continuous error spam, then some lag can be justified
<hoijui>it woudl give you everythign up to the error
<hoijui>that should be enough, no?
<jK>it needs a way to say `flush now` right before force exiting
<zerver>that is the simplest yes
<hoijui>why would you need additional log messages that are not errors after the force exit message?
<hoijui>no it is not
<hoijui>simplest is to auto-flush error messages
<hoijui>maybe the reason to not flush error messages was.. that it is useless, cause it will always be auto flushed at exit
<hoijui>..whcih is not true
<Tobi>IMHO explicit flushing/closing the log where it is needed (in crash handler) is simpler than some arbitrary choice to flush only errors, but not warnings
<hoijui>no, it is less simple
<zerver>and most guaranteed to work forever if the log system changes...
<hoijui>you may preffer it due to other stuff
<hoijui>but simplest is error auto-flush
<Tobi>it will lead to bugs
<Tobi>e.g. if someone adds an informational message after stack trace to say whether stack trace is useful or not
<Tobi>and forgets to set this to error level
<Tobi>then we have the current bug again
<hoijui>if someoen adds an exit() somewhere, and forgets to flush..
<hoijui>
<hoijui>please!
<zerver>I agree, there are two things here: 1. simple to code, 2. simple to invoke
<Tobi>linking error message with flush is bad functional coupling, separate methods are orthogonal and uncoupled, thus better
<jK>make your auto-flush on errors a configtag & add a function to force flushing
<hoijui>atuo-flush is simpler in both situations
<jK>both are happy
<hoijui>they coubple client code to backend
<hoijui>bad coupling
<Tobi>just a little bit though
<hoijui>hehe
<Tobi>anything with a buffer can have a flush
<zerver>LOG_L(L_FLUSH);
<hoijui>well it seems pretty obvious to me by now, that there seem not to be any real arguemtns pro/contra either way, except "feelings"
<Tobi>wrong
<zerver>explicit flush is CLEAR what it does
<hoijui>ahh well.. if oyu need it..
<Tobi>an API that makes explicit what you are doing is always better than having to rely on implicit behaviour
<hoijui>atuo flush oin error is also clear
<Tobi>but its implicit
<hoijui>an api that abstracts things away that can be abstracted away without loosing anything is superior
<Tobi>so its only clear if you have read the log code source
<Tobi>or the docs
<zerver>and then comes situation #2 where we want to flush even though it is no error, and we cannot
<hoijui>can go on forever
<Tobi>it isn't clear when you just read crash handler source
<hoijui>can put it in the API doc
<hoijui>or well. better in the sink doc
<hoijui>lets face it, in practise, my solution is simpoler, and there will never be a problem, even though ther ecould be one in theory
<Tobi>and this is still besides the fact that it is unorthogonal, so you'll get ppl adding a dummy error message to flush their list of warnings before it
<hoijui>its like the stupid virtual function calls thing for code that does 10000 times as much, CPU wise
<zerver> LOG_L(L_FLUSH); ftw
<hoijui>the only reason to flush is, as you always said, that we are exiting
<Tobi>no, this is simple API design, unorthogonal API = bad, and implicit, unexpected behaviour = bad
<Tobi>it would be slightly better if L_ERROR is renamed to L_ERROR_AND_FLUSH, but that's just a workaround
<zerver>and too verbose
<hoijui>and would you want the api to read LOG_FLUSH or LOG_END or LOG_EXITING, ..
<hoijui>because i think flush is bad, casue it is again backend specific
<zerver>LOG_FLUSH imo
<hoijui>other stuf could be requried for special sinks
<hoijui>which then again would need more api calls
<Tobi>for the current purpose close/finish/flush are all fine, I don't mind
<hoijui>k
<hoijui>you realize that you still dont have a single pro argumetn though?
<hoijui>your example was actualy a contra example
<hoijui>warnings shoudl not be flushed, only at exit shoudl ever be flushed
<hoijui>adn it woudl always nbe either a clena one or an error at the end
<Tobi>I realize I gave two arguments pro flush 1) orthogonal 2) explicit behavior
<hoijui>tryiogn to flush warning messages with an error log is totally bad code, and using bad code exampel as a pro arguemtn for your thing.. is stupid
<Tobi>and thus, orthogonal API that doesn't force developers to write bad code, is better
<hoijui>as said, you have disproven that arugment with your bad example
<hoijui>and i gave a contra arguemtn for the other one
<hoijui>not contra..
<hoijui>but hmm.. or is it contra?
<hoijui>well.. one that is as strong, same level of abstraction, and arugments pro the other side
<hoijui>but as the majority seem to like your thing, we'll do that
Conclusion: infolog.txt is cut of on crashes, hoijui will try to fix that.
use boost::-Detail::-Spinlock instead of boost::mutex
<zerver>spinlocks perform better when the locking time is very short and you have many thread competing for a shared resource iirc
<jK>yup
<jK>and that's the case with lua, rendering & sim
<zerver>here we have long locking times, so a spin lock would only waste CPU
<jK>sim is time critical
<jK>it shouldn't waste time in yield (a yield can take upto 10ms!!!)
<zerver>yeah, and?
<zerver>if a rendering gadget holds a lock for a long time, and sim wants that lock, the sim will be no happier if that lock happens to be a spin lock
<jK>also no says that the mutex is unlocked when the sim-thread awakes, so it can yeild again and again and ...
<jK>with a spinlock the sim thread can _directly_ begin its work when the mutex is unlocked w/o wasting time
<jK>and the chance that the mutex is locked again is null
<Tobi>however it eats 100% cpu while waiting for the lock, thereby reducing cpu availability for other threads
<jK>neither are we interested in energy saving in spring code
<Tobi>and wasting its scheduler timeslot
<Tobi>so it will get preempted again earlier
<Tobi>as with all performance things I'd say try, measure, and choose then
<jK>we aren't interested in "cpu availability for other threads" either, cause with spring should use all power it gets in spring-mt (else you would use singlethreaded one, to have a spare core for desktop etc.)
<hoijui>sounds like this can easily be choosen through a define, right?
<Tobi>I thought the point of spring mt is that it has multiple threads
<jK>yeah but each thread should have its own core
<Tobi>has it?
<jK>else spring-mt makes even less sense
<jK>running spring-mt on a single-core >_<
<Tobi>we have sound and main thread already => all cores full on c2d
<zerver>and hang detector, it is overcrowded
<jK>all those threads use <10% of a core
<hoijui>you would use spin everywhere?
<zerver>only sim and draw are real cpu eaters
<hoijui>or just for sim and render?
<Tobi>yes, but still they need CPU, if another thread is wasting CPU in a spinlock they can't run in the mean time
<jK>they can kernel thread scheduler is always gives them time
<jK>+, -s
<jK>*they can, the kernel thread scheduler ialways gives them time
<Tobi>yeah, but now they may get their share of time right when the spinlock was acquired by an important thread
<Tobi>instead of getting their stuff done *while* the important thread is waiting for the lock
<jK>we could reduce the thread niceness when waiting >_<
<jK>erm increase
<Tobi>and doesn't that require a call to the kernel?
<zerver>pfff
<Tobi>which could as well be used for the mutex?
<jK>that was a joke
<Tobi>k
<zerver>i thought mutexes were implemented with signals, so the scheduler immediately wakes up a waiting thread when the mutex is unlocked
<Tobi>anyway, I'm playing advocate of the devil, I don't really mind what we use although I don't believe spinlock will be faster, but I hope you will measure it and don't change if it doesn't give a significant improvement
<jK>but nothing is more important than the sim-thread, also it's not as the sim-thread always spinlocks and use that way 100%
<jK>the sim-thread still runs only 30 times per second
<jK>and so gives _much_ room between the single simframes for other threads
<Tobi>ok
<zerver>wrong
<zerver>sim may very well be 100%
<Tobi>yeah true, spring is still sim-cpu-bound when performance is most needed..
<zerver>the way I see it, draw thread repeatedly grabbing the same lock without letting sim in in between is not happening at all, and if it happens, it is very rare and not causing any problem
<jK>then you would get 2fps in default spring
<zerver>many people do, and they are whining about it
<Tobi>server slows down game first
<jK>you never profiled it
<Tobi>unless they turned it off
<jK>saying it doesn't happen w/o having it ever profiled, is naive
<Tobi>have you profiled spring with spinlocks?
<zerver>reading about how it is implemented should be enough
<zerver>actually I think that with a spinlock the same lock can be grabbed twice (if the thread is not active at the critical moment) while with a normal mutex this cannot happen
<zerver>badly designed spinlocks are used in the schoolbook examples to illustrate livelocks
<jK>I don't think that _boost_'s implementation is bugged
<zerver>that is not what I said, only that the condition that you wanted to prevent, repeated locking of the same lock, can definitely happen with a spinlock, but with a normal mutex I am not so sure
<zerver>need to read about it...
<zerver>thanks for suggesting anyway
<zerver>next?
<Tobi>don't you have fair locks for this?
<Tobi>that guarantee going through all threads in round robin fashion or so
<zerver>thread awakening order is a different story
<zerver>so far we are talking about a two-thread situation
<zerver>a normal mutex is not fair IIRC, since a random thread is awakened
<Tobi>yeah, there iirc with mutex it will always switch threads *if* the other thread was already waiting for the lock once the other releases it
<Tobi>otherwise it is random
<Tobi>and with multiple threads contending it is random too iirc
<zerver>yup, and since we have two threads here, fairness is not an issue
<Tobi>random sidenote: iirc in java a Lock spinlocks for a few hundred cycles and then does mutex
<zerver>so spinlock is, as I see it now, not an improvement
Conclusion: no change, boost::mutex will be kept
