Dev meeting minutes 2011-09-26

Dev meeting minutes 2011-09-26

Minutes of the meetings between Spring developers are archived here.
Post Reply
User avatar
Spring Developer
Posts: 2299
Joined: 28 Jun 2007, 07:30

Dev meeting minutes 2011-09-26

Post by jK » 03 Oct 2011, 23:38

Date: 2011-09-26
Present: abma, hoijui, jK, Kloot, Tobi, zerver
Guests: [teh]Beherith[PiRO]

  • [teh]Beherith[PiRO] has some questions to kloot related to roam + unsynced heightmap (and maybe others)
    • ILog - when/how to flush?
  • Release plan

_The meeting________________________________________________________________
>> Tobi joined!
>> jK joined!
>> [teh]Beherith[PiRO] joined!
>> hoijui joined!
>> Kloot joined!
>> zerver joined!
>> abma joined!
abma: hi all! :)
zerver: hiall
Kloot: lo

[teh]Beherith[PiRO] has some questions to kloot related to roam + unsynced heightmap (and maybe others)
abma: [teh]Beherith[PiRO]: your part :)
hoijui: hey :-)
[teh]Beherith[PiRO]: hi
[teh]Beherith[PiRO]: Will unsynced heightmap be enabled in next release?
[teh]Beherith[PiRO]: I see its not toggleable modside
hoijui: i think this is the plan, yes
hoijui: that it will eb enabled
[teh]Beherith[PiRO]: How will it act when mapdamage is off (many hosts have this cause of being able to see buildings and splosions)
[teh]Beherith[PiRO]: I saw its a compile flag, but im guessing hosts will update to enable ground deformation with unsynced heightmap
Kloot: it has no relation to mapdamage, just the visualization is different
[teh]Beherith[PiRO]: my issue with it is, that I failed to get VBO-s working, so am currently building display lists of each bigtexsquare patch geomety
[teh]Beherith[PiRO]: and I have hooked in explosions to update the patches
[teh]Beherith[PiRO]: along with a slowupdate-like (one per 30%15+5 frames) recalculation of a single patch
jK: there are new events for this
[teh]Beherith[PiRO]: the current large load of roam is recalculating the variance tree
[teh]Beherith[PiRO]: making a display list doesnt seem too expensive on my 8800gt
jK: it will kill any ati
[teh]Beherith[PiRO]: oh really?
[teh]Beherith[PiRO]: damn
jK: (creation & deletion of dlist is damn slow on them)
[teh]Beherith[PiRO]: i guess ill just push to git and ask you where im failing with vbo-s
[teh]Beherith[PiRO]: i implemented ROAM with vertex buffers as well, but its performace is bad compared to displaylists, too much BW
abma: +1 :D
abma: (for pushing)
[teh]Beherith[PiRO]: also, will the unsynced heightmap update _only_ the parts that get freshly in view, or will it allow us to see a whole large crater if a unit is standing on its edge
[teh]Beherith[PiRO]: ?
jK: o_O
jK: it will only update those parts that are in LOS
[teh]Beherith[PiRO]: cause my testing seemed to show that the whole crater becomes visible at once
Kloot: only those parts of the crater that are in its los
[teh]Beherith[PiRO]: ( its more than a week old testing build, my bad)
hoijui: maybe when it looked over the craters border?
[teh]Beherith[PiRO]: so it will have areas with large cliffs
hoijui: it saw it all at once
Kloot: potentially yes, if the craters get deep enpough
jK: sure there are cliffs and I already thought about to smooth them
jK: but first bugfixing then fancy new things
[teh]Beherith[PiRO]: i tested with a small LOS unit, and a very very large crater, and the whole thing came into view at once, no tearing
[teh]Beherith[PiRO]: of course
jK: use [L]
[teh]Beherith[PiRO]: sharp cliffs cause high poly load due to the nature of ROAM
[teh]Beherith[PiRO]: I used L, thanks :D
[teh]Beherith[PiRO]: but its an old build, ill retest now
[teh]Beherith[PiRO]: those are all the questions I have for now, thanks for your attention and time
abma: thanks to you, too
abma: soo.. next
ILog - when/how to flush?
hoijui: hmm..
hoijui: yep
zerver: flush only when the app crashes
hoijui: i think L_WARNING or L_ERROR and above
hoijui: couldn't that be too late?
Tobi: what zerver says IMHO
jK: too late for what?
hoijui: i am prety sure that flushign all L_WARNING, L_ERROR and L_FATAL records will never casue performance problems
hoijui: if stack is corrupted for example
jK: except you got a lua error that spamms 30MB
hoijui: or something else..
jK: -m
zerver: :)
hoijui: yeah :P
jK: (yeah I got infologs that were 130MB large)
hoijui: that would also be a problem if you flush it later
hoijui: or even more so then
Tobi: in dedicated the size of errors didn't even matter, just flush might do bad things to performance with certain OSes/FSes
jK: yep
hoijui: dedicated shoudl not have lua errors also
jK: if at all call it from other thread imo
hoijui: so really never flush
hoijui: ?
hoijui: ahh.. ok.. now i finally get it.. why you want a thread ;-)
Tobi: other thread is needless complexity IMHO, keep it at flush only in crash handler (just close the file cleanly...), and maybe in certain debug modes
hoijui: ok
Tobi: plus flush in other thread could still cause perf problems I suppose cause of locking (or is it safe to call fwrite and fflush on same stream at the same time from multiple threads?)
hoijui: what we if disable flush for dedicated only?
jK: don't think so tobi, still locking takes much less time then waiting for flush
jK: *than
Tobi: but if flush happens to take long, rest of app is still blocked on next log message
hoijui: dont you have to lock during the whole flush time?
hoijui: yeah that
Tobi: unless we use a synchronized queue to pass log messages to a different thread that does all the writing ... but rather not
jK: quite rarelly that you hit such a situation (in most cases there are like ~2-10 messages per min)
jK: also the helper thread wouldn't directly flush, it would wait a few second before doin so -> large block of messages go through w/o locking
zerver: separate thread is overkill imo, but yes that could be done
Tobi: I don't like multithreaded things that happen to work "most of the time", but aren't designed to work "all the time"
Tobi: you can't predict when the next message will come in
jK: yeah, but the helper thread would flush max all 5-10 seconds -> the time of locking is reduced to nothing
jK: so there is no "most of the time"
hoijui: i alos dont like the thread thing
Tobi: maybe yes, maybe not
Tobi: if you want to introduce this extra complexity, sure go ahead, but I vote -1
Tobi: (btw, time of locking is only reduced to nothing if flushing takes no time, which implies it doesn't need to run in a thread at all ...)
hoijui: i know..
hoijui: we have two extra thread
hoijui: s
hoijui: the main thread writes to a ring buffer in memory
hoijui: the first logging thread writes from there to a file
hoijui: and the third thread does the flushing
Tobi: aaaaaaaaahhhhhh
hoijui: :D
abma: yeah, KISS-like
hoijui: if the ring buffer is big enough, it will be so fast man!
hoijui: iGenius
zerver: we can make 3 monitoring threads to check that the flusing thread actually works
abma: we should use a socket and a second process
hoijui: so what abotu removing flush for deidcated only?
hoijui: i am a bit worried with removign it compleetly cause..
hoijui: on windows we have no console output by default
hoijui: so the only way to see log mesages woudl be to shutdown the engine
hoijui: under normal conditions
Tobi: can you actually open infolog.txt while spring has it open on windows?
zerver: yes
Tobi: kk
zerver: remove flush for all imo, I just noticed how it degrades performence when I had lots of debug prints
hoijui: debug prints appear on DEBUG builds only
zerver: unless you modify the source
zerver: if you would run spring on e.g. a thin client, network flusing could be real slow
hoijui: but well... if you think it is better this way, then ok, lets remove it
hoijui: aehh...
hoijui: :P
hoijui: k so.. remove flushing compleetly then, for now
hoijui: "use C++ streams instead of the C way used now?"
hoijui: i guess this does not make much difference, right?
Tobi: both use same underlying API
Tobi: just default buffering settings or so might be different
hoijui: i just put that there cause i was a bit puzzled when .. wihtout a flush, it only wrote to file at process end
hoijui: ok :-)
hoijui: yeah...
Tobi: quite possibly it write after a while too if you log enough
hoijui: ok
zerver: yep
hoijui: i am just used to streams on java, that they flush muhc more often.. or at least that is how i remember
hoijui: ok ...
Tobi: maybe they have a smaller buffer by default
hoijui: thread safty (order of records? split records? ...)
hoijui: mm
hoijui: yeah so.. what do we need wiht thread safty exactly
abma: on win32 you can open files with FILE_FLAG_NO_BUFFERING
Tobi: making buffer size smaller is the correct solution btw if you would have the flush just to have stuff appear on console / in file
hoijui: i am thinking on.. log messages might appear in different order on different sinks
hoijui: mmm
hoijui: .. i guess this is not really an issue
hoijui: othe problems coudl be, a log message appearing in the middle of an other message
hoijui: and then are the border cases when setting up/shutting down sinks
hoijui: (which is not likely to be a problem in spring, as long as we dont add/remove sinks somewhere in the middle of the game)
abma: hmm, keep it as it is and make a release and lets see if it makes problems?
zerver: Each fprintf etc is atomic, so the lines could be interleaved, but that's it
hoijui: ahh ok, nice :-)
hoijui: so we should be pretty much safe already
zerver: It might be good to have some indication of what thread wrote the line, if it was not the main thread
hoijui: hmmm
zerver: I have seen interleaved stacktraces...
hoijui: yeah can do that
zerver: chained crash :P
hoijui: how would i get that info?
jK: interleaving messages are race conditions, not threadsafety
abma: GetCurrentThreadId() ?
hoijui: ok
zerver: Threading::IsMainThread(), Threading::GetCurrentThreadId() etc
hoijui: isnt there some other way, to get soemthign like "G" for the GML thread
jK: erm -1
jK: it will just clutter
hoijui: -1 for what exactly?
jK: "some indication of what thread wrote the line"
zerver: clutter is no issue, since 99.9% will be main thread
hoijui: if it was a single char, i think it is no problem (plus max 3 chars for separation)
hoijui: yeah that
hoijui: well for now at least
jK: before doing that introduce log_lock_sections
hoijui: anyway.. we have like 12 or so chars wasted for frame
hoijui: hm?
zerver: I would say just print the thread id
jK: (section that bind all logoutput to the code between log_lock_begin() & log_lock_end(), anyother log messages should get buffered then and appended in _end())
zerver: There is no IsRenderThread() atm, but there is a IsSimThread()
hoijui: ok...
jK: chat & infolog are already a pain to read
jK: imo it needs less clutter and not more
hoijui: well .. i dont see muhc point in thread id if it just telsl you that it is not main
zerver: there is a point, you know which lines belong together :P
hoijui: we can add the thread indication on L_WARNING+ only
hoijui: :P
jK: zerver, make sections
hoijui: my solution is less clutter and adds more value
hoijui: i didn ot get the section idea yet
hoijui: ah..
jK: it allows to >lock< logoutput to just one thread
hoijui: i think now i got it
hoijui: dont like that
Tobi: I suggest that if there will be log_lock_begin & log_lock_end it will buffer the messages of the *current* thread until the _end call, not all others
Tobi: much simpler implementation wise
hoijui: yeah true
zerver: deadlock warning!
Tobi: nah
jK: yeah, but that can cause troubles when it is crashing in between
hoijui: this does not need any locks
jK: e.g. stacktrace translator crashes
hoijui: not if we cleanup logging propperly
jK: -> no stacktrace print
hoijui: and if we dont do that, then ...
Tobi: there's always the debugger in case it crashes so badly that the crash handler crashes
hoijui: well.. we miguht have no log at all
abma: next?
hoijui: :D
hoijui: i would do a textual ID fo the thread, if it is not main
Tobi: [LCC]jK: same thing if other threads are buffered, then messages of those threads might be lost
hoijui: no threading stuff
hoijui: i dont think any loggin apis have soemthign like that log lock stuff
Tobi: probably you can't solve this with a single process running unmanaged code, because any memory can be corrupted / any thread can crash at any time / etc.
hoijui: if you really need it, you could make a wrapper over ILog
jK: mostly other threads contain less important data than the code that locks it
Tobi: how do you know two threads aren't in a log_lock_begin-end block at the same time?
hoijui: really.. adding any threading stuff that deals only with logging .. seems quite insane to me
hoijui: using threads/locks to make log messages appear in a different order...
zerver: yeah, no thread specifically for logging
Tobi: +1
zerver: could possibly use one of the existing threads though
hoijui: SHT!
hoijui: next
zerver: but overkill
hoijui: as the other guest is not here...

Release plan
hoijui: that
Tobi: (unless you actually *lock* the logging system, but that is deadlocky, as zerver mentioned already)
hoijui: i will remove the flush and make sure that infolgo is not constantly appending to old file (on windows)
hoijui: then we could do a test release, i think
abma: +1 for the release :)
zerver: yeah
hoijui: and then it woudl be good to have some windows testers
hoijui: test release, not release
abma: :-/
hoijui: there are still some things open for release
hoijui: UHM for example
hoijui: and versioning stuff
hoijui: and i think we had way too few windows testers yet
abma: imo you don't get more with a test release
hoijui: we can just ask one or two, and have soem games with them
hoijui: but yeah.. guess we wont have huge ammounts :D
abma: so why wait?
hoijui: UHM for example
hoijui: and versioning stuff
abma: whats with versioning?
abma: the {develop} tag?
hoijui: new branching and versioning scheme needs to be done in cmake
hoijui: adn some.. ahh explained this too often already :D
abma: then these two points should be top prioritiy...
hoijui: mmm
hoijui: guess we are finnished?
zerver: y
hoijui: with the agenda at least
hoijui: about the too long path on TheRockFinal problem...
hoijui: coudl path finder use seomthign like ROAM for the low resolution map? ;-)
hoijui: instead of equaly sized squares
hoijui: equaly*
Kloot: it could
hoijui: so there is no theoretical impossibility there? or it beeing totally stupid?
Kloot: but there are many practical problems with that approach (I thought about a rewrite like this a few times)
hoijui: ok
hoijui: like how ot find out which part a unit is on?
Kloot: no, that would be e-z
hoijui: ok
abma: as you maybe have seen, now the version is displayed in mantis...
abma: i hope its useful
hoijui: what version?
Kloot: but terrain deformations would become very very expensive, and the shortest path found across a single large square would always be a straight line
abma: "Product Version"
hoijui: ahhh, as a column in the table :-)
hoijui: ok :-)
hoijui: mm k Kloot :/
hoijui: the second thing, abotu a straight line beeing hte shortest path.. is that a problem?
Kloot: yes, if the subsquares within it have wildly different movement costs associated with them
hoijui: ahh ok
hoijui: wouldnt that be solved by the mid-res path finder?
hoijui: or could that system only use two resolutions?
hoijui: we have 3 reolutions now, right?
hoijui: resolutions*
Kloot: yep, but they don't intermix
hoijui: ok
Kloot: (at least for finding the initial path)
jK: if it uses dynamic resolution maps, it won't have huge squares with "wildly different movement costs"
jK: in such cases the name already says that it would split up such a square until the squares are homogeneous (all squares in it share the ~same costs)
Kloot: yes, it would, because the tesselation would still be binary
Kloot: pass/cannot pass
hoijui: ah ok.. i also though about it more like jk did
Kloot: maybe it could be based on some cost discretization scheme
Kloot: but that would have to be carefully weighted or most leafs would just end up as 1x1 squares
hoijui: mm
hoijui: so it would be ~10 values instead of binary?
Kloot: right
jK: use 3 maps with different weights ^^ (joke)
Kloot: :x
0 x

User avatar
Zero-K Developer
Posts: 3803
Joined: 19 May 2006, 19:13

Re: Dev meeting minutes 2011-09-26

Post by Licho » 04 Oct 2011, 03:02

Is chat and other things also stored using ILog? In that case flush after every message...
It's read on both server and client ends in zero-k (text to speech, planetwars, modstats etc)
0 x

User avatar
Spring Developer
Posts: 2299
Joined: 28 Jun 2007, 07:30

Re: Dev meeting minutes 2011-09-26

Post by jK » 04 Oct 2011, 09:57

everything is.
you can either write a lua widget to pipe it to a file, or pipe bash output.
0 x

User avatar
Zero-K Developer
Posts: 3803
Joined: 19 May 2006, 19:13

Re: Dev meeting minutes 2011-09-26

Post by Licho » 04 Oct 2011, 19:01

Thats really silly, why should I lua something that engine should do with higher performance..

I need whole log instantly from dedicated server too, lua does not run there!!

You can make it async without delaying transmission, use async calls or make a new thread if there is no better way. Overhead of extra idle thread is zero - callstack in ram - maybe if it was used. Overhead of extra widgets and hacks needed to make engine work as before is likely to be much higher.
0 x

User avatar
Former Engine Dev
Posts: 4342
Joined: 22 Sep 2007, 09:51

Re: Dev meeting minutes 2011-09-26

Post by hoijui » 04 Oct 2011, 23:57

i had a talk with licho, and after some time we found out, that there is no problem really, as he only uses stdout, and the whole flushing/no-flushing business here is about the log-file only.
0 x

Post Reply

Return to “Meeting Minutes”