Dev meeting minutes 2011-09-26

Dev meeting minutes 2011-09-26

Minutes of the meetings between Spring developers are archived here.
Post Reply
User avatar
jK
Spring Developer
Posts: 2299
Joined: 28 Jun 2007, 07:30

Dev meeting minutes 2011-09-26

Post by jK »

Date: 2011-09-26
Present: abma, hoijui, jK, Kloot, Tobi, zerver
Guests: [teh]Beherith[PiRO]


_Agenda_____________________________________________________________________
  • [teh]Beherith[PiRO] has some questions to kloot related to roam + unsynced heightmap (and maybe others)
    • ILog - when/how to flush?
  • Release plan


_The meeting________________________________________________________________
>> Tobi joined!
>> jK joined!
>> [teh]Beherith[PiRO] joined!
>> hoijui joined!
>> Kloot joined!
>> zerver joined!
>> abma joined!
abma: hi all! :)
zerver: hiall
Kloot: lo

[teh]Beherith[PiRO] has some questions to kloot related to roam + unsynced heightmap (and maybe others)
abma: [teh]Beherith[PiRO]: your part :)
hoijui: hey :-)
[teh]Beherith[PiRO]: hi
[teh]Beherith[PiRO]: Will unsynced heightmap be enabled in next release?
[teh]Beherith[PiRO]: I see its not toggleable modside
hoijui: i think this is the plan, yes
hoijui: that it will eb enabled
[teh]Beherith[PiRO]: How will it act when mapdamage is off (many hosts have this cause of being able to see buildings and splosions)
[teh]Beherith[PiRO]: I saw its a compile flag, but im guessing hosts will update to enable ground deformation with unsynced heightmap
Kloot: it has no relation to mapdamage, just the visualization is different
[teh]Beherith[PiRO]: my issue with it is, that I failed to get VBO-s working, so am currently building display lists of each bigtexsquare patch geomety
[teh]Beherith[PiRO]: and I have hooked in explosions to update the patches
[teh]Beherith[PiRO]: along with a slowupdate-like (one per 30%15+5 frames) recalculation of a single patch
jK: there are new events for this
[teh]Beherith[PiRO]: the current large load of roam is recalculating the variance tree
[teh]Beherith[PiRO]: making a display list doesnt seem too expensive on my 8800gt
jK: it will kill any ati
[teh]Beherith[PiRO]: oh really?
[teh]Beherith[PiRO]: damn
jK: (creation & deletion of dlist is damn slow on them)
[teh]Beherith[PiRO]: i guess ill just push to git and ask you where im failing with vbo-s
[teh]Beherith[PiRO]: i implemented ROAM with vertex buffers as well, but its performace is bad compared to displaylists, too much BW
abma: +1 :D
abma: (for pushing)
[teh]Beherith[PiRO]: also, will the unsynced heightmap update _only_ the parts that get freshly in view, or will it allow us to see a whole large crater if a unit is standing on its edge
[teh]Beherith[PiRO]: ?
jK: o_O
jK: it will only update those parts that are in LOS
[teh]Beherith[PiRO]: cause my testing seemed to show that the whole crater becomes visible at once
Kloot: only those parts of the crater that are in its los
[teh]Beherith[PiRO]: ( its more than a week old testing build, my bad)
hoijui: maybe when it looked over the craters border?
[teh]Beherith[PiRO]: so it will have areas with large cliffs
hoijui: it saw it all at once
Kloot: potentially yes, if the craters get deep enpough
jK: sure there are cliffs and I already thought about to smooth them
jK: but first bugfixing then fancy new things
[teh]Beherith[PiRO]: i tested with a small LOS unit, and a very very large crater, and the whole thing came into view at once, no tearing
[teh]Beherith[PiRO]: of course
jK: use [L]
[teh]Beherith[PiRO]: sharp cliffs cause high poly load due to the nature of ROAM
[teh]Beherith[PiRO]: I used L, thanks :D
[teh]Beherith[PiRO]: but its an old build, ill retest now
[teh]Beherith[PiRO]: those are all the questions I have for now, thanks for your attention and time
abma: thanks to you, too
abma: soo.. next
ILog - when/how to flush?
hoijui: hmm..
hoijui: yep
zerver: flush only when the app crashes
hoijui: i think L_WARNING or L_ERROR and above
hoijui: couldn't that be too late?
Tobi: what zerver says IMHO
jK: too late for what?
hoijui: i am prety sure that flushign all L_WARNING, L_ERROR and L_FATAL records will never casue performance problems
hoijui: if stack is corrupted for example
jK: except you got a lua error that spamms 30MB
hoijui: or something else..
jK: -m
zerver: :)
hoijui: yeah :P
jK: (yeah I got infologs that were 130MB large)
hoijui: that would also be a problem if you flush it later
hoijui: or even more so then
Tobi: in dedicated the size of errors didn't even matter, just flush might do bad things to performance with certain OSes/FSes
jK: yep
hoijui: dedicated shoudl not have lua errors also
jK: if at all call it from other thread imo
hoijui: so really never flush
hoijui: ?
hoijui: ahh.. ok.. now i finally get it.. why you want a thread ;-)
Tobi: other thread is needless complexity IMHO, keep it at flush only in crash handler (just close the file cleanly...), and maybe in certain debug modes
hoijui: ok
Tobi: plus flush in other thread could still cause perf problems I suppose cause of locking (or is it safe to call fwrite and fflush on same stream at the same time from multiple threads?)
hoijui: what we if disable flush for dedicated only?
jK: don't think so tobi, still locking takes much less time then waiting for flush
jK: *than
Tobi: but if flush happens to take long, rest of app is still blocked on next log message
hoijui: dont you have to lock during the whole flush time?
hoijui: yeah that
Tobi: unless we use a synchronized queue to pass log messages to a different thread that does all the writing ... but rather not
jK: quite rarelly that you hit such a situation (in most cases there are like ~2-10 messages per min)
jK: also the helper thread wouldn't directly flush, it would wait a few second before doin so -> large block of messages go through w/o locking
zerver: separate thread is overkill imo, but yes that could be done
Tobi: I don't like multithreaded things that happen to work "most of the time", but aren't designed to work "all the time"
Tobi: you can't predict when the next message will come in
jK: yeah, but the helper thread would flush max all 5-10 seconds -> the time of locking is reduced to nothing
jK: so there is no "most of the time"
hoijui: i alos dont like the thread thing
Tobi: maybe yes, maybe not
Tobi: if you want to introduce this extra complexity, sure go ahead, but I vote -1
Tobi: (btw, time of locking is only reduced to nothing if flushing takes no time, which implies it doesn't need to run in a thread at all ...)
hoijui: i know..
hoijui: we have two extra thread
hoijui: s
hoijui: the main thread writes to a ring buffer in memory
hoijui: the first logging thread writes from there to a file
hoijui: and the third thread does the flushing
Tobi: aaaaaaaaahhhhhh
hoijui: :D
abma: yeah, KISS-like
hoijui: if the ring buffer is big enough, it will be so fast man!
hoijui: iGenius
zerver: we can make 3 monitoring threads to check that the flusing thread actually works
abma: we should use a socket and a second process
hoijui: so what abotu removing flush for deidcated only?
hoijui: i am a bit worried with removign it compleetly cause..
hoijui: on windows we have no console output by default
hoijui: so the only way to see log mesages woudl be to shutdown the engine
hoijui: under normal conditions
Tobi: can you actually open infolog.txt while spring has it open on windows?
zerver: yes
Tobi: kk
zerver: remove flush for all imo, I just noticed how it degrades performence when I had lots of debug prints
hoijui: debug prints appear on DEBUG builds only
zerver: unless you modify the source
zerver: if you would run spring on e.g. a thin client, network flusing could be real slow
hoijui: but well... if you think it is better this way, then ok, lets remove it
hoijui: aehh...
hoijui: :P
hoijui: k so.. remove flushing compleetly then, for now
hoijui: "use C++ streams instead of the C way used now?"
hoijui: i guess this does not make much difference, right?
Tobi: both use same underlying API
Tobi: just default buffering settings or so might be different
hoijui: i just put that there cause i was a bit puzzled when .. wihtout a flush, it only wrote to file at process end
hoijui: ok :-)
hoijui: yeah...
Tobi: quite possibly it write after a while too if you log enough
hoijui: ok
zerver: yep
hoijui: i am just used to streams on java, that they flush muhc more often.. or at least that is how i remember
hoijui: ok ...
Tobi: maybe they have a smaller buffer by default
hoijui: thread safty (order of records? split records? ...)
hoijui: mm
hoijui: yeah so.. what do we need wiht thread safty exactly
abma: on win32 you can open files with FILE_FLAG_NO_BUFFERING
Tobi: making buffer size smaller is the correct solution btw if you would have the flush just to have stuff appear on console / in file
hoijui: i am thinking on.. log messages might appear in different order on different sinks
hoijui: mmm
hoijui: .. i guess this is not really an issue
hoijui: othe problems coudl be, a log message appearing in the middle of an other message
hoijui: and then are the border cases when setting up/shutting down sinks
hoijui: (which is not likely to be a problem in spring, as long as we dont add/remove sinks somewhere in the middle of the game)
abma: hmm, keep it as it is and make a release and lets see if it makes problems?
zerver: Each fprintf etc is atomic, so the lines could be interleaved, but that's it
hoijui: ahh ok, nice :-)
hoijui: so we should be pretty much safe already
zerver: It might be good to have some indication of what thread wrote the line, if it was not the main thread
hoijui: hmmm
zerver: I have seen interleaved stacktraces...
hoijui: yeah can do that
zerver: chained crash :P
hoijui: how would i get that info?
jK: interleaving messages are race conditions, not threadsafety
abma: GetCurrentThreadId() ?
hoijui: ok
zerver: Threading::IsMainThread(), Threading::GetCurrentThreadId() etc
hoijui: isnt there some other way, to get soemthign like "G" for the GML thread
jK: erm -1
jK: it will just clutter
hoijui: -1 for what exactly?
jK: "some indication of what thread wrote the line"
zerver: clutter is no issue, since 99.9% will be main thread
hoijui: if it was a single char, i think it is no problem (plus max 3 chars for separation)
hoijui: yeah that
hoijui: well for now at least
jK: before doing that introduce log_lock_sections
hoijui: anyway.. we have like 12 or so chars wasted for frame
hoijui: hm?
zerver: I would say just print the thread id
jK: (section that bind all logoutput to the code between log_lock_begin() & log_lock_end(), anyother log messages should get buffered then and appended in _end())
zerver: There is no IsRenderThread() atm, but there is a IsSimThread()
hoijui: ok...
jK: chat & infolog are already a pain to read
jK: imo it needs less clutter and not more
hoijui: well .. i dont see muhc point in thread id if it just telsl you that it is not main
zerver: there is a point, you know which lines belong together :P
hoijui: we can add the thread indication on L_WARNING+ only
hoijui: :P
jK: zerver, make sections
hoijui: my solution is less clutter and adds more value
hoijui: i didn ot get the section idea yet
hoijui: ah..
jK: it allows to >lock< logoutput to just one thread
hoijui: i think now i got it
hoijui: dont like that
Tobi: I suggest that if there will be log_lock_begin & log_lock_end it will buffer the messages of the *current* thread until the _end call, not all others
Tobi: much simpler implementation wise
hoijui: yeah true
zerver: deadlock warning!
Tobi: nah
jK: yeah, but that can cause troubles when it is crashing in between
hoijui: this does not need any locks
jK: e.g. stacktrace translator crashes
hoijui: not if we cleanup logging propperly
jK: -> no stacktrace print
hoijui: and if we dont do that, then ...
Tobi: there's always the debugger in case it crashes so badly that the crash handler crashes
hoijui: well.. we miguht have no log at all
abma: next?
hoijui: :D
hoijui: i would do a textual ID fo the thread, if it is not main
Tobi: [LCC]jK: same thing if other threads are buffered, then messages of those threads might be lost
hoijui: no threading stuff
hoijui: i dont think any loggin apis have soemthign like that log lock stuff
Tobi: probably you can't solve this with a single process running unmanaged code, because any memory can be corrupted / any thread can crash at any time / etc.
hoijui: if you really need it, you could make a wrapper over ILog
jK: mostly other threads contain less important data than the code that locks it
Tobi: how do you know two threads aren't in a log_lock_begin-end block at the same time?
hoijui: really.. adding any threading stuff that deals only with logging .. seems quite insane to me
hoijui: using threads/locks to make log messages appear in a different order...
zerver: yeah, no thread specifically for logging
Tobi: +1
zerver: could possibly use one of the existing threads though
hoijui: SHT!
hoijui: next
zerver: but overkill
hoijui: as the other guest is not here...

Release plan
hoijui: that
Tobi: (unless you actually *lock* the logging system, but that is deadlocky, as zerver mentioned already)
hoijui: i will remove the flush and make sure that infolgo is not constantly appending to old file (on windows)
hoijui: then we could do a test release, i think
abma: +1 for the release :)
zerver: yeah
hoijui: and then it woudl be good to have some windows testers
hoijui: test release, not release
abma: :-/
hoijui: there are still some things open for release
hoijui: UHM for example
hoijui: and versioning stuff
hoijui: and i think we had way too few windows testers yet
abma: imo you don't get more with a test release
hoijui: we can just ask one or two, and have soem games with them
hoijui: but yeah.. guess we wont have huge ammounts :D
abma: so why wait?
hoijui: UHM for example
hoijui: and versioning stuff
abma: whats with versioning?
abma: the {develop} tag?
hoijui: new branching and versioning scheme needs to be done in cmake
hoijui: adn some.. ahh explained this too often already :D
abma: then these two points should be top prioritiy...
hoijui: mmm
hoijui: guess we are finnished?
zerver: y
hoijui: with the agenda at least
hoijui: about the too long path on TheRockFinal problem...
hoijui: coudl path finder use seomthign like ROAM for the low resolution map? ;-)
hoijui: instead of equaly sized squares
hoijui: equaly*
Kloot: it could
hoijui: so there is no theoretical impossibility there? or it beeing totally stupid?
Kloot: but there are many practical problems with that approach (I thought about a rewrite like this a few times)
hoijui: ok
hoijui: like how ot find out which part a unit is on?
Kloot: no, that would be e-z
hoijui: ok
abma: as you maybe have seen, now the version is displayed in mantis...
abma: i hope its useful
hoijui: what version?
Kloot: but terrain deformations would become very very expensive, and the shortest path found across a single large square would always be a straight line
abma: http://springrts.com/mantis/view_all_bug_page.php
abma: "Product Version"
hoijui: ahhh, as a column in the table :-)
hoijui: ok :-)
hoijui: mm k Kloot :/
hoijui: the second thing, abotu a straight line beeing hte shortest path.. is that a problem?
Kloot: yes, if the subsquares within it have wildly different movement costs associated with them
hoijui: ahh ok
hoijui: wouldnt that be solved by the mid-res path finder?
hoijui: or could that system only use two resolutions?
hoijui: we have 3 reolutions now, right?
hoijui: resolutions*
Kloot: yep, but they don't intermix
hoijui: ok
Kloot: (at least for finding the initial path)
jK: if it uses dynamic resolution maps, it won't have huge squares with "wildly different movement costs"
jK: in such cases the name already says that it would split up such a square until the squares are homogeneous (all squares in it share the ~same costs)
Kloot: yes, it would, because the tesselation would still be binary
Kloot: pass/cannot pass
hoijui: ah ok.. i also though about it more like jk did
Kloot: maybe it could be based on some cost discretization scheme
Kloot: but that would have to be carefully weighted or most leafs would just end up as 1x1 squares
hoijui: mm
hoijui: so it would be ~10 values instead of binary?
Kloot: right
jK: use 3 maps with different weights ^^ (joke)
Kloot: :x
User avatar
Licho
Zero-K Developer
Posts: 3803
Joined: 19 May 2006, 19:13

Re: Dev meeting minutes 2011-09-26

Post by Licho »

Is chat and other things also stored using ILog? In that case flush after every message...
It's read on both server and client ends in zero-k (text to speech, planetwars, modstats etc)
User avatar
jK
Spring Developer
Posts: 2299
Joined: 28 Jun 2007, 07:30

Re: Dev meeting minutes 2011-09-26

Post by jK »

everything is.
you can either write a lua widget to pipe it to a file, or pipe bash output.
User avatar
Licho
Zero-K Developer
Posts: 3803
Joined: 19 May 2006, 19:13

Re: Dev meeting minutes 2011-09-26

Post by Licho »

Thats really silly, why should I lua something that engine should do with higher performance..

I need whole log instantly from dedicated server too, lua does not run there!!

You can make it async without delaying transmission, use async calls or make a new thread if there is no better way. Overhead of extra idle thread is zero - callstack in ram - maybe if it was used. Overhead of extra widgets and hacks needed to make engine work as before is likely to be much higher.
User avatar
hoijui
Former Engine Dev
Posts: 4344
Joined: 22 Sep 2007, 09:51

Re: Dev meeting minutes 2011-09-26

Post by hoijui »

i had a talk with licho, and after some time we found out, that there is no problem really, as he only uses stdout, and the whole flushing/no-flushing business here is about the log-file only.
Post Reply

Return to “Meeting Minutes”