[teh]Beherith[PiRO] has some questions to kloot related to roam + unsynced heightmap (and maybe others) abma: [teh]Beherith[PiRO]: your part :) hoijui: hey [teh]Beherith[PiRO]: hi [teh]Beherith[PiRO]: Will unsynced heightmap be enabled in next release? [teh]Beherith[PiRO]: I see its not toggleable modside hoijui: i think this is the plan, yes hoijui: that it will eb enabled [teh]Beherith[PiRO]: How will it act when mapdamage is off (many hosts have this cause of being able to see buildings and splosions) [teh]Beherith[PiRO]: I saw its a compile flag, but im guessing hosts will update to enable ground deformation with unsynced heightmap Kloot: it has no relation to mapdamage, just the visualization is different [teh]Beherith[PiRO]: my issue with it is, that I failed to get VBO-s working, so am currently building display lists of each bigtexsquare patch geomety [teh]Beherith[PiRO]: and I have hooked in explosions to update the patches [teh]Beherith[PiRO]: along with a slowupdate-like (one per 30%15+5 frames) recalculation of a single patch jK: there are new events for this [teh]Beherith[PiRO]: the current large load of roam is recalculating the variance tree [teh]Beherith[PiRO]: making a display list doesnt seem too expensive on my 8800gt jK: it will kill any ati [teh]Beherith[PiRO]: oh really? [teh]Beherith[PiRO]: damn jK: (creation & deletion of dlist is damn slow on them) [teh]Beherith[PiRO]: i guess ill just push to git and ask you where im failing with vbo-s [teh]Beherith[PiRO]: i implemented ROAM with vertex buffers as well, but its performace is bad compared to displaylists, too much BW abma: +1 :D abma: (for pushing) [teh]Beherith[PiRO]: also, will the unsynced heightmap update _only_ the parts that get freshly in view, or will it allow us to see a whole large crater if a unit is standing on its edge [teh]Beherith[PiRO]: ? jK: o_O jK: it will only update those parts that are in LOS [teh]Beherith[PiRO]: cause my testing seemed to show that the whole crater becomes visible at once Kloot: only those parts of the crater that are in its los [teh]Beherith[PiRO]: ( its more than a week old testing build, my bad) hoijui: maybe when it looked over the craters border? [teh]Beherith[PiRO]: so it will have areas with large cliffs hoijui: it saw it all at once Kloot: potentially yes, if the craters get deep enpough jK: sure there are cliffs and I already thought about to smooth them jK: but first bugfixing then fancy new things [teh]Beherith[PiRO]: i tested with a small LOS unit, and a very very large crater, and the whole thing came into view at once, no tearing [teh]Beherith[PiRO]: of course jK: use [L] [teh]Beherith[PiRO]: sharp cliffs cause high poly load due to the nature of ROAM [teh]Beherith[PiRO]: I used L, thanks :D [teh]Beherith[PiRO]: but its an old build, ill retest now [teh]Beherith[PiRO]: those are all the questions I have for now, thanks for your attention and time abma: thanks to you, too abma: soo.. next ILog - when/how to flush? hoijui: hmm.. hoijui: yep zerver: flush only when the app crashes hoijui: i think L_WARNING or L_ERROR and above hoijui: couldn't that be too late? Tobi: what zerver says IMHO jK: too late for what? hoijui: i am prety sure that flushign all L_WARNING, L_ERROR and L_FATAL records will never casue performance problems hoijui: if stack is corrupted for example jK: except you got a lua error that spamms 30MB hoijui: or something else.. jK: -m zerver: :) hoijui: yeah jK: (yeah I got infologs that were 130MB large) hoijui: that would also be a problem if you flush it later hoijui: or even more so then Tobi: in dedicated the size of errors didn't even matter, just flush might do bad things to performance with certain OSes/FSes jK: yep hoijui: dedicated shoudl not have lua errors also jK: if at all call it from other thread imo hoijui: so really never flush hoijui: ? hoijui: ahh.. ok.. now i finally get it.. why you want a thread Tobi: other thread is needless complexity IMHO, keep it at flush only in crash handler (just close the file cleanly...), and maybe in certain debug modes hoijui: ok Tobi: plus flush in other thread could still cause perf problems I suppose cause of locking (or is it safe to call fwrite and fflush on same stream at the same time from multiple threads?) hoijui: what we if disable flush for dedicated only? jK: don't think so tobi, still locking takes much less time then waiting for flush jK: *than Tobi: but if flush happens to take long, rest of app is still blocked on next log message hoijui: dont you have to lock during the whole flush time? hoijui: yeah that Tobi: unless we use a synchronized queue to pass log messages to a different thread that does all the writing ... but rather not jK: quite rarelly that you hit such a situation (in most cases there are like ~2-10 messages per min) jK: also the helper thread wouldn't directly flush, it would wait a few second before doin so -> large block of messages go through w/o locking zerver: separate thread is overkill imo, but yes that could be done Tobi: I don't like multithreaded things that happen to work "most of the time", but aren't designed to work "all the time" Tobi: you can't predict when the next message will come in jK: yeah, but the helper thread would flush max all 5-10 seconds -> the time of locking is reduced to nothing jK: so there is no "most of the time" hoijui: i alos dont like the thread thing Tobi: maybe yes, maybe not Tobi: if you want to introduce this extra complexity, sure go ahead, but I vote -1 Tobi: (btw, time of locking is only reduced to nothing if flushing takes no time, which implies it doesn't need to run in a thread at all ...) hoijui: i know.. hoijui: we have two extra thread hoijui: s hoijui: the main thread writes to a ring buffer in memory hoijui: the first logging thread writes from there to a file hoijui: and the third thread does the flushing Tobi: aaaaaaaaahhhhhh hoijui: :D abma: yeah, KISS-like hoijui: if the ring buffer is big enough, it will be so fast man! hoijui: iGenius zerver: we can make 3 monitoring threads to check that the flusing thread actually works abma: we should use a socket and a second process hoijui: so what abotu removing flush for deidcated only? hoijui: i am a bit worried with removign it compleetly cause.. hoijui: on windows we have no console output by default hoijui: so the only way to see log mesages woudl be to shutdown the engine hoijui: under normal conditions Tobi: can you actually open infolog.txt while spring has it open on windows? zerver: yes Tobi: kk zerver: remove flush for all imo, I just noticed how it degrades performence when I had lots of debug prints hoijui: debug prints appear on DEBUG builds only zerver: unless you modify the source zerver: if you would run spring on e.g. a thin client, network flusing could be real slow hoijui: but well... if you think it is better this way, then ok, lets remove it hoijui: aehh... hoijui: hoijui: k so.. remove flushing compleetly then, for now hoijui: "use C++ streams instead of the C way used now?" hoijui: i guess this does not make much difference, right? Tobi: both use same underlying API Tobi: just default buffering settings or so might be different hoijui: i just put that there cause i was a bit puzzled when .. wihtout a flush, it only wrote to file at process end hoijui: ok hoijui: yeah... Tobi: quite possibly it write after a while too if you log enough hoijui: ok zerver: yep hoijui: i am just used to streams on java, that they flush muhc more often.. or at least that is how i remember hoijui: ok ... Tobi: maybe they have a smaller buffer by default hoijui: thread safty (order of records? split records? ...) hoijui: mm hoijui: yeah so.. what do we need wiht thread safty exactly abma: on win32 you can open files with FILE_FLAG_NO_BUFFERING Tobi: making buffer size smaller is the correct solution btw if you would have the flush just to have stuff appear on console / in file hoijui: i am thinking on.. log messages might appear in different order on different sinks hoijui: mmm hoijui: .. i guess this is not really an issue hoijui: othe problems coudl be, a log message appearing in the middle of an other message hoijui: and then are the border cases when setting up/shutting down sinks hoijui: (which is not likely to be a problem in spring, as long as we dont add/remove sinks somewhere in the middle of the game) abma: hmm, keep it as it is and make a release and lets see if it makes problems? zerver: Each fprintf etc is atomic, so the lines could be interleaved, but that's it hoijui: ahh ok, nice hoijui: so we should be pretty much safe already zerver: It might be good to have some indication of what thread wrote the line, if it was not the main thread hoijui: hmmm zerver: I have seen interleaved stacktraces... hoijui: yeah can do that zerver: chained crash hoijui: how would i get that info? jK: interleaving messages are race conditions, not threadsafety abma: GetCurrentThreadId() ? hoijui: ok zerver: Threading::IsMainThread(), Threading::GetCurrentThreadId() etc hoijui: isnt there some other way, to get soemthign like "G" for the GML thread jK: erm -1 jK: it will just clutter hoijui: -1 for what exactly? jK: "some indication of what thread wrote the line" zerver: clutter is no issue, since 99.9% will be main thread hoijui: if it was a single char, i think it is no problem (plus max 3 chars for separation) hoijui: yeah that hoijui: well for now at least jK: before doing that introduce log_lock_sections hoijui: anyway.. we have like 12 or so chars wasted for frame hoijui: hm? zerver: I would say just print the thread id jK: (section that bind all logoutput to the code between log_lock_begin() & log_lock_end(), anyother log messages should get buffered then and appended in _end()) zerver: There is no IsRenderThread() atm, but there is a IsSimThread() hoijui: ok... jK: chat & infolog are already a pain to read jK: imo it needs less clutter and not more hoijui: well .. i dont see muhc point in thread id if it just telsl you that it is not main zerver: there is a point, you know which lines belong together hoijui: we can add the thread indication on L_WARNING+ only hoijui: jK: zerver, make sections hoijui: my solution is less clutter and adds more value hoijui: i didn ot get the section idea yet hoijui: ah.. jK: it allows to >lock< logoutput to just one thread hoijui: i think now i got it hoijui: dont like that Tobi: I suggest that if there will be log_lock_begin & log_lock_end it will buffer the messages of the *current* thread until the _end call, not all others Tobi: much simpler implementation wise hoijui: yeah true zerver: deadlock warning! Tobi: nah jK: yeah, but that can cause troubles when it is crashing in between hoijui: this does not need any locks jK: e.g. stacktrace translator crashes hoijui: not if we cleanup logging propperly jK: -> no stacktrace print hoijui: and if we dont do that, then ... Tobi: there's always the debugger in case it crashes so badly that the crash handler crashes hoijui: well.. we miguht have no log at all abma: next? hoijui: :D hoijui: i would do a textual ID fo the thread, if it is not main Tobi: [LCC]jK: same thing if other threads are buffered, then messages of those threads might be lost hoijui: no threading stuff hoijui: i dont think any loggin apis have soemthign like that log lock stuff Tobi: probably you can't solve this with a single process running unmanaged code, because any memory can be corrupted / any thread can crash at any time / etc. hoijui: if you really need it, you could make a wrapper over ILog jK: mostly other threads contain less important data than the code that locks it Tobi: how do you know two threads aren't in a log_lock_begin-end block at the same time? hoijui: really.. adding any threading stuff that deals only with logging .. seems quite insane to me hoijui: using threads/locks to make log messages appear in a different order... zerver: yeah, no thread specifically for logging Tobi: +1 zerver: could possibly use one of the existing threads though hoijui: SHT! hoijui: next zerver: but overkill hoijui: as the other guest is not here...
Release plan hoijui: that Tobi: (unless you actually *lock* the logging system, but that is deadlocky, as zerver mentioned already) hoijui: i will remove the flush and make sure that infolgo is not constantly appending to old file (on windows) hoijui: then we could do a test release, i think abma: +1 for the release :) zerver: yeah hoijui: and then it woudl be good to have some windows testers hoijui: test release, not release abma: :-/ hoijui: there are still some things open for release hoijui: UHM for example hoijui: and versioning stuff hoijui: and i think we had way too few windows testers yet abma: imo you don't get more with a test release hoijui: we can just ask one or two, and have soem games with them hoijui: but yeah.. guess we wont have huge ammounts :D abma: so why wait? hoijui: UHM for example hoijui: and versioning stuff abma: whats with versioning? abma: the {develop} tag? hoijui: new branching and versioning scheme needs to be done in cmake hoijui: adn some.. ahh explained this too often already :D abma: then these two points should be top prioritiy... hoijui: mmm hoijui: guess we are finnished? zerver: y hoijui: with the agenda at least hoijui: about the too long path on TheRockFinal problem... hoijui: coudl path finder use seomthign like ROAM for the low resolution map? hoijui: instead of equaly sized squares hoijui: equaly* Kloot: it could hoijui: so there is no theoretical impossibility there? or it beeing totally stupid? Kloot: but there are many practical problems with that approach (I thought about a rewrite like this a few times) hoijui: ok hoijui: like how ot find out which part a unit is on? Kloot: no, that would be e-z hoijui: ok abma: as you maybe have seen, now the version is displayed in mantis... abma: i hope its useful hoijui: what version? Kloot: but terrain deformations would become very very expensive, and the shortest path found across a single large square would always be a straight line abma: http://springrts.com/mantis/view_all_bug_page.php abma: "Product Version" hoijui: ahhh, as a column in the table hoijui: ok hoijui: mm k Kloot :/ hoijui: the second thing, abotu a straight line beeing hte shortest path.. is that a problem? Kloot: yes, if the subsquares within it have wildly different movement costs associated with them hoijui: ahh ok hoijui: wouldnt that be solved by the mid-res path finder? hoijui: or could that system only use two resolutions? hoijui: we have 3 reolutions now, right? hoijui: resolutions* Kloot: yep, but they don't intermix hoijui: ok Kloot: (at least for finding the initial path) jK: if it uses dynamic resolution maps, it won't have huge squares with "wildly different movement costs" jK: in such cases the name already says that it would split up such a square until the squares are homogeneous (all squares in it share the ~same costs) Kloot: yes, it would, because the tesselation would still be binary Kloot: pass/cannot pass hoijui: ah ok.. i also though about it more like jk did Kloot: maybe it could be based on some cost discretization scheme Kloot: but that would have to be carefully weighted or most leafs would just end up as 1x1 squares hoijui: mm hoijui: so it would be ~10 values instead of binary? Kloot: right jK: use 3 maps with different weights ^^ (joke) Kloot:
Is chat and other things also stored using ILog? In that case flush after every message... It's read on both server and client ends in zero-k (text to speech, planetwars, modstats etc)
Thats really silly, why should I lua something that engine should do with higher performance..
I need whole log instantly from dedicated server too, lua does not run there!!
You can make it async without delaying transmission, use async calls or make a new thread if there is no better way. Overhead of extra idle thread is zero - callstack in ram - maybe if it was used. Overhead of extra widgets and hacks needed to make engine work as before is likely to be much higher.
i had a talk with licho, and after some time we found out, that there is no problem really, as he only uses stdout, and the whole flushing/no-flushing business here is about the log-file only.
Users browsing this forum: No registered users and 0 guests
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot post attachments in this forum
Home
Download
Login
Register
FAQ
Search
