Present: hoijui, Tobi, zerver, abma, Auswaschbar, jK, Kloot
===Agenda===
* Welcome
* Release plan
* Bugs to fix for the next release
* The big refactor
* Next meeting
=== Welcome ===
[RoX]Tobi: fyi I can be in meeting but will be preparing dinner in the mean time :)
[ARP]hoijui_g5: hello
zerver: hi
[ARP]hoijui_g5:
zerver: agenda looks promising, we can start with Anything else? (WVTTK)
[ARP]hoijui_g5: well.. the release thign shoudl eb wuite some
[ARP]hoijui_g5: and if it is not, there is still what i wrote in the Notices
[ARP]hoijui_g5: the big refacter
[ARP]hoijui_g5: should be quite some*
[ARP]hoijui_g5: did jk say he wont come?
[ARP]hoijui_g5: Tobi said he might have some time.. but .. maybe he is afk, or doing other stuff
[RoX]Tobi: I am present but will be preparing dinner in the mean time :)
[ARP]hoijui_g5: ok
[ARP]hoijui_g5: gues we shoudl start?
zerver: y
[ARP]hoijui_g5: ouh.. someoen for minutes
[ARP]hoijui_g5: abma?
abma: y
abma:
[ARP]hoijui_g5: ok.. thanks
[ARP]hoijui_g5: you also want to be the .. real time meeting manager for today?
[ARP]hoijui_g5: should i?
abma: i could do it next time
[ARP]hoijui_g5: ok
[ARP]hoijui_g5: yeah.. hello everyone...
=== Release plan ===
zerver: Wow we just released, good job
[ARP]hoijui_g5: guess we shoudl talk about... the bugs, and when to make a new release
[ARP]hoijui_g5: :D
[ARP]hoijui_g5: as we all know, it is just an RC, really
zerver: Yeah, will probably be a new release within a week
[ARP]hoijui_g5: mm
zerver: Ok, the bugs...
[ARP]hoijui_g5: the path finding and the no-sync-checking for spring-dedicated are probably the worst bugs
[ARP]hoijui_g5: kloot... is pathfinding fixed with the disablign of heatmap?
[ARP]hoijui_g5: Kloot*
zerver: i played on many hosts and > 70% seem to spit the sync disabled error
[ARP]hoijui_g5: ah and after these.. probably spring-mt desyncing?
Kloot: yes, most of the pf issues reported are
[ARP]hoijui_g5: ok
zerver: yeah, i had one or two desyncs
[ARP]hoijui_g5: and.. you think it is a good solution, or you want to do somethign else before a new release?
Kloot: I would call it a hotfix :)
[ARP]hoijui_g5: the no-sync-checking surely is fixed with my commit .. the autohosts where you do not get that warning .. i guess BD/bibim already use the fix there, or they are using the old version still
[ARP]hoijui_g5: ok
Kloot: but heatmapping needs a bit more work to be useful
Kloot: right now it creates one more problem than it solves
[ARP]hoijui_g5: zerver, maybe we can do a sync-debug run after the meeting? .. somewhen today
[ARP]hoijui_g5: mm :/
zerver: first think to check is that compiler flags for both builds are identical
[ARP]hoijui_g5: mmm
[ARP]hoijui_g5: ok.. well .. well check that after meeting
zerver: yeah
===Bugs to fix for the next release===
[ARP]hoijui_g5: what other bugs shoudl we care for?
zerver: there is area reclaim, but that is pretty minot
Kloot: area reclaim and projectile rendering
Kloot: and maybe one the 432857328957 ATI regressions
zerver: invisible projectiles typically caused by zero alpha, due to a missing SetTeamColor()
[RoX]Tobi: the servers on springrts.com use fixed dedicated server indeed (many relay hosts and springies)
Kloot: I know, but team color is always set for all of the projectile types that have models
[RoX]Tobi: if we want to make a bugfix release, not that I will be on vacation from next weekend till ~31 august
[RoX]Tobi: *note
zerver: fixed wrt sync check disable?
[RoX]Tobi: yes
zerver: k
zerver: we need to force all to update
zerver: bump protocol version
[ARP]hoijui_g5: yeah.. that is reuqired anyway for kloots pathign change
zerver: kloot: yeah it seems all those have SetTeamColor, so maybe something else this time
[ARP]hoijui_g5: regarding ATI regressions:
[RoX]Tobi: so that would be 0.82.4 then
[ARP]hoijui_g5: here:
[ARP]hoijui_g5: Spring\LuaUI\Config\BA.lua
[ARP]hoijui_g5: aehh.. http://springrts.com/phpbb/viewtopic.ph ... 94#p443994
[ARP]hoijui_g5: one guy reports he has to delete Spring\LuaUI\Config\BA.lua every game, for spring not to crash
[ARP]hoijui_g5: might help finding ..
[ARP]hoijui_g5: hey jk
[ARP]hoijui_g5: we already started
[LCC]jK: hi
[LCC]jK: sorry for being late
[ARP]hoijui_g5: were talkign about which bugs are important
[ARP]hoijui_g5: most important are already (hot-)fixed in master (pathing, DS sycn checking)
zerver: mt desync, projectile rendering, area reclaim ...
[ARP]hoijui_g5: zerver and me will have a look at mt desycn later
zerver: WRT mt desync, something that might cause it is tampering with synced data from a rendering thread
zerver: even a temporary data change is enough
[ARP]hoijui_g5: you have an idea where that might happen?
zerver: no, not right now. compiler flags, static variable...
zerver: i probably desynced more games than i think, because of sync checking disabled
[ARP]hoijui_g5: mm
zerver: is the compiler same now as with 0.81 ?
[ARP]hoijui_g5: soo. kloot, jk.. any ideas abotu some of the AIT problem yet?
[ARP]hoijui_g5: newer mingw
zerver: kk
Kloot: I've many ideas yes, but no way to test them
[ARP]hoijui_g5: also newer boost i think
[ARP]hoijui_g5: mm :/
Kloot: at some point I'll have to get an ati+win32 machine I guess
[ARP]hoijui_g5: licho offers an ATI for 6 months
[ARP]hoijui_g5: mmm
Kloot: yeah that's a nice offer, but I don't like making promises I don't know I can keep
[ARP]hoijui_g5: mm
[ARP]hoijui_g5: maybe .. someone in the spring community is willign to offer his old system when upgrading
Kloot: plus fixing all issues with ATI's different driver versions is likely a fulltime job >_>
[ARP]hoijui_g5: hehe :D
[ARP]hoijui_g5: yeah.. we woudl of course, not make any guarantees
[ARP]hoijui_g5: warantiees?
[ARP]hoijui_g5: promises
zerver: yeah, ATI needs to hire some real programmers
zerver: i bet it outsourced driver development to china
Kloot: those are all in the direct3d department
[LCC]jK: their opengl drivers are written by monkeys
zerver: pretty leet monkeys though
[ARP]hoijui_g5: i could setup a thread, askign for donatiing ATI systems/cards, explaining the ATI core problem (bad Opengl ...), and suggesting to bug ATI to push OpenGL support
[LCC]jK: ATI doesn't push OpenGL and they never will
[LCC]jK: the only thing they do it fixing stuff after they got happened (after releases of games etc.)
[LCC]jK: while NVidia helps the developer teams before the release
[ARP]hoijui_g5: yeah.. that is not our business.. its just to make sure everyone knows whos to blame
zerver: we can take over ATIs OGL driver development and make some $$$
[ARP]hoijui_g5: as right now, it is the spring engine dev teams fault.. practically always
[ARP]hoijui_g5: mmm jk should apply for that job :D
[ARP]hoijui_g5: though... they will check.. and find out that he is an nv fanboi :D
[ARP]hoijui_g5: aehm...
[LCC]jK: like those linux gurus working at m$?
[ARP]hoijui_g5: mmm
[ARP]hoijui_g5: yeah.. woudl be too bad for you i guess
[ARP]hoijui_g5: but .. lets get real agian...
[ARP]hoijui_g5: are we done wiht bugs discussing?
[ARP]hoijui_g5: is there any ATI bug that needs to be in next version?
zerver: i'm happy and im running ati
zerver: but i never use the fancy gfx anyway
[ARP]hoijui_g5: ok, i guess.. thats a no then
abma: i've ati too... with low settings i've few problems
[ARP]hoijui_g5: the green-screen thing?
Kloot: I suppose 2034 is somewhat critical
abma: no bug, that makes the game unplayable
zerver: yeah, that has been in for ages
[LCC]jK: the green screen thing is a widget issues imo
abma: [ARP]hoijui_g5: yep, but only when i'm playing in windowed mode
[ARP]hoijui_g5: mm ok
zerver: i remember tropical being particularly affected by green tinge
zerver: sometimes disappears at start
[LCC]jK: that's a diff green screen
[ARP]hoijui_g5: maybe we shoudl tell Licho to try with 10.7b, as this seems to fix some stuff?
Kloot: minimap startboxes caused that afaik
[LCC]jK: that bug is old
[ARP]hoijui_g5: or does it only fix the textures thing?
zerver: ok
Kloot: 10.7 still has the 3do bug according to 2033
[RoX]Tobi: would be nice btw if you could set target version to 0.82.4 for bugs that should really be fixed in that version
[RoX]Tobi: that we we can track progress on mantis' roadmap page
[ARP]hoijui_g5: mm ok
[ARP]hoijui_g5: Kloot, but i heard 10.7b does not
[RoX]Tobi: I'm updating some more issues now
Kloot: oh, didn't know there was a 'b' out
[ARP]hoijui_g5: i think it is somewhat unofficial/hidden
[ARP]hoijui_g5: it was linked ot in a forum thread somewhere
[ARP]hoijui_g5: to*
[ARP]hoijui_g5: no idea where
[ARP]hoijui_g5: ok so.. should we talk about the big refactor?
zerver: y
abma: y
[LCC]jK: when will the stacktrace translator be ready?
[ARP]hoijui_g5: i have a java app for the meantime
[ARP]hoijui_g5: you want it?
[LCC]jK: java .. no ^^
[RoX]Tobi: unknown, I've it on my todo but I don't know if I get around to it before vacation
[LCC]jK: k
[ARP]hoijui_g5: pff.. this comminuty.. really!
[ARP]hoijui_g5: too much java hate
[ARP]hoijui_g5: ok.. so the refactor..
[ARP]hoijui_g5: one thing...
Conclusions:
Try to fix all bugs before tobi is on vacation from next weekend:
http://springrts.com/mantis/roadmap_page.php
=== The big refactor ===
[ARP]hoijui_g5: Tobi, you wrote you do not want rendering to depend on Sim
[ARP]hoijui_g5: (except maybe through events)
[ARP]hoijui_g5: why? or say.. i think that woudl be.. unpractical
[RoX]Tobi: no
[RoX]Tobi: I mean I don't want that rendering depends on Sim to call methods in rendering code
[RoX]Tobi: e.g. I don't want some CreateUnit in sim to call unitRendering->addunit()
[ARP]hoijui_g5: ahaa ok
[ARP]hoijui_g5: taht solved then
zerver: RenderUnitCreated... solved already
[ARP]hoijui_g5: i added some more at the end of the big reafactoring etherpad
[ARP]hoijui_g5: regardign sim/rendering separation
[ARP]hoijui_g5: might make it into a DDD forum post
[RoX]Tobi: (in other words, it should theoretically be possible to still compile sim code, without any changes, after deleting all rendering code
[ARP]hoijui_g5: mm
[ARP]hoijui_g5: yeah .. agree
Kloot: yeah and it would be nice if everybody worked toward that goal...
Kloot: every time someone #includes grounddecalhandler somewhere I die a little inside
[LCC]jK: the opposite directions is much harder to avoid
[LCC]jK: -s
[ARP]hoijui_g5: :D kloot
[ARP]hoijui_g5: yeah... i guess we agree, that the opposite direction is ok
[RoX]Tobi: opposite direction is ok
zerver: groundDecalHandler needs a rewrite then...
[ARP]hoijui_g5: of course, it have to be const calls
[RoX]Tobi: although it would be nice if rendering would get data from some clear interfaces in sim
[ARP]hoijui_g5: mmm
[RoX]Tobi: but that is less important than avoiding calls from sim directly into rendering, imo
[ARP]hoijui_g5: maybe.. read what i have under "Criteria" in the etherpad, and discuss that
zerver: can u pm me the etherpad link?
[ARP]hoijui_g5: done
zerver: thx
[ARP]hoijui_g5:
Modus (-v [ARP]hoijui_g5) von taspringmaster.clan-sy.com
zerver: "Two complete Sim states, a const one, and a new one (getting actualized by the Sim thread)?"
zerver: I guess the "getting actualized" is the tricky part
[ARP]hoijui_g5: hmm..
zerver: from a performance point of view i mean
[ARP]hoijui_g5: why?
zerver: lots of units moving around, much data to copy
[ARP]hoijui_g5: i mean.. that is not different then now, is it?
zerver: it is, not we copy no data at all
zerver: *now
[ARP]hoijui_g5: ahh k
Kloot: the simulation state is _huge_
[LCC]jK: that just happens 30times per second
[LCC]jK: also it doesn't need to copy everything esp. heightmap etc.
[ARP]hoijui_g5: 30 times a second.. it only hcnages values.. no?
[RoX]Tobi: would at least need something like copy on write so you don't need to copy 100s of features/units/other state that isn't modified at all most frames
zerver: yeah, big game --> huge sim state
[ARP]hoijui_g5: mm
[RoX]Tobi: possibly slight inaccuracies in rendering do not matter
[RoX]Tobi: i.e. if some heightmap pixels are already in their value for the next frame
[LCC]jK: the problem I see is more how to get the render threads using such decoy sim objects?
[ARP]hoijui_g5: why decoy? .. would be equal, except that they do not changed anymore
[ARP]hoijui_g5: how would copy on write work?
[ARP]hoijui_g5: would need 3 states, no?
zerver: that is too big to copy
zerver: especially since spring is a cpu whore already
zerver: only 30 times a second does not help
[LCC]jK: with copy on write you can't use memcpy, also you need a lot more if-clauses
zerver: i wonder how it would look with less frequent copying
[LCC]jK: I don't expect that the important sim states are bigger than 10-30MB
[RoX]Tobi: yeah with all the lack of encapsulation it will be much work to add it
[ARP]hoijui_g5: if it could be one block in memory.. :D
[LCC]jK: CUnit + CMovetypeAI < 100kB?
[ARP]hoijui_g5: we coudl use memcpy
zerver: yeah, some kind of memory pool with contiguous objects
Kloot: that still breaks when copying objects with member pointers to other objects
[ARP]hoijui_g5: mm
zerver: then each object needs a pointer to "rendering related sim data", no pointers allowed
zerver: rofl
zerver: yeah, need three states for double buffering
[ARP]hoijui_g5: hmm?
[LCC]jK: double buffering would save mutex locking
zerver: otherwise synchronization will take away too much CPU
zerver: yeah, render thread renders from one state while sim updates the other
[ARP]hoijui_g5: that measn, all sim code always has to update a new state (all values in it)?
[ARP]hoijui_g5: even if the vlaue is not really changed (in a logical way)
zerver: yep
[ARP]hoijui_g5: hmm..
[ARP]hoijui_g5: sounds liek best solution so far...
[ARP]hoijui_g5: what are the downsides?
zerver: to determine what has changed takes too much time
zerver: anyway, it is hard to determine what is "essential data" for rendering
[ARP]hoijui_g5: all code has to be rewritten? :D
[LCC]jK: essential data would e.g. be unit pos
zerver: especially with LUA, some ppl may not agree on the limited set of data we chose for rendering
[LCC]jK: unitdefid is essential too
[ARP]hoijui_g5: why would we have to limit that?
[LCC]jK: why limit it?
[LCC]jK: you can still fallback to mutex locking for non-buffered data
[LCC]jK: just try to limit the time spend in it
zerver: because it is complex structures containing pointers that cannot be copied
[ARP]hoijui_g5: mm
[ARP]hoijui_g5: pointers again :/
zerver: for example the unit models, pieces and mechanics, not sure how to copy that
zerver: obvious this is a huge refactor
[ARP]hoijui_g5: umm... my brian is shutting off agian, going to eat
[ARP]hoijui_g5: this solution shoudl be written down. maybe there i nthe etherpad
[ARP]hoijui_g5: may try to do it later, if i can get it together
[ARP]hoijui_g5: and noone else did
zerver: i think it will fail tbh
[ARP]hoijui_g5: mm :/
[ARP]hoijui_g5: well.. it seems quite godo to me, except the pointer problem..
[ARP]hoijui_g5: maybe we find a solution for that
[ARP]hoijui_g5: till later!
[ARP]hoijui_g5: ha! Auswaschbar is on!
[ARP]hoijui_g5: maybe he has an idea.. invite?
zerver: heil auswasch oh mighty :)
[LCC]jK: cu need to go
zerver: anyway wrt MT desync, i tried reconnect after desync and it did not desync that time
[ARP]hoijui_g5: invited Aus
[ARP]hoijui_g5: he did not answer yet
zerver: so this is a non-deterministic desync
[ARP]hoijui_g5: gone*
zerver: hard to reproduce
[ARP]hoijui_g5: mm
zerver: probably draw/sim messing with some shared data
zerver: hi auswach!
Auswaschbar: long time no see
zerver: indeed
[ARP]hoijui_g5: yeah.. hey! :D (food not ready yet :/ )
[ARP]hoijui_g5: ok so...
[ARP]hoijui_g5: for separting sim/rendering, we were tlaking about using 2 or 3 sim states
[ARP]hoijui_g5: one "finalized"/unchangeable, used by rendering, and the other(s) used by sim only
[ARP]hoijui_g5: a problem with most solutions seems to be pointers to other data
[LCC]jK: the problem is more how to decide which one is used where
[LCC]jK: esp. much code runs in synced and unsynced code
[LCC]jK: -code +enviroment
[RoX]Tobi: could probably base it on a TLS global
zerver: yeah
Auswaschbar: I'm not up to date, but why do you want to use 2 seperate states for sim and gfx?
[ARP]hoijui_g5: for multithreading
[ARP]hoijui_g5: yes
[ARP]hoijui_g5: aehh no.. it is only sim state
[ARP]hoijui_g5: like.. different versions of the sim state
zerver: double buffered
[LCC]jK: Auswaschbar, that's the discussion :)
[ARP]hoijui_g5: eg.. current frame (used by rendering) and nextframe
zerver: to have several complete sim states is not doable imo, too complicated to copy
zerver: it has to be a very limited set of data, otherwise keep the current design
[LCC]jK: yup, buffering the most important data to reduce mutex lock times
zerver: a limited, pointer free, set of data, that is the conclusion for now i think
zerver: i also need to eat something
Auswaschbar: so like this? http://img46.imageshack.us/img46/4686/flowcharti.png
zerver: almost, it needs three states
zerver: sim is updating one, while rendering has exclusive access to the other
zerver: then it switches
zerver: this eliminates the need for repeated mutex lockings and such
[ARP]hoijui_g5: hmm.. that with limited data state.. soudns way too complex for me
[RoX]Tobi: this buffer could be per object, right?
[ARP]hoijui_g5: there will be lockign requried o pointer related stuff too.. and so .. will be a complete mess, no?
Auswaschbar: well, wouldn't you only need to lock when swapping buffers?
zerver: one way to determine what the limited data needs to be, is to look at what data rendering is accessing right now...
zerver: yes, one lock per swap
[RoX]Tobi: it seems to me that with full internal encapsulation and some smart class design it's possible to make classes that essentially make a copy of all their data members the first time they're modified since some time
[RoX]Tobi: and have some kind of special reference objects fullfilling the role "I'm a pointer to the old state of unit 1276"
[ARP]hoijui_g5: hmm.. and they could do that by using memcpy + actualizing pointers?
[ARP]hoijui_g5: sounds good...
[RoX]Tobi: since it would remain the same instance of the class (from outside), pointers remain constant
[RoX]Tobi: internally it would have separate "storages" for it's data members
[RoX]Tobi: no lock would be needed on the initial copy since it would just be 2 threads reading the same data
[RoX]Tobi: and after that 1 thread continues reading the oldState data
[RoX]Tobi: and the new one makes modifications in the newState data
[RoX]Tobi: btw does accessing TLS data have any significant performance impact? (like, at least 10x time to access normal global vars?)
[RoX]Tobi: maybe after dinner I can try to prototype a small example
[RoX]Tobi: since I doubt it is completely clear from this chatter only
[RoX]Tobi: going to eat now, bbl
[ARP]hoijui_g5: perfect tobi
[ARP]hoijui_g5: looks like nobody saw a flaw so far.. (or all were too hungry)
[RoX]Tobi: lol yeah
abma: ok, next point
[ARP]hoijui_g5: hehe :D :D
===Next meeting===
[ARP]hoijui_g5: smae agian for me...
===Anything else? (WVTTK)===
abma: we should talk about progress of the bug fixing for the current release
abma: i would want to talk about an meeting with mod-devs
abma: and i have some ideas for unitsync
abma: tobi is on vacation from next weekend till ~31 august
[ARP]hoijui_g5: ahh yeah :/
) structure
