Present: zerver, jK, hoijui, abma, Tobi
=== Welcome ===
<hoijui> hey!
<Tobi> yo
<zerver> hi
<hoijui> lets just start wiht stuff we dont need the others for
=== push Tobis rapid client ===
* put into installer?
* include install instructions on download page?
<zerver> any reason not to?
<hoijui> i heard quite soem time, that users would like to get stuff through rapis, but dont want to use Zero-K
<hoijui> mm :D
<hoijui> i dont know any
<zerver> im one of those
<hoijui> Tobi.. is there some simple.. script or soemthing we coudl put into the installer?
<zerver> zero k wanter 2GB disk space...
<zerver> *wanted
<hoijui> something that.. installs pip and fetches rapid or something?
<Tobi> probably one of those py2exe executables aegis and/or lurker made
<hoijui> or how woudl you do it?
<hoijui> ahhh
<hoijui> mm sounds good
<zerver> has to be exe yes, otherwise no good
<Tobi> but personally I haven't got around to looking whether I can reproduce that build myself
<hoijui> ok
<hoijui> i guess this method lacks updatability?
<Tobi> yeah, atm I can not make windows releases
<zerver> need a build bot
<hoijui> ok
<hoijui> well... maybe for this release, we coudl just copy an exe from aegis or lurker on /dl/, and include it into the installer
<Tobi> that's possible yeah
<hoijui> ok.. i'll go after that then
<hoijui> next?
<Tobi> since I really don't know when I get around to it that may be the best solution for now indeed
<hoijui> ok
<hoijui> mm also seems pretty usable/stabel to me, as it is.. so this shoudl be fine
<abma> rapid for windows: http://springrts.com/phpbb/viewtopic.php?f=12&t=23699 (+instructions how to create it)
<abma> imho py2exe only works on windows
=== un-support solid archives? ===
<zerver> what is that?
<hoijui> i got told, that in the past, we did not suport solid archives, but now we do
<hoijui> soid archives are quite bad for mods and maps, aren't they?
<hoijui> like.. when you want to look for ModInfo.lua only
<hoijui> so i suggest, removign suport for solid archives, or at least show some big ugly warning
<Tobi> it might be we didn't support them because the 7-zip library was too old
<hoijui> mmm
<Tobi> in any case a warning doesn't hurt
<hoijui> yeah.. maybe first a warning... and for next next release, remove it
<hoijui> support for it, i mean
<hoijui> btw, in case someone doesnt know what solid archive means:
<Tobi> if it can be disabled using the new library..
<hoijui> means, you have to decompress the whole archive to get one file
<hoijui> mmm :/
<Tobi> yeah
<hoijui> in the worst case, we had to patch it
<zerver> ah, like tar files
<hoijui> didnt know tar is like that :D
<Tobi> technically tar isn't, but only .tar.something :)
<hoijui> ahh ok,.. yeah then
<hoijui> so yeah... only topic left is: Release Plan
<zerver> ya
<hoijui> but Kloot is not here.. so.. makes no sense
<hoijui> lua split alos makes no sense
<hoijui> without jk.. and was also discussed last week already
<hoijui> so.. is there soemthign else?
<zerver> abma, thanks for bug report, however that bug shouldnt be discussed here
<hoijui> am i also able to restart buildbot if it had to be?
<hoijui> i see there are targets for that in the Makefile under ~buildbot/
<hoijui> hmm... guess i am, as user buildbot :D
<Tobi> yup
<Tobi> 'make stop start' would restart everything
<hoijui> mm ok
<hoijui> soo.. should we call it over?
<hoijui> one thing maybe...
<hoijui> cherry picking is not so fun anymore
<hoijui> too much difference between release branch and master already
<hoijui> maybe we shoudl adopt all of that other model of using git
<hoijui> as in.. have all fixes in branches
<hoijui> instead of in develop
<jK> hi
<jK> sorry that I am late, but my whole new pc is broken :<
<zerver>
<jK> took me some time to reinstall my 10yo old one components
<hoijui> aww :/
<hoijui> hey
<hoijui> well.. we are already done
<hoijui> except.. you want to discuss lua split a 3rd time wiht zerver
<jK> *sniff* it cost me 300 euros, hope that minimum the gpu gets under warranty :'<
<hoijui> it really all blew up?
<jK> mainboard destroyed everything :<
<hoijui> ahh :/
<jK> gpu, mobo, cpu
<jK> just ram is unchecked
<hoijui> discs?
<jK> the rest is 100% destructed
<zerver> how did all blow up?
<jK> ah hdd works too
<zerver> power surge?
<hoijui> mm good
<jK> yeah
<hoijui> i use USV on all my systems.. need to here
<hoijui> had that two times in the far past, everythign gone
<jK> a psu connector was loose and got unplugged while the pc ran -> boom
<hoijui> :/
<hoijui> an internal one? of from PSU to power?
<jK> PSU -> mobo
<zerver> sounds like poor design, but i guess it is hard to make it error tolerant
<jK> it was one of those ATX2.x connectors
<jK> normally it didn't started when it wasn't connected
<jK> but I never thought that it would destruct everything when it gets unplugged ...
<jK> stupid mainboard
<zerver> must be some MB voltage regulation stuff that went nuts
<hoijui> do you have anythign to discuss jk? we kind of already ended the meeting
=== lua split ===
<zerver> lua split :)
<hoijui> :D
<hoijui> i cant imaigne he is in the mood for that now
<hoijui> it tends to get frustrating
<jK> now I don't think that the inet shop will recoup all componets, so it will be very expensive for me :<
<hoijui> mmm
<zerver> [00:12:22] <[LCC]jK> THERE ARE NO SUCH GADGETS
<hoijui> :D D:
<zerver> i just remembered there is kloots healtbar gadget also
<zerver> so gfx gadgets do exist...
<jK> hmm no kloot here
<hoijui> jk, make a foto of you on the ground, and the molten parts of your machine around you.. and you sobbing.. loking up to god to ask "why!!?!?"
<hoijui> and use it to beg for money
<jK> lol
<hoijui> donations for spring deving
<hoijui> one free feature request for every 50euros or so
<jK> rofl
<abma> :-D
<jK> I still hope to get ~2 (of the 4) components recouped
<jK> *1-2
<jK> minimum the gpu :<
<jK> it cost the same as mobo+cpu
<jK> already waisted 60euros on a redundant cpu just to check if the mobo still works :/
<hoijui> mmm
<zerver> i found kloot's gadget https://github.com/spring/spring/blob/m ... efault.lua
<zerver> we could remove support for gadget based rendering, then maybe lua split isnt needed
<zerver> otherwise, i was hoping we could at least make a decision today as to whether synced lua needs to be split or not
<zerver> and then discuss the merging of my branch later...
<jK> btw kloot's gadget is just a fallback solution
<hoijui> in Java, all GUI rendering is done in a separate thread, which onyl does rendering (and leighteirght parts of event handling)
<jK> there is the healthbars widget which is used by 99% of all ppls
<zerver> thats good ofc
<zerver> but it exists, and like i said sennas mod also has rendering gadgets
<jK> rendering alone is not a reason
<jK> much time spend on rendering is
<zerver> yeah
<zerver> iirc kloot said he had to make this a gadget for some reason, i think it was he needed luaRules->DrawUnit
<jK> also the unsynced luagadgets will be split no matter what sooner or later when I got the time & energy to do so
<jK> you don't need DrawUnit for healthbars ...
<zerver> just some optimization i guess
<jK> the widget exists and it works and it is highly optimized
<jK> nah, just for 1:1 copying of the way how the engine it does -> lazy dev'ing
<zerver> i see
<zerver> but if some mod needs some lua gfx that the user must not be allowed to disable, it must be a gadget right?
<jK> not really
<jK> technically it can be done in LuaUI too
<jK> just the current widgethandler has no easy solution for it
<jK> (but I work on a new one with such features)
<zerver> but /luaui disable
<jK> and?
<jK> luaui disable breaks nearly all mods
<jK> it is just a developer debug command, to test if a widget is interfering with the engine
<jK> -,
<zerver> i was just trying to say that luaui is something optional extra by nature
<zerver> and it can crash too ofc
<jK> no it isn't anymore
<jK> LuaUI gets always autoloaded
<jK> you can just disable it at runtime, but you can't at gameloading
<zerver> yes, i know it ignores the LuaUI=0 setting
<hoijui> the only argument agasint this, woudl be if soem users woudl have a legitimite reason to play wiht luaui disabled
<hoijui> is there a reason for that?
<zerver> old buggy gfx driver :)
<hoijui> as jk siad, there is practically no mod except BA that would support playing that way
<hoijui> yeah but mod devs wont adjust their mods to work that way
<hoijui> i mean... if it crashes becuase of an other widget, you coudl siable that widget, if it crashes becuase of any drawing.. it woudl also likely crash if that same drawing was done by a gadget, no?
<zerver> yup
<zerver> would it be easy to disallow gadget rendering?
<jK> ...
<hoijui> yeah that coudl be good
<zerver> if we do that, then im fine with commiting only the batching stuff like jK suggested
<jK> you should stop to think about disallowing things, instead you should think how to get things working how they are atm
<jK> I know that many modders currently use gadgets to render gameplay important things
<zerver> yes.. but to get this working 100% requires a split for synced lua
<hoijui> adn if they'd move that to LuaUI, their mods woudl run better wiht MT spring?
<zerver> without split, yes
<zerver> with split, gadget rendering is fine with MT
<zerver> So, situation is, gadget rendering will run bad with MT and solutions are:
<zerver> * disallow gadget rendering
<zerver> * split synced lua
<hoijui> ok
<zerver> any other solution, otherwise I suggest vote on this now
<hoijui> jk.. you say.. moving drawing to luaui is not good? rather wait for the split?
<hoijui> from my very distant point of view, having all drawing done in the same place is cleaner
<jK> modders won't adjust anything in their mods
<jK> the maximum you can expect from them is to replace file A with file B
<jK> or to do a regex replaces
<jK> -a
<zerver> oh darn lazy modders :)
<hoijui> what if you make some optional "mode", default disabled, that disallows rendering in non LuaAI
<hoijui> or gives error messages
<jK> you underestimate the amount of lines of lua code
<hoijui> then moddevs coudl check their mods for MT compatibility
<hoijui> so if they want thier mods to work well wiht MT, they can adjust, an otherwise, they have to prevent their users to use MT, until the split is around
<hoijui> that wouls make things clear for the transition period
<hoijui> .. i guess, having all rendering done in luaui is not a problem when split states get introduced?
<zerver> mmm... this can be done one step at a time of course
<zerver> but i was hoping for a vote for/against splitting
<hoijui> jk?
<hoijui> why are you so absent? :D
<Tobi> imo only trepan-style split should be done, it's not really clear to me what kind of split is being discussed now
<jK> because I mentioned my opinion on this topic multiple times already
<zerver> the split discussed is Sim/Draw split to enable MT to run gadget based rendering without problems
<hoijui> jk, youre split would be trepan style?
<hoijui> your*
<jK> yup
<Tobi> so after this split unsynced gadget Lua can't use LuaSyncedRead anymore?
<abma> ehm.. its a bit off-topic, but i'll have to go soon:
<abma> i want to promote a bit, that spring is an engine and there are no mods, only games
<abma> maybe some of you already saw, that i've edited the wiki a bit, that in most important parts, only "game" is used
<abma> on wikipedia i changed this, too
<abma> hopefully this will make it cleaner for other sites, when they link to us... i saw many sites, still linking to us, saying spring is a game
<Tobi> nice
<abma> (yes, "ta spring" was a game, but thats history...)
<abma> </promotion>
<jK> I don't understand external source name spring as a game nor why they still name it "ta spring"
<jK> *sources
<zerver> i guess trepan style split does not solve MT problems?
<jK> IMO the wiki is very specfic in that case (that spring is an engine and that there is no "ta spring" anymore)
<zerver> hoi, do you vote?
<zerver> abma?
<hoijui> aehm...
<zerver> you like to disallow gadget rendering
<hoijui> does trepan style split solve MT drawing slowness?
<zerver> no, that was something completely different iirc
<hoijui> ok
<jK> "solve" is relative
<jK> it helps a lot
<zerver> im looking for the minutes of the meeting where we discussed it
<Tobi> it opens the door to a proper solution as I see it
<hoijui> and jk, what do you think about suggesting to move drawing to luaui?
<hoijui> i dont think that we should right away disallow it in gadgets
<Tobi> luaui-draw can still call LuaSyncedRead right?
<zerver> with trepan split?
<Tobi> trepan split wouldn't affect LuaUI at all
<hoijui> but suggesting it, and making it easy to see where it is not done yet, would allow mode devs to make their stuff run fast with MT if they want to invest time in that
<Tobi> I mean in the situation where LuaUI events are batched
<Tobi> so sim doesn't need to wait
<zerver> batching works fine i think
<Tobi> but the call-ins can still call out into LuaSyncedRead, right?
<hoijui> so.. is there any argument against suggesting/helping mod devs to move drawing to luaui?
<hoijui> jk?
<zerver> it has some performance disadvantages in the sense that all events must be pumped when objects are deleted, to make sure no invalid IDs exist in the bahc
<zerver> and... my split only affects the MT build to begin with
<jK> it doesn't make sense to move all rendering to lua
<jK> esp. when it is related to things gadgets do
<zerver> i'm voting to split, but could change my vote to disable gadget rendering if enough ppl support that
<jK> first you should apply the batching
<zerver> the batching is fine, we are not voting about that now
<hoijui> aehm.. ok i guess the luaui rendering idea is dead then
<hoijui> and .. as i understand, nobody is agaisnt a split, question is just if it should be jk/trepan split, or zerver split
<Tobi> I vote against zerver split because I have serious doubts wrt maintainability at introducing different code path after different code path. In my opinion the best approach would be to 1) trepan-split unsynced/synced gadgets, 2) batch LuaUI and unsynced gadgets, 3) try giving each Lua state its own heap and 4) implement some kind of buffering of simdata so no simlock is necessary when calling LuaSyncedRead stuff in LuaUI/unsynced gadget.
<jK> +1
<zerver> hm, and who is willing to code that?
<hoijui> +1
<zerver> im suggesting flow field path finding with neural networks
<hoijui> i guess MT would be mianly fixed wiht 2) ?
<jK> (1) is my job, (2) already done by you, (3) is Tobi, (4) is shared work
<abma> +1 from me for tobis suggestion...
<Tobi> 4) is super hard without lots of cleanup and refactoring I think, at least I wouldn't really know how to approach that in a good way yet
<zerver> im just thinking this will never happen
<abma> sorry, i've to go... bye
<jK> but (4) is already a current problem
<jK> just zerver ignored it
<jK> so it is independent
<Tobi> well I know that there's almost nil chance *I* will get around to it, but since my opinion was desired I've given it
<Tobi> (re zerver)
<zerver> thx
<zerver> not sure what simlock you are talking about though
<zerver> locking only occurs on deletion of objects, there is no penalty for invoking luasyncedread
<Tobi> I already found spring has way too many code paths long ago with the 100s of config settings, 1000s of unit variables and interaction between all of them, and I know that for maintainability it would be best to reduce this
<Tobi> I assumed there must be some kind of locking when reading sim data so sim doesn't modify it in the mean time
<zerver> no, it reads data that is partially modified
<zerver> so this boils down to whether treapan style split solves MT problems
<hoijui> so 4 is not needed?
<jK> (4) is needed to avoid runtime crashes/errors
<Tobi> and that works 100% reliably for stuff that may get re-allocated and for iterators that are invalidated in the mean time? (e.g. command queue)
<hoijui> that would mean, the first 3 would have to solve the MT isuses together?
<hoijui> mm k
<zerver> not for this, only for BIG REFACTOR
<zerver> does trepan style split allow concurrent access for sim/draw to luarules/luagaia?
<Tobi> yes
<Tobi> since both would be a separate state
<zerver> ok
<Tobi> so draw callins can run at same time as sim callins
<Tobi> though SendToUnsynced and SYNCED are the problematic things there, as we identified a few meetings ago iirc :)
<zerver> hmm, it sounds like there is very little difference between my split and trepan then
<Tobi> it's just that locks need to be taken when modifying any non thread safe data structure in sim (e.g. at least all of the STL)
<jK> zerver, a trepan-style split is just a `clean` version of yours
<jK> which helps to maintain it
<zerver> ok, so help clean up mine then
<jK> more work than to begin with zero
<Tobi> btw I heard a mutex lock/unlock is ~4 times faster then a main memory reference, which kinda changed my picture on using a lock occasionally vs having megabytes of state duplicated
<hoijui> mmm
<jK> "main memory reference"
<zerver> *uncached* main memory reference
<Tobi> hence main memory reference yeah
<hoijui> yeah.. is a bit vague
<Tobi> http://axisofeval.blogspot.com/2010/11/ ... -know.html
<hoijui> i though that practically never happens these dasy
<jK> you mean thread local storage?
<hoijui> as CPUs are smart enough to have most stuff in cache
<hoijui> reading a value from ram
<hoijui> is what i though of
<zerver> should trepan split be applied to single thread build also?
<Tobi> yes
<Tobi> I bet spring is full of cache misses atm, afaik cache coherency and stuff like that has never been considered seriously for any spring code
<zerver> i would still suggest to base a trepan style split on my branch simply because that code exists now
<zerver> and it has the SYNCED.EXPORT and SendToUnsynced batching already, so why rewrite that?
<jK> I once tried to optimize data locality in heatmapping and it showed that cache misses take less time than the needed time spend on additional math to avoid them
<Tobi> interesting
<jK> (I tried to split a huge array into small squares of AxA boxes)
<Tobi> yeah that is a way I know about too, I tried to make such a bitmap class sometime but got bored halfway
<Tobi> iirc the math was basically just one modulo extra.. could hurt though if modulo is expensive
<hoijui> related: http://springrts.com/wiki/Engine_Profiling
<hoijui> made this wiki page
<jK> yeah, the math was quite simple still too expensive
<hoijui> instructions for profiling spring under linux
<hoijui> with BDs instructions
<jK> iirc 1 modulo and 1 sub
<zerver> So decision is: trepan style split and batching?
<jK> +1
<zerver> Someone tell me what I need to clean up to conform with trepan splitting
<jK> move the splitting to LuaHandleSynced and optimize SYNCED further
<hoijui> it looks like jk does just nto trust you to do it right
<hoijui> am i wrong?
<jK> for further optimizations you need further lua knowledge
<zerver> so why was trepans patch so huge?
<jK> hehe you should be lucky that it wasn't commited
<zerver> does this not indicate that my split in luahandle is cleaner?
<jK> there was a lot stuff in it that makes multithreading even harder
<jK> still his ideas were brilliant
<jK> so LuaRules was able to call LuaUI functions in sandboxes
<hoijui> btw, if someone wants me to do profiling of something, i can do that, and send png or SVG
<jK> e.g. for profiling/timing
<hoijui> i cant do it with debug symbols though.. it runs forever (say, isntead of 10s, it runs for more then 4h)
<jK> I profile all my changes already myself
<hoijui> ok :D
<hoijui> you use an other method?
<zerver> jK, how would you optimize SYNCED.EXPORT?
<jK> I use custom commands with 1M-loops, I use gprofile and I check the fps ^^
<jK> zerver, I wouldn't copy the data at all to c++
<jK> instead I would directly copy it lua->lua
<hoijui> mm ok :D
<zerver> but you cannot, that requires locking
<jK> less worse than copying 1GB per second
<zerver> locking is what we are trying to get rid of here you see, and that 1GB per second copying just shows what I knew already from start
<zerver> which is, that EXPORT works only for sending stuff in conjunction with SendToUnsynced
<jK> you said that it is something around 1GB
<zerver> and you must nil it afterwards
<jK> I didn't tested anything yet
<zerver> kk
<jK> and you will never get rid of all locks
<jK> but it needs to be fast, and I expect a lot of this way
<zerver> maybe not, but enough locks, those locks that matter
<jK> but can't say anything further w/o tests
<jK> copying data between those lua states should be very fast, locking may not hurt there
<Tobi> the lock needs to be taken too though when any lua that might modify SYNCED runs though, unless _G in synced is made some kind of metatable that can do lock/unlock around each assignment
<jK> ?
<zerver> even if you speed it up by a factor 10, it is still too slow
<Tobi> and if it's a proxy table anyway you can also copy only the changes
<jK> the SYNCED table just needs to be synchronized at gameframes & SendToUnsynced
<Tobi> ah right, I was thinking you was talking about SYNCED being proxy that directly reads from synced with locks
<zerver> it would only work if you could identify what has changed and only copy that
<Tobi> that is easy
<jK> hmmm Tobi, that's worth an attempt too
<Tobi> [LCC]jK: well that exactly has the problem that this lock-for-SYNCED needs to be acquired always when synced code runs
<Tobi> unless you are talking about the part after the comma of course :) (_G metatable)
<jK> I meant heaving a SYNCED proxy that reads from synced state
<jK> but as said those things need to be tested
<Tobi> if _G is metatable with __index that proxies to real _G and __newindex that stores in _newG (and performs same wrapping recursively when a table is stored) then it is possible to copy only changes, afaics
<Tobi> will get hairy and also needs to be tested
<jK> yup
<Tobi> also no clue if Lua might have some caveats when _G is a proxy
<jK> possibly, you (/I) need to test 3-5 different ways
<zerver> tobi, do you have any opinoin wrt having the split in luahandle vs luahandlesynced?
<Tobi> I think split must be outside both because 1 handle == 1 state ?
<Tobi> I don't remember the code very well though at the moment
<zerver> i think luahandle is cleaner, because luahandle callins are available in luahandlesynced as well
<jK> possibly it needs a new class, but that's cosmetic
<zerver> so lua_state selection must occur in luahandle callins
<zerver> which is exactly what im doing atm
<Tobi> won't it pollute LuaUI that way, since that inherits from LuaHandle too ?
<zerver> not pollute, it only opens up new possibilities
<zerver> such as threaded LuaUI, not that we need it now, but still
<jK> ???
<jK> a zombie state is not threaded LuaUI
<jK> LuaLanes would be multithreaded LuaUI
<zerver> i meant that it opens up possibility of splitting the unsynced luahandle (LuaUI) as well
<jK> there is no synced LuaUI
<jK> so it can't be splitted
<jK> splitting it means creating zombies
<zerver> but there is sim thread luaui
<jK> no
<zerver> if batching is disabled
<jK> there is an unsynced LuaUI
<zerver> yes
<jK> and only an unsynced LuaUI
<jK> nothing more
<zerver> turn on mode 4 in my branch and you will be enlightened
<Tobi> I'm gonna get some sleep, gn
<zerver> night
<zerver> good that we decided what split to use at least
<zerver> For example CLuaHandle::UnitCreated needs to have a lua_state selection applied, because LuaHandleSynced inherits this function
<jK> CLuaHandleSynced can just create a 2nd CLuaHandle for the unsynced part
<zerver> yes, but you need to access this second handle from LuaHandle
<jK> why?
<zerver> because that is entry point of the execution ?
<jK> no, the entry point is the EventHandler
<jK> and you register there the synced and the unsynced LuaHandle
<zerver> ok, didnt know u could load gadget into CLuaHandle
