Present: <abma>, <hoijui>, <jk>, <beherith>, <zerver>

<beherith> hi
<abma> hi!
<abma> hopefully jk & zerver will join soon
<beherith> okies
<hoijui> hey
<hoijui> nothing startedyet
<abma> yeah, zerver is needed
<abma> most / close to all topics are related to spring-mt
<abma> [LCC]jK: anything to say about LuaJIT without zerver present?



=== LuaJIT ===
<jk> semi bad news
<jk> I got it running in a not syncing way
<jk> but the performance increase is behind expectations
<beherith> does lua present such a large load in sim?
<jk> at start of game where LuaUI:DrawWold dominates the usage of it goes 50% -> 40%
<jk> duno how it performs in end games
<jk> for that I am currently trying to setup a benchmark suite
<abma> hmm, i think the performance increase isn't so high, because of so many lua callin/outs, but that is only a guess. so, better test with a benchmark... :)
<jk> btw anyone knows how a startscript.txt has to look like when I want to start a demo (I want to use a startscript to start it cause I can set mingamespeed there)
<jk> it's either that or Lua just really spends a lot time in c++
<jk> FFI might help in that case (FFI = extension in LuaJIT, so lua can natively call c++ functions -> JIT'ed)
<beherith> you can have a widget to set mingamespeed when you use a demo (I did something similar when recording XP stats off lots of replays
<jk> still oprofile never shown a GL function in Lua as very time intensive
<jk> I work with ZK currently I would additionaly have to setup a config then ...
<jk> (to make user widgets work)
<abma> [LCC]jK: see the buildslave
<abma> https://github.com/spring/spring/blob/d ... K_data.lua
<abma> or https://github.com/spring/spring/blob/d ... Format.txt
<abma> DemoFile=
<jk> hmmm when I think about it I really have to give it the old + new specs as input
<jk> very dumb btw
<beherith> Could I please present the LOS thing quickly, as I cant stay too long, but I dont know if it interests zerver
<jk> -> with setting DemoFIle, you still _must_ set GameType MapName, modoptions, PLAYERS (the one IN the demo), ...
<hoijui> squeeze in Behes topic now?
<hoijui> me ok
<hoijui> can continue with this later?
<jk> sure

Conclusion: LuaJIT works, but the performance boost is not as high as expected yet, may need more profiling/tweaking



=== Behe's LOS shader ===
<beherith> So you have probably seen the forum post, and my real question is, is there anything fundamentally flawed in my idea, or which version (Lua shader or SSMF plugin) would you prefer?
<jk> Pre-Processing FX is the way to go
<beherith> Also, I would like to ask jK if it is possible (if I where to go the SSMF route) to swap out shaders (or recompile) so as not to waste texture lookups when not running in LOS mode
<beherith> I know its possible to swap/recompile, but is it too slow?
<jk> else ghost, features drawn in airlos will be drawn as normal (w/o noise) and the ground below it with it
<beherith> ok, im sold on the post processing - its a bit slower than using the ssmf shader - but infinitely better looking and more customizable
<jk> erm Post
<beherith> hmm?
<beherith> oh
<jk> <jk> Pre-Processing FX is the way to go <jk> erm *Post
<beherith> is the concept of using the core of the fog shader and the depth texture to render a solid method, or do you know of any better ways of doing it?
<jk> it's really time Spring renders depth into a FBO with bound textures (not RBOs), so you don't have to copytotexture each time you want to access depth information in a shader
<beherith> also, is a texture lookup of a noise texture (with nearest neighbour) more expensive than using a glsl function to compute noise? Or is this too hardware dependant?
<jk> hw dependant
<beherith> Should I try to implement an FBO? Could it then be possible to do multi render target postprocessing?
<beherith> I know most mods couldnt do it, because they draw so much stuff in different parts of lua
<beherith> but I would gladly forego most of those effects for bloom+deferred lighting
<jk> you mean implement FBO in engine?
<beherith> yeah
<jk> you can, if you know how
<beherith> Ok :D
<beherith> thanks for the info, and for listening. Also, the changes required to do the $infotex binding and forcing the generation of LOS texture are minimal, may I submit a patch?
<jk> regarding that ... I have your request on the todo list for a very long time
<beherith> dont worry - i have it done in my branch
<jk> but wanted to unify gl.Texture with other texture functions in Lua
<beherith> ah ok
<jk> e.g. UnitMaterials (unit shaders)
<jk> but it showed to be a much bigger problem than I thought
<beherith> that would be sweet, just shoehorned trim color (secondary team color) into your framework
<beherith> thanks again for everything!
<beherith> I doubt I could be of much further use for the rest of the meeting, but I'll lurk if its ok

Conclusion: There is some nice WIP for LOS visualisation



=== LuaJIT/benchmark ===
<jk> where should buildslaves upload their image results (graphs) to?
<abma> jK: same directory, where the win32/macosx/... installer is uploaded?
<abma> 7 subdirectory of it
<abma> / subdirectory of it
<abma> or keep it on the same host as the buildslave, if possible
<abma> but i think an upload would be nicer
<jk> if it is on same host, it couldn't be accessed from buildbot website
<abma> yep
<abma> only a proxy would be possible then
<abma> or redirect
<jk> <_< gamespeed seems to be saved in demo
<jk> so no need for a startscript, when I recorded the demo already at high speed



=== Remove lua DualStates? ===
<abma> hi zerver! :)
<zerver> hi
<abma> still it seems nobody uses it
<abma> also it makes code much more complicated
<abma> jk can say more to that
<jk> LuaJIT needs to be dynamically linked, so code changes to it are not possible
<jk> esp. the lua_state changes were a problems
<hoijui> me bed
<abma> zerver: ping
<jk> but even when I semi workaround it (https://github.com/spring/spring/commit ... ab193af9c3) I still get problems from time to time in the GetXYZ(lua_State *L) functions
<jk> e.g. lua threads/coroutines have their own lua_State*, but aren't created by Spring instead they are by lua itself
<jk> and so they won't show up in std::map<const lua_State*, SLuaContextData> luaContextData;
<jk> causing the problems in GetXYZ(lua_State *L) functions
<jk> and this is not trivial to fix
<jk> so I would recommend a removal of the DualStates esp. cause they aren't used
<zerver> dual states is the only thing that unleashes full MT performance, so I do not agree
<jk> it doesn't when bad written and not used
<zerver> and I don't think almost any game dev is aware that the dual states exist, so "not used" is not exactly an argument
<jk> they do, but it's just to complicated to use them
<zerver> programmers were slow to adopt threads, so we remove that feature from the compiler... not exactly
<abma> not used makes it possibly to remove it and nobody will have a problem / complain
<jk> and not used is an argument, when it prevent further development on the code
<jk> +s
<zerver> threading is complicated in case you didn't know
<jk> no one wants to rewrite all the lua, neither are DualStates needed
<zerver> dual states is needed as much as luajit...
<jk> LuaUI (takes the major part of luatime) can already run 100% in renderthread (if your batches code is working correctly)
<jk> no, luajit if it works (duno yet) would give everyone a performance increase w/o any extra work from the gamedevs/luadevs
<abma> zerver: i know its complicated. but as it currently is, imo it isn't really helpful. its broken every release :-/
<abma> also current engine performance seems to be ok
<abma> it can be improved, yeah, for sure
<zerver> not sure what to say, only that it is usually not me who breaks MT but someone else
<jk> if your mutex system would be more obvious ppl could add them themselves
<jk> but you have like 7 different macros & many different input argument names
<zerver> and i did not understand why luajit needs dynamic linking... plz enlighten me
<jk> its dev doesn't want it to be embedded and makes it extra complicated when you try doing so
<zerver> so it can be statically linked, and we should do that imo
<zerver> the only thing I am aware of that always needs a dll in windows is a hook
<jk> static linking = compiling .a/.lib into the exe (different topic)
<jk> dynamic/shared linking = needs .so/.dll
<jk> still none of those lets you to modify the code
<zerver> because of this dev's restrictions?
<jk> no cause the .a/.so are system libs
<jk> to compile & edit them yourself, you would need to setup a complete new framework in the buildprocess
<jk> we neither compile devil, freetype, ... ourself
<zerver> okay
<zerver> still it can be done
<zerver> and how can game devs know/understand dual states, when I wrote the docs like 2 weeks ago?
<jk> discussion between you and me in meetings/forum?
<zerver> sure, maybe that is better
<zerver> anyway if we can disallow/disable all gadget based rendering, then I can possibly support the idea of removing dual states, but not otherwise
<jk> gadget rendering takes like 5% of the cputime
<zerver> e.g., so that gadgets are invoked by sim only
<jk> it that blocks your rendering thread something is really wrong
<jk> *if
<zerver> shared resources do that, it is their nature to block
<jk> 5% are 5% it's like reducing the GHz of your core by 5% -> not noticeable
<jk> so if the drop is bigger it means the thread _yields_/sleeps
<zerver> you cant say it is 5% when it is a dynamic phenomenon that depends on the actual code that the game dev writes
<zerver> and the actual game being played, how high the FPS is etc
<jk> I know the codes
<jk> and I profiled that stuff a lot
<zerver> i am sure we can solve the luajit issue in a good way, and we don't even need to fight about it
<zerver> I can write a 100% cpu rendering gadget for you
<zerver> lol
<abma> hmm, continue that in a dedicated thread?
<abma> it looks like it could take much longer and maybe tobi/kloot have some thoughts, too
<jk> I could write a crash widget and?
<abma> zerver: ping
<zerver> plz be serious
<zerver> if rendering widgets were a problem (they were) then gadgets are too. there is no difference
<jk> there are less gadgets & they are less GUI heavy ones
<zerver> yeah, luckily most ppl do write a widget instead of a gadget for that...
<abma> can we continue that topic in the forum?
<zerver> I guess so
<abma> i only want the savegame topic now... the other(s) can be done later

Conclusion: No consensus, but there is also no point in discussing this before LuaJIT works without desyncs



=== Can engine savegame support be removed? ===
<abma> it seems like it doesn't work... also we have lua gadgets that can save / load a game
<zerver> engine save does not work, correct
<abma> the question came, because of some bug about it
<jk> but it's still needed even with lua save/lod
<jk> *load
<abma> is it needed as a whole or or just some parts?
<jk> also to fix it it CREG just needs to be modified to print warning if CREG data structure size != C++ class size
<jk> +a
<zerver> jK, I think the CObject::Serialize FIXME you put in was the last nail in the coffin for the engine save
<jk> that's minor
<zerver> okay, not part of load/save?
<jk> problem are not serialized class members
<jk> death dependence isn't such important to break loading
<jk> +as
<zerver> i'd expect it to crash if death deps go, but maybe i am wrong
<jk> might be, still that's a runtime issue which can be fixed with a buildbot validation script
<jk> problem is that currently saving/loading breaks with every change to a c++ class
<jk> CREG really needs to print a warning if someone forgot to update the CREG structure after such a change
<zerver> well, creg, just like MT, is inherently complex and it is hard to change that
<zerver> try to make the code that gives this warning and you will see
<jk> sure there is some very crazy stuff in CREG
<jk> but most of the work is just to add CR_MEMBER()s
<zerver> yeah, easy to forget, just like the situation with MT/mutexes
<zerver> need to rewrite MT so that it warns "you forgot to add a mutex on line 84 in Game.cpp"
<jk> in case of mutexes it is more complicated
<zerver> indeed
<jk> still we very often know that a gml mutex is needed, but we don't know which name it must have neither what macro should be used
<zerver> I'll try to make some docs about it
<jk> thx
<abma> good night!

Conclusion: Savegame support cannot be removed at this time

By the way jK, regarding my small mod of Lua, did you try to mod LuaJIT in the same way before compiling it?

lj_state.c line 262 and lj_obj.h line 567.

It should be much easier and sufficient for all your testing needs. I hope there is a way to work around that desync issue.

Thank you jk for biting those constants from my n³ lua :D

Removal of save does not make me happy at all. I'd much prefer if we had something that bundled the entire lua state and all components, so the only thing lua coders need do is refresh invalidated objects when some trigger event is launched. Meh CREG/C++

LuaJIT will never sync. You would need to rewrite it to make it so.
And the current `desync` just comes from the fact that I added an option to switch the inbuilt synced Lua with the system installed LuaJIT shared library.
If LuaJIT is really faster and worth the effort, it would need a damn lot of work. LuaJIT is incompatible with Spring's FPU settings, so all LuaJIT code would need to be `wrapped`, and I duno how that conflicts with threading (FPU settings might be per-process and not per-thread). Then many files need to get compiled twice (once for synced Lua and once for LuaJIT).

Not too keen on losing the ability to draw in gadgets... why is this needed zerver?

Gadget rendering has a negative performance impact in MT spring unless you rewrite the gadget according to this:
http://springrts.com/wiki/Spring_Multit ... ing_Models (mode 3 or higher)

It was suggested that all game devs (except for me) are too lazy to ever rewrite gadgets, and for this reason these Lua Threading Models should be removed.

I don't like the idea of having a problem and removing the only known workaround. Therefore I suggested to remove both the source of the problem and the workaround.

So it is not something that is needed per se, it was more like the end result of a discussion about cleaning out some complex code parts.

write a gadget that rewrites the gadgets?

Most "game devs" just use what spring has.

the rest of us would have to refactor all of our code. That is a bit much to ask. it isn't about laziness it is about the fact that we would lose months if not a year or more refactoring code rather than add features. The threaded code may be even harder to work with and create NEW bugs. For something that is an experimental branch in the engine.

I am not sure who was calling who lazy, but those of us that have adopted lua to build functionality in our projects are far from lazy.

The introduction of a scripting language into Spring, or any software actually, is not to improve performance, but to improve ease of customability.

It is so people who are not hardcore coders can still implement neat ideas.

So I don't think making Lua gadgets suddenly super hard to write to gain a little performance is the correct course of action.

I still find JKs implementiation of a jit-compiler a worthy endavour.


What also could be help, if spring made had a optimizerinfolog, that told about bad coding practices. Not used all locals? Functions or large tables not localized before going into function? Spring OptiLog becomes your pittiless master. As a dev, this might one day become a dreaded question.

SPRING is soooo slow.
Post Optilog.

Biggest Problem i think is that every mod approaches spring with a diffrent mindset, thus leading to much weight on the frontwheels, while the backwheels go free.

So in this case. one unusual matter might do more then a thousand optimisations. Scholar the devs. Hold lua-performance classes on video. I read that good CA-Coding Practice Site, but i still use table.insert from time to time, and i have no clue why that one is forbidden.

>-ou cant spoon out what the aqueduct is pumping in.

All ur code... well that depends. Adapting a game for mode 3 or 4 is typically pretty minor work. It depends on how many rendering related gadgets the game has of course, BA has almost no such gadgets and can be converted to mode 4 in a few minutes. Mode 5 and 6 is _much_ more work, and also harder to work with.

so you are saying that mode 2-4 is minor. Does this work fine with regular spring? Is is highly sensitive to the existence of the experimental branch(MT), aka if you scrap MT, will my code suddenly break?

If if is a matter of simple alteration of coding practices I can accommodate you.

Mode 2 is the default mode, you are probably using it already.

All suggested changes for modes 3-6 (with exception of the EXPORT table of mode 3, which I don't recommend to use anyway) use existing API and are therefore compatible with ST spring regardless of whether MT is scrapped in the future or not.

MT is part of the main branch, so not experimental (although the last few releases have indeed been broken for various reasons).

For CREG:

Some kind of static verification (at build slave) may also work. E.g., have CREG dump existing class members for each known class and check that against list of class members extracted from the corresponding header file. (Same technique could even be used to generate the CREG definitions, though that makes the build system more complicated.)

But indeed, a simple size check might get us a long way there, and is much simpler to implement.