<[RoX]Tobi> let's start
=== Agenda ===
<[RoX]Tobi> Welcome
<[RoX]Tobi> Who makes minutes this time?
<[RoX]Tobi> Release plan
<[RoX]Tobi> Desired development direction, two things:
<[RoX]Tobi> Get rid of Lua-able things of engine, or move as much Lua-able things into engine as possible, or don't care?
<[RoX]Tobi> Policy with regards to deprecation of APIs etc.
<[RoX]Tobi> thegeorge crowd path-finder stuff
<[RoX]Tobi> Next meeting
<[RoX]Tobi> Anything else? (WVTTK)
<[RoX]Tobi> so, welcome
=== Release plan===
<[RoX]Tobi> I renamed `progress of stuff...'
<[RoX]Tobi> what I plan is to make 0.82.1 monday evening or tuesday or so
<[RoX]Tobi> preferably with the 2 things I put in roadmap in mantis fixed in it, but don't know if there is time for that ofc
<[RoX]Tobi> that are mantis 2011 and 2004, http://springrts.com/mantis/roadmap_page.php
<[RoX]Tobi> then let that sit for a week or so so ppl can test even more and then put that live
<[RoX]Tobi> any comments/opinions?
<Kloot> so 82.1 will be a second RC?
<[RoX]Tobi> yes
<[RoX]Tobi> and if in that week no big bugs are found it will also become release
<[LCC]jK> yeah a 2nd RC wouldn't be that bad
<zerver> that is good imo, there were plenty of bugs
<zerver> few bugs were reported simply because of few testers
<Kloot> yeah, and those were mostly found by us
<[RoX]Tobi> also, SL devs asked to rebuild installer to include new SL, and source package was broken with 0.82.0, according to hoijui
<[RoX]Tobi> so we need to bump version in any case
<[RoX]Tobi> and also the amount of fixes indeed warrants new build, imo
<zerver> yes
<[RoX]Tobi> plus I prefer to not make release directly from master, but always put it into `RC' mode for at least a few days, and only then put it live on server
<[RoX]Tobi> that can also allow packagers to make packages, if they are fast :p
<[RoX]Tobi> anyway is this ok?
<[LCC]jK> yes
<Kloot> yep
<zerver> Ya
<[RoX]Tobi> ok
Conclusion:
- There will be another RC shortly, as there were plenty of bugs
- Release will be delayed about a week
=== Desired development direction, two things:===
<Kloot> should at least catch critical bugs before the flood of noncritical ones after release goes live
<[RoX]Tobi> I get impression sometimes engine tries to move in multiple directions at the same time, so I have put 2 points on the agenda to bring hopefully a bit more focus
<[RoX]Tobi> the first is
=== Get rid of Lua-able things of engine, or move as much Lua-able things into engine as possible, or don't care? ===
<[RoX]Tobi> sometimes now features are added that could be easily Lua-ed, sometimes there are features that have been present for ages and are easily Lua-ed, etc.
<[RoX]Tobi> does it make sense to have a general policy what we do with those things?
<Kloot> imo yes
<zerver> If it is LUA-able but much faster if implemented engine side, it should be kept in the engine imo
<zerver> Prime example is healthbars
<[LCC]jK> IMO everything that is Lua-able in less code (<10kB) should be done in Lua
<[RoX]Tobi> any numbers on the speed/performance difference?
<zerver> healthbars is very lua-able but slow, particular with MT of course since it has issues with lua rendering
<Kloot> I'd say only engine-side stuff that is heavy on math (movetypes) should not be directly Lua'ed
<Kloot> but general game logic and presentation/UI code AMAP
<[LCC]jK> esp. that teamhighlighting (when ppl lag or dropped) should be really done in LuaUI
<zerver> not until MT is fixed
<[RoX]Tobi> I agree with Kloot yeah
<zerver> :)
<[RoX]Tobi> though I see the problem with MT
<[LCC]jK> <zerver> not until MT is fixed > has nothing to do with it ...
<zerver> just be careful when moving gfx related stuff to lua
<[LCC]jK> it isn't gfx related ...
<zerver> yeah, thats right, it just fiddles with colors
<zerver> so should be fine
<[RoX]Tobi> maintainance wise I'd prefer it though if we could work towards a structure where those engine `widgets' are really a bit like a plugin, and not hardcoded into the main code
<[LCC]jK> most are already CInputRecv
<[RoX]Tobi> they could be that, or inherit from the EventClient class
<[RoX]Tobi> if they need to act on sim events
<Kloot> hmm, but eg healthbars are part of unitdrawer
<Kloot> and would be a very minimal eventclient
<[LCC]jK> same as collision volume drawer
<Kloot> seems you would get spam of very tiny modules then
<[RoX]Tobi> probably yeah, but that's imo easier to grasp than long chains of ad hoc code to handle such things
<[LCC]jK> hmmm in the case of the model renderer it might make sense, because the file is already pretty large
<[RoX]Tobi> as in, makes sense to put it all together in that file?
<[LCC]jK> as in making the large file smaller
<[RoX]Tobi> ok
<[RoX]Tobi> indeed
<[RoX]Tobi> so is there a consensus that in general if something can easily be done in Lua, it should not be in the engine?
<[LCC]jK> +1
<Kloot> +1
<zerver> +0
<[RoX]Tobi> ok, good
<zerver> with the exception of draw call-ins
<[RoX]Tobi> of course that doesn't mean everything should suddenly be moved asap, but the guideline for new things should be that code shouldn't be in engine if it's reasonably easy to L
<zerver> sure
<zerver> i think 'basic' features needed to play should never be moved to LUA even if it can be easily done
<zerver> the engine should provide a basic set of features to enable reasonable play with as little LUA as possible
<zerver> all LUA is currently bad for MT, because it is single threaded and reduces parallelism
<[LCC]jK> teamhighlight isn't reasonable to me
<Kloot> define reasonable
<zerver> that is hard
<Kloot> also the fact that Lua and MT don't mix is a design problem that should not stand in the way of reducing modularity
<zerver> but e.g. a player list is kind of important so moving that to lua would be bad
<[RoX]Tobi> I keep forgetting, what exactly was the problem, that Draw* call-ins run in render thread, and need to acquire lock on sim thread for calls like GetUnit* ?
<zerver> problem is lua code may access different objects
<zerver> some of which require synchronization
<zerver> only one thread can be inside LUA VM...
<[LCC]jK> that's not a problem
<zerver> if there are no draw call ins at all, then there is no problem
<[RoX]Tobi> so maybe the Draw call-ins need to run in a different VM
<[LCC]jK> e.g. you can run other eventclients in the time you wait for the release of the LuaVM lock
<zerver> possibly
<[LCC]jK> you don't need to hardlock a thread just because there is a mutex
<zerver> true
<[LCC]jK> you could even create a eventqueue for the luahandles, and push those events on it, so just one thread (async) runs the lua
<[LCC]jK> (and sync them back in SimFrames)
<[RoX]Tobi> yeah
<[RoX]Tobi> want to discuss this more now or shall we move on to next point?
<zerver> move on
<[RoX]Tobi> ok, next point I have written is:
Conclusion:
- Things doable in LUA, particularly new things, should be done in LUA unless there are specific performance gains from an engine-side implementation
- It is a good idea to discuss it with other devs before deciding to move existing stuff from engine to LUA
=== Policy with regards to deprecation of APIs etc. ===
<[RoX]Tobi> now it's mostly arbitrary
<[RoX]Tobi> sometimes things are removed at once for next release, sometimes stuff is deprecated for6 releases and only removed after that
<[RoX]Tobi> should be make this consistent, or keep it arbitrary?
<[LCC]jK> example for the 2nd?
<zerver> keep the deprecated stuff as long as it works
<[LCC]jK> afaik all stuff got just removed
<zerver> fix deprecation if you feel like doing it, otherwise not
<[RoX]Tobi> in unitsync some things were deprecated for several releases before finally being removed
<Kloot> if something is deprecated but kept around for 6 releases that's probably a sign the change was forgotten
<Kloot> rather than implementations still depending on it
<[RoX]Tobi> yes, that's not the point though
<Kloot> it is, because even with a consistent policy people will inevitably forget
<[LCC]jK> it is, we have releases so rarely that the `maintainer` doesn't want to mark something deprecated and then 8months later he is allowed to remove it ...
<[RoX]Tobi> the length of time something is deprecated isn't really what matters imo, only the fact whether there is a transition period or not
<Kloot> transition periods make more sense when releases are frequent
<Kloot> say less than 3 months apart
<[LCC]jK> I don't want a todo list with lots of dates when I am allowed to remove a deprecated feature, it just slows down my development
<[RoX]Tobi> I don't really mind personally if we decide that anything can be changed arbitrarily at any time without thinking about backward compat though, but it doesn't really make sense to spend time making some things backward compat, when other things are just removed/changed without backward compat
<[RoX]Tobi> ok, so consensus is deprecating stuff is not needed, just remove/change at once?
<zerver> i'd say try to make it backwards compatible unless it complicates the code significantly
<[RoX]Tobi> +1
<[LCC]jK> when stuff is rewritten it makes sense to keep backward compability, but when stuff gets just removed (e.g. those Lua functions) deprecation just slows down the development
<[LCC]jK> yeah +1
<zerver> mod makers just get pissed of if they have to work all the time to keep up with the changes
<[LCC]jK> true, but they were annoyed by the AUC remove, still it was announced (and they didn't said anything then) and they didn't converted modinfo.tdf->modinfo.lua yet
<zerver> :)
<[RoX]Tobi> ok, so that is clear
<Kloot> the AUC change was very lastminute ofc
<[LCC]jK> I knew that why I added that forum thread
<Kloot> but even if it had been made 5 months ago they would still not have
<[RoX]Tobi> yeah that is true
<[LCC]jK> still most of the moddevs who are annoyed didn't fixed their mods for the latest git to that time
<[LCC]jK> yeah
<[RoX]Tobi> why?
<[RoX]Tobi> thegeorge will join for the next point
<Kloot> increasing*
<[RoX]Tobi> yes
Conclusion:
- Fixing deprecated stuff is not a priority, but anyone is of course welcome to do it
- The engine should be kept backwards compatible unless it would bloat the code a lot
=== Thegeorge crowd path-finder stuff ===
<[LCC]jK> hi george
<[RoX]Tobi> thegeorge_irc, possibly you can best explain?
<thegeorge_irc> Hi
<[RoX]Tobi> and also welcome
<thegeorge_irc> Thanks
<thegeorge_irc> I wanted to do some research on crowd based pathfinding algorithms and wanted to test these with the spring engine.
<thegeorge_irc> So I had a look ad the pathfinding and pathfollowing segements of the code.
<[RoX]Tobi> was that after the refactorings by abma already?
<thegeorge_irc> The problem I encountered was, that it is not possible to implement other pathfinding algorithms than the one which is currently implemented without changing large portio
<thegeorge_irc> The pathfinder in spring only creates a path a unit is supposed to follow.
<thegeorge_irc> Then on UnitHandle->update() every Unit is moved in direction to the next waypoint (simple spoken)
<thegeorge_irc> So one could only change the way how the units follows it's waypoints or how the path is created.
<thegeorge_irc> abma's refactoring made it easier to change the pathfinder
<[LCC]jK> how does your ideas differ from the current implementation?
<[LCC]jK> -es
<thegeorge_irc> But to use a different approach of pathfinding as whole, it is necessary to perform pathfinding and unit movements as whole.
<thegeorge_irc> I wanted to implement a pathfinding based on crowd density. The basic Idea is to creade a density map and than scale the velocity of each unit based on the density.
<[LCC]jK> hmmm my idea was to convert the heatmap into a flowmap
<thegeorge_irc> So in areas with high density, the unit moves with the group as in areas with low density the unit moves on its own (say max speed)
<zerver> that sound like a good idea
<[LCC]jK> so instead of holding the heat of a square the flowmap holds the speed vector
<[LCC]jK> so if an unit has the same speed vecotr as the flowmap it just follow the flow
<[LCC]jK> but if it wants to walk against the flow (its speed vector differes from the flowmap) it searches a new path
<thegeorge_irc> The problem I had is that I don't knew where to put the code for the overall calculations which had to be performed
<[LCC]jK> it can easily calculated via via: if ( dot(unit->speed, flowmap[x][y]) < 0.25 ) { %search other way% }else{ %follow the flow% }
<[LCC]jK> -via
<zerver> challenge is to design good behavior where two friendly flows meet
<zerver> so that the unit going against the stream does not get stuck
<zerver> if enemy gets stuck that is fine though :)
<[LCC]jK> yeah, but still better than a density/heatmap where units avoid each others even when they are walking in the same direction
<zerver> indeed
<zerver> this is very interesting research, i might want to participate
<zerver> major brainfuck, but that is the whole point
<Kloot> I happen to be working on a CC implementation for another project, so I'll join in too
<thegeorge_irc> Cool
<zerver> we need kick-ass pathfinding lol
<zerver> just hope that we can make a common goal
<[RoX]Tobi> it sounds interesting but I don't have time, so I will not join
<[LCC]jK> is it worth to try to keep some of the old pathfinder, or is it better to rewrite everything?
<zerver> who made it?
<Kloot> a guy called k-man
<Kloot> back in 2004/2005
<zerver> ok
<Kloot> and it's generally good code
<zerver> well, maybe as first step try to change the interface and then adapt the existing pathfinder for that interface
<[LCC]jK> just a bit complicated and less documented
<Kloot> but for crowd simulations you need another paradigm than A*
<[LCC]jK> o_O
<[LCC]jK> A* always follows the gradient, how does that differ from crowd sim?
<thegeorge_irc> Maybe change the code, so that it is easier to implement a crowd pathfinder.
<thegeorge_irc> crowd does not allway follow the gradient
<Kloot> A* follows the gradient from the perspective of a single agent
<Kloot> at one point in time
<thegeorge_irc> The croed follows the crowd and the gradient
<Kloot> where "gradient" is defined by whatever heuristic you use
<Kloot> ie. euclidean distance
<Kloot> to do crowd simulation you need to reevaluate the gradient at every step
<Kloot> for every agent
<[LCC]jK> isn't it that crowd sim just adds a new map, which's gradient is just taken into account to the heightmap gradient?
<[LCC]jK> yeah
<zerver> we should definitely try to get rid of the A* 8-directional behavior
<thegeorge_irc> There are other pathfinding approaches as crowd based pathfinding and A* which are also interresting.
<Kloot> yep, the underlying graph for A* is just 8-connected
<Kloot> so you always get TTD-style paths
<Kloot> iirc imbaczek wrote some code to post-process the paths to reduce that
<[LCC]jK> hmm I just see 1 other way to prevent this 8-dir connections
<Kloot> anyway people write PhD theses on this subject, so plenty of room for experimentation
<zerver> indeed, this is very advanced
<zerver> and it may be hard to determine whether changes are overall good or bad to the algorithm as a whole
<zerver> there are so many scenarios that have to be tested
<Kloot> true
<[LCC]jK> so any timeline?
<Kloot> I know that reactions to supcom2's crowd pathfinder were very mixed at least
<zerver> ok, would be interesting to know what the issues were
<thegeorge_irc> A small tank can push away a 1000feet giant robot
<thegeorge_irc> in supcom2
<thegeorge_irc> even if both units ar from a different faction
<zerver> rofl
<abma_irc> hi, sorry for beeing so late
<[LCC]jK> so a I would recommend this timeline:
<thegeorge_irc> The pathfinder could also be implemented so that it is replaceable. So the mod chooses which pathfinder it want to use (old a* or new one)
<[LCC]jK> 1. wait for Release and then merge abma's branch?
<[LCC]jK> 2. copy of the old pathfinder and remove any fanzy stuff (heatmapping, 8-dir post-processing, ...)
<[LCC]jK> 3. change it so that paths get updated each simframe instead of 'caching paths'
<[LCC]jK> 4. add new crowd stuff
<[LCC]jK> -?
<thegeorge_irc> The pathfinding stuff is also linkt to the MovementType classes which are called from the unit handler
<thegeorge_irc> I recommend moving the pathfinding and movement stuff into one new interface as these two things has to be changed together for a crowd based pathfinder
<[LCC]jK> ?
<[LCC]jK> the movetypes just tells the pathfinder how much it costs walk on a heightmap square
<[LCC]jK> -s
<Kloot> no
<Kloot> the movedefs do in combination with movemath
<Kloot> the movetypes implement pathfollowing
<Kloot> well just groundmovetype
<abma_irc> rts/Sim/MoveTypes ..?!
<thegeorge_irc> yeah
<[LCC]jK> 90% of the code in those classes is universal
<[LCC]jK> it would need a new interface between those and the pathfinder, and not a merge of those
<thegeorge_irc> Pathfinder and Pathfollower
<[RoX]Tobi> welcome abma_irc
<[RoX]Tobi> I'm eating dinner, maybe we can discuss next meeting later
<[RoX]Tobi> or anyway, I'm available then
<[RoX]Tobi> so you guys can finish the meeting when you are done
<zerver> kk bye
<abma_irc> bye
<thegeorge_irc> bye
<[LCC]jK> I would like a way to separating the pathfinding from the movetypes
<abma_irc> Kloot, thegeorge_irc: as you two want to implement the new pathfinder.. does jk's plan make sense to change / add to a new pathfinder?
<zerver> i think i need to look at the pathfinding in more detail before discussing this
<zerver> so im off to make the minutes
<Kloot> point #3 of jk's timeline will be the largest issue
<Kloot> updating each path every frame
<[LCC]jK> yup
<thegeorge_irc> It must be possible to perform operations regarding all units per simframe and to change the way the units move.
<thegeorge_irc> Thus every unit continues moving along some path, the pathfinder does not neccessary need to be changed
<Kloot> regardless of the exact implementation there are some universal requirements
<Kloot> such as the ability to determine for a given square of terrain whether it is passable for a given type of unit
<[LCC]jK> such stuff can be cached
<[LCC]jK> (the current pathfinder doesn't cache those afaik)
<Kloot> yes, so long as the PF system can query it
<Kloot> but it needs to know about the movetypes
<[LCC]jK> still it needs to be done a more modular way than the current PF
<Kloot> the current also precalculates estimate paths
<abma_irc> hm, any more suggestions to that point?
<Kloot> so a higher degree of decoupling means less possibility for caching
<[LCC]jK> yeah, but it splits the path into separate cached waypoints, IMO it is better to separate the single pathfinding steps
<thegeorge_irc> Hmm, I have to think about this. And I don't know the engine very well and have to look further into it.
<[LCC]jK> e.g. caching/multithreading heightmap gradient/costs map
<Kloot> I will look into getting references to movedefs out of the pathfinder after merging abma's branch
<abma_irc> hm.. in short: do step 1+2 of jk's suggestions after release, step 3+4 seems to need more discussions / next meeting!?
<Kloot> probably :)
<abma_irc> next point on the agenda? (should be release plan)
<Kloot> heh, actually next point is "next meeting"
<abma_irc> ok, i didn't read something about the releaseplan: then next meeting
Conclusion:
- Abma has done interesting work on the pathfinder side, which will be merged after release
- Thegeorge is working on a new interface to enable writing more advanced pathfinders
- Other devs are also interested to join in on the work, as the current pathfinder is far from perfect
=== Next meeting ===
<Kloot> I'm +1 on same time next sunday as usual
<abma_irc> me too
<[LCC]jK> yup
<Kloot> zerver?
<Kloot> hmm we'll hear from him and tobi later
=== Anything else? (WVTTK) ===
