Dev meeting minutes 2010-05-16

Dev meeting minutes 2010-05-16

Minutes of the meetings between Spring developers are archived here.
Post Reply
Tobi
Spring Developer
Posts: 4598
Joined: 01 Jun 2005, 11:36

Dev meeting minutes 2010-05-16

Post by Tobi » 17 May 2010, 20:24

Date: 16-05-2010
Present: hoijui, jK, Kloot, Tobi
Not present: zerver
Next meeting: same time same place next week

Agenda
  • Welcome
  • Review meeting minutes past meeting
  • Review 'how shall we keep track of who's doing what? Just meeting?'
  • Progress of stuff to be done before release
  • New buildbot (compile from other repos?)
  • Current buildbot installer problem
  • New infolog upload site
  • Make changes to the lobby-server/-protocol a smaller problem
  • Next meeting
  • Anything else? (WVTTK)
  • End
Review meeting minutes past meeting

Tobi: any comments on it?
Tobi: I myself found it pretty unreadable ;-)
hoijui_g5: so should we write more clearly in here? :/
Kloot: yeah that's a general problem of irc imo ;p
Kloot: interleaving threads of conversation and crosstalk :]
Tobi: yeah with face to face meeting you do some filtering already while listening
Tobi: so assuming I'll make them again I think it will work better if I remove more and keep only main points in form of bullet lists

Main point:
  • Tobi will try to make minutes more readable
Review 'how shall we keep track of who's doing what? Just meeting?'

Tobi: the etherpad solution we decided upon previous week is only online for less than a half week, but any comments on it already?
Kloot: think it depends on how well everyone keeps their part updated
hoijui: i wanted to know.. is it ok to have my part so long/detailed?
hoijui: or should i shorten it drastically?
jK: yeah wanted to say that
Tobi: maybe it would work better if you put short bullet list in the main part, and move the details to the bottom?
Tobi: so the main list of all devs together ideally fits on one (big) screen
jK: yeah perhaps add a footer with more details
hoijui: will do that then

sidetrack:
hoijui: i though about importing the gobby document to etherpad, but there seems to be no way to keep the text->user relation, and as half of it is chat..
Tobi: correct
Tobi: maybe we should look at it there at some point, it's the one BrainDamage was poking people for, right?
hoijui: yep that one

Main point:
  • etherpad solution of next week is online for half a week
  • hoijui asks whether it's ok that his part is long/detailed
  • it's decided that it's better to keep all main parts together fitting on approx. one screen and put details after that (to keep easy overview)
Progress of stuff to be done before release

Tobi: jK: plan remains the same?
jK: plan is the same, but it delayed a bit (run into a lot bugs this week)
Tobi: ok
Kloot: maybe better to postpone the new shader classes until after the next release considering how much stuff has already been added / changed (in rendering specifically)
Tobi: yes, but then I assume many of those new bugs would still need a fix
jK: are there any shader bugs in the current master? (the bugs I run into were other nature)
Kloot: there is one: S3O lighting is too dark on certain maps (ex. spawn a core_commander on smalldivide)
jK: you can use my 1:1 glsl copy of the arb shader
Kloot: will take a look at that yeah
jK: also found out that it improves the image quality a lot if you move the lighting computation from the vertex shader to the pixel shader (dot(normal,sun) * sunDiffuse+sunAmbient)
Kloot: all lighting is per fragment in mine (they're not 1:1 translations of the arb code)
jK: in the arb it isn't and because I made a 1:1 copy, I wondered why normalmapping made the unit brighter even with a full up_vector normalmap
jK: I would even call it a bug in the arb shaders, after I saw the huge difference, it's like day and night
Tobi: either way suppose it'll just be fixing bugs for a while before RC, independent of how many weeks that takes exactly
jK: k, I will just wait for the commit until (nearly) everything is finished in the commit. I can also make a branch, but git fails for me some how at such stages.
Tobi: jK you can also make multiple commits and use git pull --rebase to re-apply them to latest changes from master
jK: I mean I won't commit anything unfinished till the RC (if at all then it has to be in a workable/RC state)

Tobi: btw in mantis you can set target version of bug to 0.82, that way the roadmap page will work and show list of bugs that still need fixing before release

main points:
  • plan from previous meeting delayed a bit due to lots of bugs
  • image quality improves a lot if you move lighting computation from vertex shader to pixel shader
  • Kloot will look at jK's 1:1 GLSL copy of the ARB shader
  • in mantis you can set target version of bug to 0.82 for roadmap page
New buildbot (compile from other repos?)
(hoijui put this point on the agenda)

hoijui: is it feasable, and do we need it?
hoijui: i mean.. as it is now (with bibims buildbot) we can only compile spring from the main repo on github
hoijui: of course i can just put all my branches there that i want to test compile
hoijui: so the reson to allow to build from other repos, would be to keep the main repo more clean
hoijui: it is not really an important thing.. and would need access rights or something, i guess
Tobi: feasible - I don't know, I don't recall buildbot easily allows this (but then again I didn't do much with it's git stuff yet)
Tobi: allowing *any* repo is definitely no-go, that's like giving entire world shell access on server, so if anything it would be a whitelist
hoijui: yeah well.. let us discuss that when it is really in use
Tobi: yeah, I should find some way to motivate myself to finish it sooner :)
hoijui: so the point was not jsut the compile form other repos, but the buildbot in general
Tobi: what about it specifically?
hoijui: we just keep this in the list for next week?
Tobi: ok

main points:
  • Compiling from other repositories? => At best whitelist.
  • Discuss it when it's really in use => Tobi should continue on this
  • Item will be recurring next week
Current buildbot installer problem
(hoijui put this point on the agenda)

jK: will a new buildbot fix the installer generation?
hoijui: maybe, i would guess so
hoijui: but as that bug is really.. i dont know how it works, i cant guarantee that. but on the tests with bibim, it did not happen
Tobi: what was the bug with it?
hoijui: base files are not found
hoijui: and.. even though investigating with bibim, and trying it locally, i was not able to find out why
hoijui: locally it worked and soemtimes, it also works on hte buildbot but we do not know when
Tobi: strange
hoijui: it seems to put the files into a wrong dir, but not always, and .. yeah

main points
  • heisenbug with installer generation in current buildbot..
New infolog upload site
(hoijui put this point on the agenda)

jK: was it updated so you can see the stacktraces w/o downloading a zip?
hoijui: yeah
Tobi: http://infologs.springrts.com/list
hoijui: ahh.. no :D
Tobi: visualisation is still a bit bad :)
hoijui: Zydox was workign on it.. i will ask him when i see him, next time
jK: it got updated but you still need to download a zip, no?
hoijui: yeah you do, i was wrong, bad memory
hoijui: it will also need ot translate backtraces for windows crashes
Tobi: hoijui: was there anything about it in particular that you wanted to discuss?
hoijui: ..no :D
hoijui: i jsut have it there cuase..
hoijui: i dont want it to be forgotten

jK: as long as you can't see the stacktraces on the page itself, it's unusable to me
Tobi: personally I remain of same opinion, that it'll be mostly useful for statistics, and not so much for actual debugging (i.e. you can see that some bugs are very 'popular', and then you can focus on those instead of one that occurred only once),
hoijui_g5: yeah... for both things, it needs more/better visualization
Tobi: but without a good way to reproduce all the data doesn't help really much and that will remain a thing the user himself needs to spend effort on (which he wont, apart from some exceptions...)
hoijui: maybe... if one bug is appearing often, we will also get forum posts. or mantis reports, likely
Tobi: either way it's good for drawing attention to popular bugs ;-)
Kloot: ya, it's rare that you can fix a bug just by looking at the trace long enough
hoijui_g5: and have to ask for reproduction instructions there
jK: yeah, the zip should contain the replay and not the infolog
Tobi: why not both?
hoijui: yeah both, but most of the infolog should already be visible on the page
jK: or both (infolog.txt might be useful to know the running widgets and other runtime issues)
hoijui: and there should be a way to create custom filters or something
Tobi: you can, probably, by modifying the source :)
hoijui: but... i have little mood to learn the web toolkit
Kloot: that might be easier if we standardize infolog output and segment it better
hoijui: yeah.. true
Tobi: yeah that would help too
hoijui: though.. the parsing of infolog is mainly done, as i understood, eg.. they have it all separated
Tobi: although widgets/gadgets could still dump whatever they want in it
hoijui: at least i was told so (for Linux and Windows)
hoijui: so it is really mainly the GUI/webinterface stuff that is missing
Tobi: yes
hoijui: will have to talk to koshi and zydox about that

main points:
  • jK: as long as you can't see stacktrace without downloading zip it's unusable for me
  • don't forget it!
  • will be mostly useful for statistics, it's rare you can fix a bug by looking at the trace long enough
  • replay should be included
  • hoijui will talk to koshi and zydox about it
Make changes to the lobby-server/-protocol a smaller problem

Tobi: not really Spring engine thing, but anyway
hoijui: i discussed that with BD yesterday
Tobi: at least the enforced single engine version is kinda frustrating from release manager POV
hoijui: right. forgot about that, will add that to the list
hoijui: other things:
hoijui: we could add property sets
Kloot: only way you can make changes a smaller problem imo is by introducing custom protocol params
hoijui: per channel, battle, user
hoijui: yeah... that is kind of.. something like that
hoijui: an other idea was, making commands modular, and pluginable
Kloot: or by defining protocol extensions
hoijui: i did not like that so much
hoijui_g5: extensions = new commands?
Tobi: nah, there are more solutions then just that IMO, like very stable protocol on start to handshake protocol version and/or a mechanism for client/server to advertise / subscribe to protocol extensions
Kloot: something like that, think fixed collections of extra commands which lobbies can declare support for
hoijui: the thing is.. if ther is an extension.. that still menas a new server version, and possible failures..
hoijui: and.. in practise, this means, it will never see the light of day as it is now
hoijui: an other idea would be, to have a test/backup server, which auto updates to latest master of TASServer or so
Tobi: most pressing issue atm though is lack of *active* server maintainer
hoijui: BD has commit access there already
Tobi: as in not maintain the code, but full rights to do deployments as he see fit too
hoijui: and nobody wants to do that, cause he wil only be shouted at when it fails ;-)
hoijui: so we need a way to be pretty sure the new version works fine
Kloot: and he would have to deploy on the main server because nobody uses the backup one :)
Tobi: yes and yes :)
Tobi: if even anyone uses backup server 99% chance he doesn't report any bugs he can find unless they prevent playing completely and also it will never be stress tested as much as normal server (unless normal server is down :))
hoijui: yeah.. i do not think that performance is really an issue so much
hoijui: but if we have a test server, where i and lobby devs can easily run patches on, then we would surely use that server as well
Tobi: atm not, but this was after getting bigger server and hot-optimising some code on the production server (quite long ago already), so there could definitely be fatal regressions wrt performance, if that isn't tested
hoijui: :D i am pretty sure it wont happen to me
Tobi: I suppose I can make a test server account on spring server and give some peeps access
hoijui: maybe we can also ask koshi & BD as they will be using it often
Tobi: sure
hoijui: i will do that
Tobi: also should I poke aegis a bit about uberserver, maybe even with deadline (1 month from now or so?), so the way is opened to assigning new *active* server maintainer?
Kloot: tbh I doubt he'll suddenly become more active
Tobi: or just smoothly assign it to someone else who's active with that test server account?
Tobi: Kloot: idea is mostly to make the situation clear, independent of the outcome
hoijui: is ok for me
Tobi: now it's 'yes I'm active and there's just one bug left'-state for over a year or so and that blocks a bit work of others on server
Kloot: just make it clear that the monopoly on server dev work is lifted (if there ever was)
Tobi: with this test account?
Tobi: well anyone can always do development of course but hard thing with server is that there's only 1 of it running so there's no nice competition as with lobbies etc.
Kloot: yeah true, but you can always hand out a few more testaccounts to interested reliable people
Tobi: anyway I think it's clear enough for now - will make test server account and hoijui & I will give some peeps who are trusted and want it access to it
hoijui: ook :-)
Tobi: if something good is created we can see about deployment at a later meeting
Kloot: exactly
hoijui: koshi is also setting up a test TASServer right now

main points:
  • enforced single version is frustrating - hoijui adds to list
  • protocol extensions: custom property sets / advertising/subscribing to fixed collections of protocol commands (possibly defined by server plugins)
  • biggest issue: lack of active server maintainer (as in maintaining the production copy)
  • issue with server is that there's only 'binary' competition, either your version of server is used or it isn't, but there's no in-between
  • set up test account on springrts box, give trusted people access
  • discuss deployment later if/when something deployable is created
  • poke aegis (to get active server maintainer again, independent of who this actually is)
  • koshi's also setting up a test TASServer
Anything else? (WVTTK)

Tobi: 'WVTTK' is maybe known to Kloot? it means 'Wat Verder Ten Tafel Komt' [Dutch], i.e. free translation: anything else to discuss?

Mantis 1919

Kloot: if you have some time after this meeting I'd like to talk about mantis 1919 :)
Tobi: ah COB desync
Kloot: yeah actually not COB
Kloot: so far earliest point of divergence is CLuaUnitScript::Turn
Tobi: ah you got deeper into it already than mentioned in the report?
Kloot: yeah, it seems like either Lua stack for becomes corrupt at some point (need to dump it) or there is a bug in the coroutine scheduler
Tobi: first could very well be if there's some memory corruption going on
Tobi: bug in scheduler, hmm, I suppose that's technically possible too
Kloot: the thing is that the desynced variables show very consistent values across multiple runs
Tobi: every time the arguments to ::Turn?
Kloot: yep, specifically destination is always either 0.1 or -0.1
Tobi: happen to know for which unit it was called?
Kloot: yeah, a corroach (CA unit)
Tobi: hmm corroach.lua doesn't even use Signal/SetSignalMask, nor do I see anywhere 0.1 or -0.1 destination
Kloot: no it doesn't
Kloot: there's no leading 0, search for .1 / -.1
Kloot: it seems that in some cases the Turn()'s before the Sleep call get executed and in others the Turn()'s after
Kloot: hence why I thought scheduling issue, but that doesn't seem to be the cause after debugging unit_script.lua
Tobi: that would be very strange
Tobi: I don't have a clue yet, would have to dive into it too I fear :)
Kloot: in any case whatever gets popped off the lua stack in CLuaUnitScript::Turn belongs to one of those Turns
Kloot: yeah ;P
Kloot: s/popped/read*
Tobi: I've put it on my todo
Kloot: I'll look at it some more too, but the demo is pretty slow to /skip to the desync point
Kloot: frame 32780 to be precise
Tobi: I assume then that it doesn't desync often, not like any time you put a few roaches in game it desyncs
Kloot: I couldn't reproduce it in a standalone testgame no, but the demo itself so far always diverges between two subsequent runs
[17:51:13] <[RoX]Tobi> I'll try to try it next week or so :)

Hoijui's status update & events

hoijui: i did some work on pureint this week and abstrRes
hoijui: i also wondered... spring hardly uses events. is there a reason for that? (eg, performance, buggy, ..? )
Tobi: like the event handler stuff?
hoijui: yeah but.. the event handler.. is a really ugly thing
hoijui: its .. one class that contains all the events and there is nothing else i know of
Tobi: what kind of approach did you have in mind then?
hoijui: in abstrRes, for example, when a new resource is added, i want to inform a lot of other classes
hoijui: for example: resource Mana is added -> Team has to initialize values for resource Mana to 0
Tobi: apart from the seeming bloat, what is wrong with making it another event in eventhandler class?
Tobi: ('seeming' because I think it isn't that bad considering that it doesn't drag in everything as dependencies - classes can be forward-decl'ed etc.)
hoijui: that means, every class that has interest in the ResourceAdded event, needs to implement event handler class
Tobi: though for dynamic stuff like Team I don't know..
Kloot: why do you need an event to register a resource?
Kloot: new resource types added on the fly?
hoijui: cause they might be added by the engine, some resources.lua file in the mod, or gadgets later on, during the game (that would be in the far future)
hoijui: or more simply siad: i do not know the exact way of initialization of all things
hoijui: and doing it wiht events, i do not have to know it/it can change
hoijui: as said, it is not about runtime adding only
hoijui: with events, i just do: AddListener(this), and i do not have to wory abotu initialization order
hoijui: i have never seen a design where all events of an application were in one class
Tobi: shouldn't be all IMO - just all game events (though I suppose save/load are a bit on the edge in this case)
Tobi: but better structure with multiple classes would be fine with me too
jK: I think abstRes has much bigger problems than runtime adding new restypes :x
Tobi: what kind of problems?
jK: I just throw a glance at it, but I remember a lot (esp. lua related) things aren't implemented yet
hoijui: ook
Tobi: oh I thought you meant major fundamental problems, that sounds like just some missing implementation here and there
0 x

User avatar
koshi
Lobby Developer
Posts: 1058
Joined: 14 Aug 2007, 16:15

Re: Dev meeting minutes 2010-05-16

Post by koshi » 17 May 2010, 20:34

wrt building from git repos: I see no good way of having one builder perform builds from different repos. Adding a builder each for a list of repos is trivial otoh.
0 x

Tobi
Spring Developer
Posts: 4598
Joined: 01 Jun 2005, 11:36

Re: Dev meeting minutes 2010-05-16

Post by Tobi » 17 May 2010, 20:40

Ah true yeah. I suppose then there's really nothing against adding some repos in which active development happens if there's demand for that.
0 x

User avatar
koshi
Lobby Developer
Posts: 1058
Joined: 14 Aug 2007, 16:15

Re: Dev meeting minutes 2010-05-16

Post by koshi » 17 May 2010, 21:09

Tobi wrote:heisenbug with installer generation in current buildbot..
Haven't had any issue in the couple dozen test runs I've done on my buildbot
0 x

User avatar
bibim
Lobby Developer
Posts: 901
Joined: 06 Dec 2007, 11:12

Re: Dev meeting minutes 2010-05-16

Post by bibim » 18 May 2010, 00:09

Tobi wrote:Make changes to the lobby-server/-protocol a smaller problem

hoijui: we could add property sets
Kloot: only way you can make changes a smaller problem imo is by introducing custom protocol params
hoijui: per channel, battle, user
hoijui: yeah... that is kind of.. something like that
hoijui: an other idea was, making commands modular, and pluginable
Kloot: or by defining protocol extensions
hoijui: i did not like that so much
hoijui_g5: extensions = new commands?
Tobi: nah, there are more solutions then just that IMO, like very stable protocol on start to handshake protocol version and/or a mechanism for client/server to advertise / subscribe to protocol extensions
Kloot: something like that, think fixed collections of extra commands which lobbies can declare support for
hoijui: the thing is.. if ther is an extension.. that still menas a new server version, and possible failures..
hoijui: and.. in practise, this means, it will never see the light of day as it is now

main points:
  • protocol extensions: custom property sets / advertising/subscribing to fixed collections of protocol commands (possibly defined by server plugins)
For information, the last patches I made for TASServer (currently in production) brough a way for lobby clients to declare the list of optional protocol functionalities they support, during login phase (refer to compFlags parameter of LOGIN command)
0 x

User avatar
aegis
Posts: 2456
Joined: 11 Jul 2007, 17:47

Re: Dev meeting minutes 2010-05-16

Post by aegis » 18 May 2010, 00:21

bibim wrote:compFlags
would be better to have a bidirectional system.
0 x

User avatar
bibim
Lobby Developer
Posts: 901
Joined: 06 Dec 2007, 11:12

Re: Dev meeting minutes 2010-05-16

Post by bibim » 18 May 2010, 00:39

aegis wrote:
bibim wrote:compFlags
would be better to have a bidirectional system.
Well, I guess it could be of some use later indeed if some very specific extensions not commonly implemented by servers are added. But tbh I didn't need it at all for current optional extensions (for example SPADS is totally backward compatible with old servers, only real-battle-bans and all functionalities based on account ID aren't available if server doesn't support it).
0 x

User avatar
hoijui
Former Engine Dev
Posts: 4342
Joined: 22 Sep 2007, 09:51

Re: Dev meeting minutes 2010-05-16

Post by hoijui » 18 May 2010, 10:05

btw, bibim found and fixed the installer generation problem on the buildbot!
0 x

User avatar
aegis
Posts: 2456
Joined: 11 Jul 2007, 17:47

Re: Dev meeting minutes 2010-05-16

Post by aegis » 18 May 2010, 10:16

also, :(
Tobi wrote:now it's 'yes I'm active and there's just one bug left'-state for over a year or so and that blocks a bit work of others on server
That's not true.
0 x

User avatar
hoijui
Former Engine Dev
Posts: 4342
Joined: 22 Sep 2007, 09:51

Re: Dev meeting minutes 2010-05-16

Post by hoijui » 18 May 2010, 10:19

... maybe you could elaborate what is not true, and explain what is true?
"... the youth these days!!"
0 x

User avatar
aegis
Posts: 2456
Joined: 11 Jul 2007, 17:47

Re: Dev meeting minutes 2010-05-16

Post by aegis » 18 May 2010, 10:40

just 'cause I'm a kid doesn't mean you hafta make funs of me
0 x

User avatar
Neddie
Community Lead
Posts: 9406
Joined: 10 Apr 2006, 05:05

Re: Dev meeting minutes 2010-05-16

Post by Neddie » 18 May 2010, 20:50

I think I understand where H was coming from on the forced single version from a development/build perspective, but I would like to note that general simplicity of use is another concern. We don't want new people joining the server, seeing matches using a test version of the engine, and getting bogged down trying to get into a test match. We don't want people who don't know what they're doing even knowing that a version besides the current one and/or the one their installer and thus game demands is active.

Also, aegis is active, but his activity is inconsistent. I am reasonably certain he could produce a stable feature complete version of uberserver and roll it out given a sufficiently long contiguous period of free time. I do see the problems with the present model of server development/maintenance in regards to competition, and I hope the test server solution improves the situation somewhat.

On a side/logistical note, should I input on these discussions after the fact, or would you prefer I stand away?
0 x

User avatar
hoijui
Former Engine Dev
Posts: 4342
Joined: 22 Sep 2007, 09:51

Re: Dev meeting minutes 2010-05-16

Post by hoijui » 18 May 2010, 21:12

am I H?
removing enforced, single version of the engine per server was mentioned by Tobi this time, though hi have not yet heard anyone speak against it. Problems like the ones you described should be avoidable by clever handling by the lobbies.
0 x

User avatar
aegis
Posts: 2456
Joined: 11 Jul 2007, 17:47

Re: Dev meeting minutes 2010-05-16

Post by aegis » 18 May 2010, 22:19

hoijui wrote:removing enforced, single version of the engine per server
http://springlobby.info/cgit.cgi/aegis/ ... l.py#n1018

(note the extra [engine, version] params)

iirc BD and I had this working ^^ we just lacked a way for lobbies to advertise conditional support for it, so I commented it out.

we'd ideally need a way for the server to tell lobbies that yes, the compat features you asked for are indeed supported... so a lobby doesn't assume it can send/receive server, version strings safely when it actually can't.
0 x

Post Reply

Return to “Meeting Minutes”