Dev meeting minutes 2010-05-23

Dev meeting minutes 2010-05-23

Minutes of the meetings between Spring developers are archived here.
Tobi
Spring Developer
Posts: 4598
Joined: 01 Jun 2005, 11:36

Dev meeting minutes 2010-05-23

Post by Tobi » 23 May 2010, 23:22

Agenda
  • Welcome
  • Invite Zydox to meeting wrt infolog upload stuff?
  • Review meeting minutes past meeting
  • Review 'how shall we keep track of who's doing what? Just meeting?'
  • Progress of stuff to be done before release
  • Default CMake build type
  • GPL
  • OS X
  • MT official build
  • New infolog upload site
  • Infolog submit feature
  • Next meeting
  • Anything else? (WVTTK)
  • End
Present: hoijui, Kloot, Tobi, zerver, Zydox
Not present: jK (though reading log later)

Welcome
Tobi/hoijui: [13:42:07] <jk`gentoo@IRC> might be possible that I can't participate in the meeting today, still I will idle in the channel to read the log afterwards
Tobi: for the rest, welcome all
zerver: last meeting, i simply forgot the time
Tobi: ok, can happen
hoijui: hello
Tobi: I want to start with simple question, is it ok if Zydox joins meeting too for the infolog upload stuff?
Tobi: or does anyone have serious objections? :p
zerver: sure, let him in
hoijui: yes
hoijui: ok
Tobi: ok he'll be here in a bit I assume
** [CN]Zydox joined the channel.

Review meeting minutes past meeting
Tobi: any comments on it?
Kloot: I like the new format, much more readable
Tobi: ty Kloot, took a bit more time too then previous ones

Review 'how shall we keep track of who's doing what? Just meeting?'
Tobi: any comments on this?
zerver: etherpad works ok
Tobi: does the etherpad thingie still work? can't say I used it much since I simply didn't get around to any development last few weeks
Kloot: for myself I'm already noticing a lapse in using it, but couldn't get much work done last week
Tobi: [ARP]hoijui: for you, ~ means doing now and * means todo ?
hoijui: no, ~ means Big topic (further explained in the end of the doc), * normal topic
zerver: that etherpad page cant be allowed to grow big, try to be brief
Tobi: maybe at some point we should swap the 'author > i'm doing / todo' layout for 'i'm doing / todo > author', so that 'i'm doing' definitely fits on single screen.

Main points:
  • take care first page doesn't grow too big, less detail if needed
  • for hoijuis points, ~ means big topic (explained at the end of document), * means normal topic
  • at some point maybe change from 'author > i'm doing / todo' layout to 'i'm doing / todo > author'
  • currently it's fine
Progress of stuff to be done before release
Skipped because it's mainly needs input from jK.

Default CMake build type
hoijui: now it is DEBUG2, should it be RELEASE?
Tobi: for whom does it matter, failing packagers who don't set any build type and users who build from source?
hoijui: yeah, users. for packagers it does not matter much
Tobi: I think it makes sense, IMO you may change it
hoijui: users which have no recent package on their systems repo should use RELEASE
Tobi: yeah
zerver: i have seen one or two reports of "slow running" build due to noob building debug by mistake
hoijui: those who need to suply stack traces with line numbers need DEBUG2
hoijui: ok, lets change it then.. we just have to remember to tell to use DEBUG2 when they should do a report
Tobi: what do ./configure-make packages usually do by default? or is it arbitrary?
Tobi: though even if all those defaulted to debug that doesn't necessarily mean we should do so too
hoijui: i don't know..
hoijui: packagers should use DEBUG2 or RELWITHDEBUGINFO
Tobi: maybe RELWITHDEBUGINFO is better default even?
Tobi: as fast as release and can still provide decent backtraces provided user knows gdb or the automatic backtrace thing happens to work
hoijui: would that give line numbers?
Tobi: not completely sure
hoijui: the debug info is still in the executable? or separate file?
Kloot: separate
Tobi: exe
hoijui: hehe
Tobi: separate?
Kloot: doesn't it create a .pdb ?
Tobi: I thought all cmake does was put it in exe always
Kloot: or whatever the gdb format is
Tobi: and only buildbot stripped it out into a separate file
Kloot: ah ok
Tobi: with Debug3 it definitely puts it in exe (got 246M spring exe here)
hoijui: Auswaschbar said, DEBUG3 makes no sense..
zerver: yeah, takes some time to launch big exe
hoijui: i dont remember what it has more then DEBUG2
Tobi: maybe I just use it cause I cba to find out if Debug2 had enough to debug everything I want :)
hoijui: yeah ;-) thats the same reason why i used it for a long time
Tobi: either way, RelWithDebInfo for default?
hoijui: yeah
Tobi: can always reconsider next week :)

Main points:
  • We'll use RelWithDebInfo as default build type for cmake from now on (it was Debug2). Because:
  • it is as fast as release
  • it has debug symbols
GPL
Tobi: I added this, because I think atm shard is blatantly violating GPL. do we care and if so what do we do?
Kloot: AF claimed that SJ once made an exception for AI's
hoijui: AF told me, there is an agreement that AIs are excluded from the need to be GPL compatible
Kloot: but I'd want a citation of that
hoijui: yeah.., also... even if.. i woudl like to undo that
Kloot: same
Tobi: if he really wanted an exception he would have put it in the license.html or some other file in source
Kloot: anyway we've been assuming full GPL compliance for every AI project so far
hoijui: yeah true
hoijui: i got told, that in lots of universities, you are not allowed to make your source public during the course of a project
hoijui: is that true?
hoijui: cause.. most AIs are doen in university projects
Tobi: I've never bothered asking, but didn't do any stuff shared between uni and spring either
Tobi: although occasionally I have put source of something not yet submitted online and got no complaints whatsoever about it :)
hoijui: ok :-)
Tobi: but then again the teacher had licensed it as GPL already, so maybe that changes the situation
Kloot: yeah and it depends on the university's regulations, which in practice no student ever reads
hoijui: yeah.. in my uni.. i never heard about anything like this, .. no prof would care for that :D
Tobi: either way, it's not an excuse to violate GPL
Kloot: I'm pretty sure if I started digging I could find some clauses related to publishing work done in university time
hoijui: well.. AF wont release it anyway until that date he gave..
Tobi: just don't release binaries then or include written notice valid for 3 years on how to get source
hoijui: and.. who would start a law.. thing?
zerver: what is the problem really, I thought the AIs were separate project
Kloot: they link to the engine
zerver: ok
Kloot: and include all kinds of headers from it, etc.
hoijui: hmm.. so.. it is not allowed to release anything (binaries) as long as source is not available?
Tobi: correct
hoijui: i mean.. we seem to agree on the theory, but what to do in practise
hoijui: so he can not publish new stuff from now on, or will we close his thread?
Tobi: well ofc in practice you can't do really much (except making sure no binaries are linked to on forum or so), but I think we ought to at least make a statement, cause otherwise next violation someone can just say: 'but at that time you didn't care either blablabla'
hoijui: hmmmm ok
Tobi: that's my opinion on it
hoijui: so he has to clear out links to binaries in the thread and.. can not add links to his forum?
Kloot: obviously preventing distribution is impossible
hoijui: so we make our stand clear, and disallow publishing further binaries
hoijui: that is easy to do
Kloot: but we can disallow links in any spring-project-controlled environment (forum/lobby)
hoijui: i guess we just try to talk to him nicely
hoijui: and eventually he will accept
hoijui: thats the best chance
zerver: this is a small issue imo, we should not be too hard
hoijui: well..
hoijui: there are quite some people writing config files already, integrating it into their mod/game
hoijui: and if he never releases source.., they are fucked, did the work for nothing
hoijui: with AF.. we can assume he will stick around, but for others.. who often go as fast as they go...
zerver: "for nothing", well depends on how you look at it
hoijui: it would be bad: one spring release later, it woudl be unusable, and no way to fix it
hoijui: or if they add something to the mod that needs a small change in the AI
zerver: it is bad, but they did get the binary for free :)
Kloot: imo if anyone feels he does not want to release his source, he should pick another engine
Tobi: either way I agree that just asking at first to remove binaries OR include a written notice to get source is a good start, next week we can see what is the reply and what to do further
hoijui: ok, i will talk to him if i see him online, or write pm before next weekend

Main points:
  • Shard is violating GPL by distributing binaries and no source
  • Hoijui will send a PM to AF to stop doing that (either cease distributing binaries OR include source OR include written notice on how to get source)
  • Most important is to make sure one violation that hasn't been dealt with can be used as an excuse for more violations...
OS X
hoijui: i jsut saw an other guy trying to compile this week, and.. maybe we should make clear that we need a machine for this that we have access too, or we will not help
Tobi: well do you have the interest, even if you have such machine, to get it working on OS X?
hoijui: :D yeah ok..
hoijui: i think it would not need much effort
hoijui: we could set up some.. slave buildbot there
Tobi: that may be an idea yeah
hoijui: and once it is fixed (some OS X fan can do that), it would only need msall adjustments every few commits
Tobi: that applies _after_ it starts working reasonable on OS X at some point, then it's nice to prevent simple build failure regressions
hoijui: true.. someone had to do that.., but we had some that did that a few times already.., or it sounded so at least
Tobi: but getting it to work is an OS X person's job IMO (though I don't know what the exact state of that is atm)
hoijui: yeah, i agree
zerver: but os x is basically linux :)
Tobi: 'basically' <- that's the problem :)
hoijui: some parts yes.., but in terms of packages eg.. or default dirs, its compleetly different
hoijui: there is also assembler code..
hoijui: but yeah, i think we would manage to maintain it once it runs
hoijui: the prerequisite is a machine we have available 24/7
hoijui: its not like we would put a big load on it
zerver: yeah, would be good to know when a commit breaks the OSX build at least
Tobi: just make -j4 complete spring every 5 min or so :p
hoijui: :D :D
hoijui: i think aegis once mentioned he might have that, and AF said something too
hoijui: once a day (if there were new commits)
Tobi: nah, that can be talked about, should be possible to schedule nightly build or build after 1 hour of no commits
Tobi: I still need to look up how to do that in buildbot, after fixing makensis
hoijui: are you investigating the installer fails? or you mean.. after someone fixed makensis? ;-)
Tobi: no, I was trying to get buildbot going earlier this week, but somehow makensis takes like 18 min to build installer, while full spring compile takes 4-5 min on same box
hoijui: yeah true.. it takes quite long too here
hoijui: not more then compiling spring but... there is a long periode with no output
hoijui: ah
Tobi: 15 min of it it's just sitting in D state (uninterruptable sleep) without using significant CPU or disk (AFAICS)
Tobi: I think it should take at most 3-4 mins
zerver: speaking of that, cmake does not support multi core builds?
hoijui: cmake does
hoijui: but i think MinGW does not udner windows?
Tobi: in general it does (just make -jN, where N is number of processes to spawn in parallel)
hoijui: Kloot knows, thats the reason why we still have SCons
zerver: ok, im still using scons because i wasnt able to get that -j thing working
Kloot: -j is for scons, but make itself should also allow it
Tobi: I think I heard that too sometime yeah, that make -jN doesn't work on windows
Tobi: anyway, in the mean time: any points of action wrt OS X?
hoijui: we should declare that we want a machine donated, does not have to be fully/all ours
Tobi: should we put a request for OS X developer in dev forum / as news post even?
Tobi: or first investigate what current state is?
hoijui: yeah that too, but the machine is more important
hoijui: without it, the work of the dev is soon lost again
zerver: is osx sync proven impossible?
hoijui: i think someone once tested and said he synced
zerver: ok
Tobi: it's chicken or egg, I wouldn't say either is more important - we just need both :)
hoijui: yeah.. but we had a few devs already, and never a machine
hoijui: and the machine wont run away so fast
hoijui: but otherwise true
Tobi: true yeah, but is that code already integrated? I mean all of it, I know some parts are in master
hoijui: BD told me, that there is still stuff in yokosous repo that is not integrated yet, i forgot why though
hoijui: ok so.., we should declare that we want the two, so, who will do that? :D
Tobi: I can make a post I suppose
hoijui: nice :-)
Tobi: will you try to gather some more data on what current state is? or maybe I can ask in that post too
hoijui: yeah
hoijui: BD will know most, if none of the other, former OS X devs is around
Tobi: if it's on forum it may be easier to find later on then hidden in chat log on forum :)
hoijui: yeah :-)

Main points
  • OS X keeps getting half fixed by someone who then disappears
  • No way for non-OS X devs to maintain it; we can't even test whether it builds
  • Suggestion: OS X build slave after OS X is made working properly
  • Tobi will make post to request OS X dev + machine
  • OS X does sync
  • There's still some unmerged OS X code in yokosous repository. (reason for not being merged: unknown)
MT official build
zerver: need to outline what is sufficient compatibility when it comes to MT
zerver: is everyone clear on why 100% compatilbility is impossible?
Kloot: gadgets can call opengl in the sim-thread, any others?
hoijui: would it be possible to scan all the mods gadgets for this, and refuse to load it with MT spring?
zerver: yes, but more importantly LUA code can save data while sim is running
Tobi: you mean Lua draw code?
zerver: global lua data can become invalid because sim runs in between call-ins
zerver: Update() makes a list of units, DrawXXX() draws effects for these units, but sim keeps running in between in the MT build, so the list of units is no longer valid
zerver: this is just an example of course
hoijui: but this would crash, not desync, would it?
Kloot: it might do one or the other
zerver: LUA crash, invalid unit id or something like that
zerver: and then there is also the OpenGL issue, with calls from Sim code
hoijui: ok. and how woudl the desync happen?
zerver: desync would not happen, only LUA crash
hoijui: ah ok
zerver: MT is compatible wrt sync
zerver: the OpenGL issue can be partially worked around with glShareLists that would allow some GL stuff in Sim
hoijui: can Sim not be totally GL free? (theoretically)
zerver: sure
Kloot: if a unit is deleted while lua still has a reference to it and some synced callout doesn't check for this, you might desync still
zerver: nope
Tobi: in Lua it are all unitIDs, so you'd get invalid unit ID error
Tobi: or if it's getter it usually just returns nil
zerver: synced call ins are not a problem, they run exactly the same as in single thread
zerver: draw is the only problem
Tobi: or if you're unlucky it returns properties of new unit that got same ID :p
Kloot: yes
zerver: nevertheless, glShareLists opens a new problem
hoijui: in the DrawXXX method, is it possible to prevent the crash by checkign each unit in the list agian, if its still valid?
zerver: yes, i did this with LUPS
hoijui: ok :-)
Tobi: but what if it becomes invalid between the check and calling the call-out ?
Tobi: it seems to me only way you can check is checking that every property you fetch is not nil
Tobi: (though the probability for crash would be smaller already if you check)
zerver: iirc this cannot occur
hoijui: do you lock ?
zerver: because there are som locks
hoijui: perfect.. then i see no real issue
zerver: but the locks are just for deletion of units and such
zerver: unit properties change at runtime like crazy ;P
hoijui: that mean, they might be drawn in a wrong way, or any worse?
zerver: typically some artifacts yes
zerver: some lines drawn across the screen for instance because some coordinates are partially updated
zerver: nevertheless, glShareLists makes it hard to determine if a GL call made from Sim is invalid or not
hoijui: and removing all GL stuff from Sim is too much work?
zerver: well, it is removed already
zerver: problem is the mods
hoijui: and if we just refuse to run such mods? is it hard to scan for this?
zerver: it probably is
hoijui: mm :/
Tobi: I think it is ok to include MT in installer at some point once installer generation is working decently again, but I think non-MT should stay default
Tobi: and then there is the problem, how to make it toggleable. ofc in SL user can just change spring.exe to spring-mt.exe or so
zerver: yeah, dll hell
Tobi: no clue if TASClient has option like that
Tobi: zerver: how so, does -MT use different set of DLLs than normal spring?
zerver: no, i meant the launcher thing, spring-mt.dll
Zydox: Isn't it just for the lobby clients to start with "spring-me.exe script.txt" instead of spring.exe?
Tobi: yeah
hoijui: i did not work on the launcher recently
zerver: in qtlobby you can choose executable, thats all i know
Tobi: I guess indeed it could be up to lobby authors for now to offer some configurability to set the executable to launch (as far as they don't have that already)
hoijui: maybe i will if pureint is finnished
Zydox: I would assume it's quite easy to add that option to TASClient as well
Tobi: yeah should be
Tobi: zerver: was just including it the goal, or actually making it the default?
zerver: just including it, so that it becomes more widespread (right now its kinda hidden)
hoijui: well.. what woudl that mean... would the default build of spring build normal and MT, or just the buildbot?
Tobi: buildbot could just make both I suppose
Tobi: maybe with some support from cmake so that if you enable MT the binary is called spring-mt.exe, so it doesn't overwrite the normal exe
hoijui: mm ok
zerver: yeah, until the launcher has some progress
Tobi: indeed don't want 'make spring' to go build 2 springs..
hoijui: MT spring shoudl be an additional build target, not the normal spring with some different defines
hoijui: so it does not need rebuilding all of spring when trying one or the other, everytime you switch
zerver: yup
Kloot: make two cmake build dirs then :)
Tobi: well it changes quite some global #define stuff so probably needs rebuilding almost everything
hoijui: yeah thats true.. but i meant..
hoijui: if i am in my local repo, and build nromal spring, and then MT spring, and normal spring again, and then MT spring.... it always rebuilds all of it
Tobi: ccache
hoijui: yeah...
zerver: yes, that would take time :)
Tobi: not sure how common use case that is though..
hoijui: yeah that would help, but with an additional build target, it would not cost anything anymore
zerver: msvc has got it right, each config in separate build dir
hoijui: we also have separate build targets for dedicated, and headless spring
Tobi: it's same with cmake
Tobi: you just need to decide for yourself what you put in which dir
hoijui: we would have to find a good way to share what can be shared
hoijui: so we dont have two long cmake build files with nearly identical content
Tobi: (just like in MSVC you can modify some global #define in the current build target and it will also rebuild in current dir, overwriting your previous build)
hoijui: we would have tools/MT/CMakeLists.txt, for example
Tobi: I'm not sure it's even possible with MT
hoijui: why?
Tobi: because without MT, locks aren't compiled in, and with MT, they are, and they are spread over lots of files
hoijui: hmm..., why is that a problem?
Tobi: it's a problem if you want to compile those files only once, because you need to compile them twice :d
hoijui: ah.. yeah i never though of compiling them only once
Tobi: how do you mean the sharing then?
hoijui: but to compile them to different locations/*.o files (no sharing)
Tobi: oh, just build dir?
zerver: sharing is a no-go
hoijui: yes
Tobi: as Kloot said
hoijui: yeah.. Kloot meant something else, with the same effect
hoijui: he meant to just manually specify a different build dir
hoijui: i would do that with cmake
Tobi: do you mean only on buildbot then?
hoijui: no, just excldue the MT build from defauklt build (in cmake terms: EXCLUDE_ALL)
hoijui: so it only builds when you run: make MT
zerver: and when building installer
hoijui: the buildbot will do that then
Tobi: ohh... you just want to save typing 'make spring' instead of 'make' :P
hoijui: .. no
zerver: he wants to save compile time
hoijui: yes
Tobi: yeah but point is you can already do that
hoijui: yes, as kloot said, by using different build dirs manually
Tobi: though maybe indeed for self-compiling users it's nice if it doesn't compile spring twice by default
hoijui: ..exactly
Tobi: think I get your point now :P
hoijui: ok :-)
hoijui: i could have a look at this
zerver: good
hoijui: .. the same thing is parcitcally already done for springheadless
zerver: i appreciate it
hoijui: it also uses close to all source files of spring.exe (minus Rendering)
hoijui: :-)
zerver: i think when MT is included in installer, mods will become more compatible
hoijui: yeah
zerver: i will investigate glShareLists
hoijui: making mods fail to load which have bad compatibility would also help :D
hoijui: of course only if we have at least one mod that is well compatible
zerver: and LUA incompatibility cannot be detected i think
zerver: so ppl will just have to learn to do it right
hoijui: hmm ok :/
Zydox: Is it recorded in the infolog, what LUA script which crashed?
zerver: yes
zerver: invalid GL calls also
zerver: i will expose some locks to LUA when i have time
zerver: right now LUA+MT is a bad combination

Main points:
  • MT will be included next to normal exe
  • Normal exe will remain default
New infolog upload site / Infolog submit feature
hoijui: maybe Zydox could help there
Zydox: makeing statistics regarding the errors atleast...
hoijui: yeah that.. and if we have that, we could auto spam the mod makers with emails or something :D
zerver: :)
hoijui: or just tell them to watch the page (boooring)
Zydox: Though I'd say that one issue is this... that crash reports are only submitted if spring crashes... (or desyncs if the lobby is TASClient)
Zydox: so regular errors aren't submitted...
zerver: lua mt errors go unnoticed
Zydox: But that shouldn't be to hard to fix
hoijui: well, there will still be those in infologs when people crashed
Zydox: Atleast not with the TASClient solution, there the infolog script is scanned for a crash or desync after each battle... so it should only be to add LUA errors as well...
hoijui: do you think infologs should be uploaded whenever they have MT Lua errors?
zerver: not really, will be too much i think
hoijui: ok, yeah.. i coudl imagine that too
Zydox: If I could decide, I would upload every infolog... would give the abillity to see how often a setting crashes...
hoijui: new infolog site.. can you give a link to a site that has your recent changes?
Zydox: http://infologs.zydox.se/?Crashes & http://infologs.zydox.se/?Crashes&Version=1 ?
Zydox: As it's now the percentages are based of the number of crashes
zerver: interesting for statistics is mainly the acual crash address, and if this address is not in spring.exe, the topmost one that is in spring.exe
hoijui: this is stuff .. in addition to the one on koshis machine, right?
Tobi: uploading all offers some nice possibilities, like tracing Lua functions / features to see which are really completely unused
Tobi: so we can wipe them from next Spring unless we think they are really useful
Zydox: yeah, game spread over different OS's and stuff like that as well
Tobi: would need some more spam to infolog though (and probably also decoupling infolog from chat)
Zydox: (I'm a sucker for statistics)
hoijui: the most important thing we need, is a translated stack trace
Zydox: [ARP]hoijui: zydox.se is just the development site, it's using all crashreports from yesterday and before... (3000)
hoijui: and linkage of that with github, like bibims buildbot does
hoijui: ok
Tobi: for proper statistics it's right though that you also need all infologs; since you want to calculate if there's a correlation between a setting and a crash and you need to know the % of people with that setting of the whole 'population', and the % of people with that setting who crashed (or something like that)
Zydox: Tobi, exactly
hoijui: yeah true
hoijui: but.. woudl we really be able to use that data?
hoijui: and... we are not google
Zydox: But that would basically mean that it would have to be implemented into spring and submitted automatically...
Zydox: I'm becoming the next google ;)
Tobi: more like microsoft with all it's 'send crash report' buttons :)
hoijui: :D
Tobi: don't know about privacy either
hoijui: i mean.. much more important is to get the basics up first, no?
Zydox: Should add a popup with "Are you OK to send game statistics to the spring site" and then a checkbox for "Always allow"...
hoijui: that without which it is stil compleetly useless
Tobi: yes
hoijui: first make it usefull, and then collect all the data, if it has ot be
Zydox: zerver had a nice idea for the debug symbols
zerver: the checksum, exe checksum calculated when spring crashes...
Zydox: buildserv should keep a record with debug symbols and which checksum they belonged to...
Tobi: makes sense to do that yeah besides the git describe stuff (that would be for human readable)
zerver: we thought about launching a second spring instance to reduce chance for crashes in the crash handling
zerver: like spring.exe --submitinfolog
zerver: this second instance calculates the checksum and opens a submission gui
Tobi: almost sounds like job for that launcher thing?
Zydox: would also work great if we were to start submitting everything later on, then that would just have to be called after each non crash as well
hoijui: yeah, would also profit from launcher, though.. .
Tobi: that way it would also allow the GUI to be written in a sane(r) language :)
hoijui: i think we shoudl really talk about what we want the site to do now :P
Tobi: true, we're getting sidetracked on longer term stuff
Zydox: Isn't this part of the solution on how to find out which debug symbols to link the report to?
hoijui: you mean the GUI that shows up when launchign spring.exe wihtotu arguments?
Zydox: Thought that was one of the main benefits of the crash report site?
Tobi: the checksum yeah
hoijui: spring version plus binary name should work too, as it did so far
hoijui: of course the checksum thing is better
Tobi: checksum could be nice addition to ensure integrity
hoijui: yes
Zydox: How can the information be accessed today?
zerver: version string?
hoijui: the infolog contains the full version (git describe + other stuff) of spring
hoijui: the stack trace has the binary names (spring,exe, SkirmishAI.dll, ..)
hoijui: you call a function with the stack trace + the full version
Zydox: An example: Spring 0.81.2.1 (0.81.2.1-0-g884a107{@}-cmake{MT}-mingw32MT-Sim)
Zydox: [ 5654] (8) C:\Program Files (x86)\Spring Engine\spring.exe [0x0093DCC9]
Zydox: How can I get something usefull out of that?
Tobi: currently, only by saying !translate in #buildserv
hoijui: i guess we can ask bibim for the source of his script, or just write a new one
Tobi: yes, we can talk about some API for buildbot that infolog site can talk to
hoijui: its basically jsut calling addr2line for each line in the stacktrace, using the right debug symbols
hoijui: hmm :-)
Tobi: preferably once buildbot is running better
hoijui: :D hmm..
Tobi: yeah, addr2line + lots of regexes is the trick :P
hoijui: :-) nothign you have to do, zydox, i think, you will get back a list of lines to display, probably with the github URL already included
Tobi: yeah, it should be implemented as API on buildbot since that has the data
hoijui: yeah. until this is in place, just use a dummy function that returns the stakc trace it got as input
Tobi: we can talk about that outside meeting in one-two weeks or so, when I should hopefully have kicked makensis to work harder and made buildbot put installers online and stuff
hoijui: ok, i will probably also try to debug nsis a bit
Tobi: btw Zydox do remember the 'view stacktrace without downloading zip' feature request :)
hoijui: hmm...
Tobi: anyway, since this meeting is getting extremely long already, was this enough about this point? :)
hoijui: yeah :D
Zydox: Hmm... so I should make the crash report's statistics from those to pages link to the actual reports and make it possible to display the stacktrace from there?a
Tobi: [CN]Zydox: for stacktrace may work best for now if you can just dump stacktrace on page where you also display settings and stuff now
Tobi: i.e. this page: http://infologs.springrts.com/details?id=4
hoijui: yeah, individual report site
hoijui: and a link ot the raw infolog, if possible
Tobi: more details can be discussed after meeting (and dinner possibly :)) if desired
hoijui: mm :D
Zydox: I'll see what I can do and poke you guys about it later :)
hoijui: thanks :-)
Tobi: kk thank you [CN]Zydox
  • for proper statistics all infologs should be uploaded, not just the one with crashes
  • rather focus on things that make infolog site usable soon (i.e. view stacktrace without downloading zip, integration with buildbot)
  • the latter is currently only possible using !translate in #buildserv
  • no need to upload infologs with MT errors (for now)
  • use checksum to reenforce link between executable and it's debug symbols
Next meeting
Tobi: next week same time again?
hoijui: yep
Tobi: or does the length of this meeting suggest we need more? :)
hoijui: i will be totally away form PC most of next week
hoijui: from*
Kloot: no, just more highlevel discussion :)
Tobi: yeah probably
Tobi: ok hoijui
zerver: it's fine, well see if more meetings tend to get long
Tobi: next time I'll try harder to force the meeting to fit within 1 hour :)
hoijui: :D .. i do not care so much
Tobi: anyway, next week same time same place fine with everyone?
zerver: y
Kloot: y
hoijui: yes
Tobi: kk, good

Main points
  • next meeting next week same time same place
Anything else? (WVTTK)
hoijui: yeah.., spring is not compiling currently, but zerver siad he will change it
zerver: it compiles for me with scons
hoijui: ..thats it :D
hoijui: mm ok
zerver: regular or dedicated?
hoijui: it fails to compile on GCC and on MinGW here
hoijui: regular only on gcc, dedicated on both
hoijui: it is easy to fix but.. if you want to refactor your change anyway, i will just wait for that
hoijui: i dont need it to compile right now
zerver: i have done so already
zerver: but if my cleanup still fails, can u either send me the compile error or try to fix it?
hoijui: doh! :D, sorry then, did not fetch today
hoijui: still fials to compile
zerver: ok
hoijui: as it still uses GlobalUnsycned in dedicated server
hoijui: does dedicated server need this new stuff?
hoijui: or can we #outdef it?
hoijui: the team color is synced, yeah?
zerver: no
zerver: i will outdef it
hoijui: ah, perfect then :-)

Tobi: Kloot: fyi, didn't got around to 1919 yet
Kloot: no problem, I only looked at it a bit more myself
Tobi: any new information?
Kloot: no.. if I dump the Lua stack I get basically the same as what I already knew
Tobi: ok, I'll just dive into it too later on
0 x

User avatar
aegis
Posts: 2456
Joined: 11 Jul 2007, 17:47

Re: Dev meeting minutes 2010-05-23

Post by aegis » 23 May 2010, 23:31

"make -j" definitely works on windows/mingw32

I'm willing to give dev(s) ssh access to a snow leopard box running 24/7 as long as it's up (I'm moving in the next couple of months) with a full build environment, and help them setup a script to run constantly for builds.

my internet connection can be intermittent... streaming video can make it laggy for everyone connected... after I'm moved I can setup a more permanent box with a better internet connection if people are still interested.

we could also look into cross-compiling for osx again http://devs.openttd.org/~truebrain/comp ... arwin9.txt
0 x

User avatar
AF
AI Developer
Posts: 20669
Joined: 14 Sep 2004, 11:32

Re: Dev meeting minutes 2010-05-23

Post by AF » 24 May 2010, 00:29

I would rather you had spoken to me prior to it being outed in a public chatlog. Finding the exemption without getting SJ to come along and verify it is problematic since a lot of the forum was wiped, however the exemption remains for numerous reasons, and there have been threads discussing this.

AIs are exempt and should remain so because:
  • At the time it would have made almost all the AIs in the community illegal
  • It makes academic work a lot more difficult in certain establishments
  • It limits code use in AIs to GPL code.
  • It acts as a hefty deterrent to the kind of people who are most likely to set up shop and build an AI
  • It can dampen AI competition
I support the idea that AIs should be opensourced, I intend to open source Shard, and I have left the lua component in plain text, I could easily have ran it through an obfuscator and gotten a few extra sentences in my dissertation and avoided some rather glaring looks from my supervisor.

I ask instead, what do you loose by allowing the exemption? Potential source code? I think that in that situation the people would not bother writing it in the first place if they intended it to be closed source.

What about projects like Shard? You do know there's nothing stopping me spending a day or two, upping sticks, and moving to the starcraft community or ORTS!

Its in the best interests of the community to allow AI developers to make the decision for themselves. It has worked this way for 5 years now, and attempting to wade in and make demands and assertions now does us all no good whatsoever.

There is also the moral and ethical conclusions of these actions. What about all the classic AI developers who used their projects to get jobs? What if a new generation appeared and went to job interviews and talked about their projects only to reveal at the end that their work had been banned and declared illegal because they wouldn't publicly release the source code?

I certainly would never have built NTai if I had known I'd be forced to opensource it. No AFLobby, no shard, no darkstars, no dynamic music AI, metalmaker AIs, cookiebots, and about 20 other projects, and the influence I had on countless other people would have been lost when I got bored and moved on somewhere else. Persuading Submarine to release AAI source took the community a long time, and your lucky Veylon freely contributed his source code in the middle of the discussion to avoid being drawn in.

And in the meantime, what do you intend to do about the likes of EvolutionRTS and Kernel Panic? EvoRTS uses Shard as its official AI, and Kernel Panic bundles versions with some of the newer installers. With Shard banned Evo looses its best AI, and since there isn't much interest elsewhere to develop AIs for Evolution, Forebodings in for a long wait for a replacement.
0 x

Kloot
Spring Developer
Posts: 1865
Joined: 08 Oct 2006, 16:58

Re: Dev meeting minutes 2010-05-23

Post by Kloot » 24 May 2010, 01:31

AIs are exempt and should remain so because:

* At the time it would have made almost all the AIs in the community illegal

Not relevant to the situation as it has existed for the past ~3.5 years.

* It makes academic work a lot more difficult in certain establishments

Then such work should really be done for another engine.

* It limits code use in AIs to GPL code.

Any developer knows that requirement going in. Some won't like it, some will.

* It acts as a hefty deterrent to the kind of people who are most likely to set up shop and build an AI

Speculative; it could just as easily be an attractor (sharing your code might not be such a bad prospect when you have access to that of everyone else).

* It can dampen AI competition

Also speculative, and with FOSS there is still the element of challenge / pride / whatever in doing things better than the competition (if you have to think in such terms), which goes both ways.
I ask instead, what do you loose by allowing the exemption?
The Spring project has been based on open development since its release to the community. Speaking for myself, I don't want any binary blobs taking advantage of all that work. There is also the more practical point of maintaining a consistent policy.
What about projects like Shard? You do know there's nothing stopping me spending a day or two, upping sticks, and moving to the starcraft community or ORTS!
Then do so. You're not going to hold Spring hostage with personal ultimatums.
There is also the moral and ethical conclusions of these actions. What about all the classic AI developers who used their projects to get jobs? What if a new generation appeared and went to job interviews and talked about their projects only to reveal at the end that their work had been banned and declared illegal because they wouldn't publicly release the source code?
An employer would first and foremost want to see the source and probably custom builds of it anyway, whether it was publically released or not. Stop making up all these overly-dramatic scenarios.
I certainly would never have built NTai if I had known I'd be forced to opensource it. No AFLobby, no shard, no darkstars, no dynamic music AI, metalmaker AIs, cookiebots, and about 20 other projects, and the influence I had on countless other people would have been lost when I got bored and moved on somewhere else.
And in this alternate timeline we would never have known or cared. You can also turn the argument around and ask how much Spring as a whole gained over the years from being GPL; someone like trepan might not even have joined otherwise. This is not just about your own contributions.
0 x

User avatar
AF
AI Developer
Posts: 20669
Joined: 14 Sep 2004, 11:32

Re: Dev meeting minutes 2010-05-23

Post by AF » 24 May 2010, 01:42

Quit ignoring my points and attacking my personal character

Spring was not released as open sourced originally, nor was it originally GPL. This came after the release.

I have already stated I want to open source Shard, and that it was my intention to do so as soon as I could. I assumed this was fine since SJ exempted AIs, albeit a very long time ago. I even gave a time frame of when I would be able to do so.

There is also a difference between handing potential employers source code as part of a portfolio, and being forced to publish it online for all to see. Forcing AI developers to open source would create a slew of problems, and these issues are likely to only crop up for new people who will never utter a word and just move on to something else, depriving us of potential contributors.

I agree that AIs should be open source, but I disagree with forcing it upon AI developers. It is an ideal, but it should not be the rule.
Last edited by AF on 24 May 2010, 02:50, edited 1 time in total.
0 x


User avatar
AF
AI Developer
Posts: 20669
Joined: 14 Sep 2004, 11:32

Re: Dev meeting minutes 2010-05-23

Post by AF » 24 May 2010, 01:54

Then why wasnt the exemption and license issue raised sooner? My works being marked right now, this kind of stuff should have been talked about back when I was considering my options, I've already implemented it all and handed it in now.

5 years is a very long time, and the people in question have been here for the vast majority of that and played major roles in development, and spent a lot of time meddling in license discussion threads, Shard has been available for at least 2 months now and not a word has been said about banning my AI or closing down my threads till now?

Or that vital decisions about my projects future are being made without my knowledge? I would never have known if I hadn't read this thread, and even then, it was posted on a public forum before I had even been told, and I still haven't been told or even contacted in private.

I really don't appreciate that I wasnt contacted beforehand, or even made aware, and that I had to find out in a public forum thread that one of my primary projects was brought into question like this. Did nobody think to message me before it was raised in the dev meeting log as a discussion?
Last edited by AF on 24 May 2010, 02:52, edited 2 times in total.
0 x

User avatar
aegis
Posts: 2456
Joined: 11 Jul 2007, 17:47

Re: Dev meeting minutes 2010-05-23

Post by aegis » 24 May 2010, 01:56

AF wrote:Did nobody think to message me before it was raised in the dev meeting log as a discussion?
I don't believe they had discussed it before this - it's just something Tobi brought up in this dev meeting with the statement "do we care?"
0 x

User avatar
AF
AI Developer
Posts: 20669
Joined: 14 Sep 2004, 11:32

Re: Dev meeting minutes 2010-05-23

Post by AF » 24 May 2010, 02:15

This is my degree, my dissertation is still being marked, I would rather not have the legality of my work brought into question at such a critical time, especially considering I was preparing to open source it in a months time and readying 0.26 to fix issues with the current build.

I could have done what the ICE people did, they're not releasing that AI for a while yet because of academic restrictions, but I needed to demonstrate and evaluate Shard in my write up, and test that my design worked, AI FFAs and the statistics are sufficient on their own but I wanted more information for a better write up.

If I was a newcomer whose first thread was the Shard release thread, do you think I'd be willing to sit around and continue with Shard after this?
Last edited by AF on 24 May 2010, 02:53, edited 1 time in total.
0 x

User avatar
lurker
Posts: 3842
Joined: 08 Jan 2007, 06:13

Re: Dev meeting minutes 2010-05-23

Post by lurker » 24 May 2010, 02:45

If the source is going to be available soon, why not require a written order and let the lag time in fulfilling it suffice?
0 x

User avatar
AF
AI Developer
Posts: 20669
Joined: 14 Sep 2004, 11:32

Re: Dev meeting minutes 2010-05-23

Post by AF » 24 May 2010, 02:58

Possibly, I'm sorry for blowing my top over this, but I'm in a predicament now in that it could affect the marking of my work, and I have bugs in 0.25 I need to release fixes for.

I intended to release 0.26 an hour ago, and if I had not seen this thread my Shard thread would have been locked. It also means Evo installers are drawn into this predicament until I can release the source, as well as the other games that are left with no AI by me not distributing. All the while I have users running Shard 0.25 and getting issues and blaming me.

The last time I had a situation like that it crippled NTais reputation after an old version was bundled with spring which had crippling flaws, by the time it was updated a fixed version had been available on my website for 9 months.
0 x

Kloot
Spring Developer
Posts: 1865
Joined: 08 Oct 2006, 16:58

Re: Dev meeting minutes 2010-05-23

Post by Kloot » 24 May 2010, 03:18

Quit ignoring my points and attacking my personal character
You really don't want to accuse me of personal attacks now.
Spring was not released as open sourced originally, nor was it originally GPL. This came after the release.
I suppose "since its release to the community" wasn't unambiguous.
I agree with open source AIs, but I disagree with forcing it upon AI developers.
That may be, but the GPL (irrespective of how you view it) as it currently applies to Spring in lieu of any official written exemption from its copyright holders does not offer people a choice in open-sourcing their code if they want to distribute anything.
Further more why was the exemption and license issue not raised sooner? Should it not have been raised months ago? I could have done with being made aware back in November of last year when I was proposing these things to ym supervisor?
You were secretive about the whole project to begin with, so informing you back in november would have been impossible without you making us aware of it (and your license plans) first. But I agree that the proper time to bring this up would have been right when you released the first build, so for my part an order to comply by the end of July (2010) is acceptable.
Last edited by Kloot on 24 May 2010, 14:21, edited 1 time in total.
0 x

User avatar
Zydox
Lobby Developer
Posts: 453
Joined: 23 May 2006, 13:54

Re: Dev meeting minutes 2010-05-23

Post by Zydox » 24 May 2010, 08:42

Thanks for the nice summary, Tobi :-)
0 x

User avatar
hoijui
Former Engine Dev
Posts: 4342
Joined: 22 Sep 2007, 09:51

Re: Dev meeting minutes 2010-05-23

Post by hoijui » 24 May 2010, 09:02

AF, i did raise that issue to you in private/lobby AI channel. that was at the time you did your first release. you said that there was an exception for AIs. i never heard about that, which probably has to do with it not being written in any significant place, as Kloot pointed out. it seemed fishy back then, but i was not aggressive enough at the time.

So the right way to do it, form now on (other AI projects):
If ever released in any way, it has to be GPL.
If you do it for university, and are not allowed to release source until the project is finished, you should also not publish it to the public in binary form.

Of course this could hinder development as you say, but as Kloot mentioned, it could just as well make things better. At last, this is the very nature of the GPL, and as spring is GPL, we are on this side of the road. It has its bumps too, but at least it is not going the wrong way ;-)
0 x

User avatar
AF
AI Developer
Posts: 20669
Joined: 14 Sep 2004, 11:32

Re: Dev meeting minutes 2010-05-23

Post by AF » 24 May 2010, 15:26

The exemption was not raised as a problem until yesterday. Your posts in my thread were also merely questions asking where the source code is, and saying it would be nice fi I did, no mention of the legality or of GPL or exemptions was made.

The first mention of any of this was not by you, but a post I made yesterday in the Shard thread on the last page. The words GPL license and exemption are not mentioned in that thread at all.


As for lobby, again there is no mention at all. There is only one mention of the word license in my logs for #ai:
[16:21] <AF> because I dont want to have to buy a $99 license form Apple
No mention oF the word GPL, exempt, exemption, and no PMs from you. The only references to the license issue have been after the posting of this thread, and in channels with developers discussing things when I'm not there.

There is also no reason why the words Shard and AF could not have been substituted for generic names in the log minutes to prevent defamation. The developers would have known, and it'd be obvious to me and other main community people, but it would have been a lot less insensitive.

It would also have made sense to prod me either before the meeting, after the meeting but before the thread, or to have removed that section until you had spoken with me.
0 x

User avatar
hoijui
Former Engine Dev
Posts: 4342
Joined: 22 Sep 2007, 09:51

Re: Dev meeting minutes 2010-05-23

Post by hoijui » 24 May 2010, 16:37

i am absolutely sure that i talked to you about the legality of not releasing Shard source and GPL briefly, right when you did the first release of Shard. This is when you told me about the exception.

Even if you do not believe me, it is not my or anyone else's duty to personally inform everyone that uses spring for a project that it is GPL. It is clearly specified in many places, and there is no mention of any exception in any significant place.
0 x

Tobi
Spring Developer
Posts: 4598
Joined: 01 Jun 2005, 11:36

Re: Dev meeting minutes 2010-05-23

Post by Tobi » 24 May 2010, 17:22

Can someone quote or preferably link to this exemption?

IMO the most important point is that that is cleared up; As far as anyone looking at Spring repo is concerned it's plain GPL, nothing more, nothing less.

(In other words, all engine contributions are also plain GPL, not GPL + exemption, and maybe some contributions wouldn't have been made if it was GPL + exemption...)
Kloot wrote:But I agree that the proper time to bring this up would have been right when you released the first build, so for my part an order to comply by the end of July (2010) is acceptable.
I agree with Kloot here.
0 x

User avatar
AF
AI Developer
Posts: 20669
Joined: 14 Sep 2004, 11:32

Re: Dev meeting minutes 2010-05-23

Post by AF » 24 May 2010, 18:17

I am aware spring is GPL, but it has been years and nobody has raised an issue about closed source AIs legality before, nor has anybody attempted to contest or disprove or even question the exemption till now. I would not have released otherwise.

I still cannot find any mention of people questioning the legality, I've checked the NTai thread up to the point were posts predate my announcement of Shard, and the entire Shard thread, and the #ai channel logs, and the #sy logs. Nobody mentioned it till yesterday in this thread
0 x

User avatar
Neddie
Community Lead
Posts: 9406
Joined: 10 Apr 2006, 05:05

Re: Dev meeting minutes 2010-05-23

Post by Neddie » 24 May 2010, 19:24

I don't have the time to contribute definitively to this AI & GPL discussion. While I have always been a supporter of open source as it pertains to code, I am a greater supporter of artistic rights and this has been the source of conflict in numerous discussions of aggregation, art assets and music.

I suggest that the engine developers have the right of it in general, but there are some feasibility/opportunity points raised by AF which are relevant and should be considered. It could satisfy most, though not all, to make a clear provisional exception for known/visible academic AI development which permits an earlier declared project to delay source release until the academic project is completed, provided; A. The binaries used in testing/demonstration are segregated from GPL releases i.e. engine releases, B. The engine development community is kept appraised of the academic situation, C. The developer signs a written contract to release the source following academic examination and D. A tentative timetable for release is established. This means that all known completed AI are eventually subject to GPL, while satisfying the requirements which an academic institution might impose in the interim. We might still lose some academic AI development, but this will undercut that and make the situation clear for the future.

Thoughts/Feelings on this policy proposal?

===
aegis wrote:"make -j" definitely works on windows/mingw32

I'm willing to give dev(s) ssh access to a snow leopard box running 24/7 as long as it's up (I'm moving in the next couple of months) with a full build environment, and help them setup a script to run constantly for builds.

my internet connection can be intermittent... streaming video can make it laggy for everyone connected... after I'm moved I can setup a more permanent box with a better internet connection if people are still interested.

we could also look into cross-compiling for osx again http://devs.openttd.org/~truebrain/comp ... arwin9.txt
This all seems very solid to me.
0 x

Tobi
Spring Developer
Posts: 4598
Joined: 01 Jun 2005, 11:36

Re: Dev meeting minutes 2010-05-23

Post by Tobi » 24 May 2010, 22:19

@AF I didn't even know about this exemption before this thread.. no wonder I didn't question it.

I don't recall NTAI being closed source (it was open source already when I joined Spring, since one of first things I did was port it to Linux) or seeing other AIs clearly being closed source, while distributing binaries.

EDIT: ok, correction, apparently I did know at some point that KAI was closed source, but decided for some reason not to make an issue of it. I don't know why and it might have been better to make an issue about that back then. This time I decided to discuss it with other developers, the rest is this thread.

@Neddie Something like that sounds reasonable yeah, although I'm not sure how practical it is.
0 x

Post Reply

Return to “Meeting Minutes”