Dev meeting minutes Monday, 1. August, 2011

Dev meeting minutes Monday, 1. August, 2011

Minutes of the meetings between Spring developers are archived here.
Post Reply
Tobi
Spring Developer
Posts: 4598
Joined: 01 Jun 2005, 11:36

Dev meeting minutes Monday, 1. August, 2011

Post by Tobi »

Date: 1-8-2011
Present: Kloot, abma, zerver, jK, Tobi

_Agenda____________________________________________________
  • Buildbot: some storage reduction would be good, some options are:
    • reduce size per build
    • reduce days non-release builds are kept (now 90 days)
    • reduce rate at which builds come in
  • Anything against SIGNAL_NANS:BOOL=1 as default debug|debug2|debug3 build?
  • Include http://code.google.com/p/bitwise-enum/ ?
  • Release plan
  • Release Candidate?
  • Next meeting
  • Anything else? (WVTTK)
_Meeting___________________________________________________

Buildbot: some storage reduction

<abma> i've to leave in 10 minutes :-/
<abma> about the buildbot: if a change is really required i would only reduce the days the files are kept
<abma> because: its less work, and it still allows easy regression-testing without rebuilding
<Tobi> yes
<Tobi> I wonder, how often do you (everyone here) use builds up to 90 days old?
<abma> abou SIGNAL_NANS:BOOL=1: why isn't it enabled as default for debug builds?
<abma> only one time ^^
<Tobi> ok
<abma> but afaik i didn't used the oldest build
<abma> and i think bisecting in linux should work in most cases
<Tobi> hmm I suppose I could also correlate the builds with the apache logs
<Tobi> but that's more work :)
<abma> :D
<zerver> occasionally when bisecting, you have to dig up really old stuff
<abma> yep, thats why recompile on linux is often better
<abma> maybe the build is missing that you want on the buildbot
<abma> so reducing the time builds are kept is imo the best solution
<Tobi> you can always trigger new builds of course
<abma> yes, but this takes some time :-)
<Tobi> ah I see someone else has been making space on the server too
<Tobi> true that
<jK> in theory you could just keep 1 build for each day/week
<zerver> if old builds need to be cleaned away, maybe just keep the default exe?
<jK> but writing that bash script is ... work
<Tobi> yeah but that's more work indeed :-)
<zerver> or keep one per week yeah
<Tobi> anyway it's much less urgent now, there is 50G already free on server (instead of 16G)
<zerver> mostly the debug symbols that need to be cleaned away...
<jK> true
<Tobi> yeah, cleaning certain aspects earlier than other aspects could be done too
<jK> debug symbols could be deleted after 2weeks
<abma> -1 this more work :)
<jK> more work?
<Tobi> it's easier than the keep one build per day/week option :P
<jK> just deleting debugsymbols is quite easy
<abma> yes, someone needs to script that
<zerver> sounds really easy imo
<Tobi> yeah I can do that in a min if no one objects
<jK> +1
<abma> hmm
<abma> ok, true
<abma> then, what to keep?
<abma> remove all *_dbg.7z older than time x? (but still keep them for release builds)
<Tobi> debug symbols are two third of the size
<Tobi> I don't touch release builds in auto-cleanup anyway
<abma> ok
<abma> sorry, i've to leave
<abma> cu! :)
<zerver> bye
<Tobi> ok, we'll do this :)
[21:50] ** abma_irc left the channel( Quit: Leaving. ).
<Tobi> can be next point then

conclusion:
  • We'll purge debug symbols for non-release builds a bit earlier (14 days instead of 90 days)
Anything against SIGNAL_NANS:BOOL=1 as default debug|debug2|debug3 build?

<jK> == anything against SIGNAL_NANS:BOOL=1 as default debug|debug2|debug3 build? ==
<zerver> wasn't it enabled some time ago even in release builds?
<jK> yeah, but it failed cause of Lua
<zerver> Ah, I remember coding something to disable it for MT
<jK> and the anti div0 commit still needs to be reverte
<jK> *reverted
<jK> +in Lua
<zerver> the MT rendering is a no-go with SNANs, it reads way too many partially updated unit parameters and such
<jK> o_O
<Kloot> I just solved about a billion div0's in the engine alone, dont even want to guess how many more there are in lua code that trickle in through the API
<zerver> good job :)
<Tobi> uh, why more partially updated unit parameters when signal nans is on?
<zerver> not more, only that they cause crash...
<jK> yeah I knot many lua devs are sloppy with div0 checks
<Tobi> yeah, nice work Kloot, ty :)
<jK> but div0 is already used _intentionally_ in lua
<jK> and yeah, nice work to find all the NANs in the engine :)
<Tobi> how is it used intentionally?
<Tobi> to acquire inf?
<jK> e.g. to diff +0 and -0
<zerver> SNANs in the rendering is mostly tolerated by the gfx drivers, and only cause minor visual glitches
<zerver> or NANs i mean
<jK> local x=-0; if (1/i < -math.inf) then Spring.Echo("x == -0") else Spring.Echo("x == +0") end
<jK> erm not "<", "=="
<Tobi> eww, so lua has a special case in all comparison operators to make -0 == +0?
<jK> yeah
<jK> it uses c++ comp
<jK> and in c++ it's the same
<Tobi> hmm, didn't know that
<zerver> so... keep SIGNAL_NANS optional then?
<Kloot> then all the lua functions that read from the stack (lua_checkfloat etc) need to made nan- and inf-proof
<Kloot> +be
<jK> nah, a default nan checks in Unit::SlowUpdate + warning message would already prevent 99% of all lua induced NANs
<Tobi> what is a use case for differentiating -0 and +0 btw?
<jK> in chili I use it for right aligment
<jK> (by default it is left aligned and for negative values (includeing zero) it is right aligned)
<jK> I knew I used it in another case too, but it's ages ago and only have vague idea how it was used
<Tobi> ah, ok
<Tobi> did you know tostring in Lua is stateful wrt -0?
<Tobi> > print(tostring(-0) .. tostring(0))
<Tobi> -0-0
<Tobi> > print(tostring(0) .. tostring(0))
<Tobi> 00
<Tobi> ewwwww
<zerver> weird
<jK> lol
<zerver> submit that as a bug
<zerver> we are talking debug builds here, maybe the signal nans do more good than harm in this case
<jK> hmmm in thoery it should be possible to disable SNAN in lua shouldn't it?
<Kloot> yeah, just omit the -f flag for lua sources
<jK> that should be easy
<Tobi> it's the compiler inserting a check after every (few) float instructions, and not the hardware exception?
<zerver> I thought it was hardware, fpu exception
<Tobi> it could be both, compiler inserting enable/disable exception insns, and then optimising them away within the compilation unit (just speculation)
<Kloot> hmm
<jK> from LuaHandle.cpp it looks like it alredy gets disabled
[22:28] ** Kloot left the channel( Ghosted ).
[22:28] ** Kloot joined the channel.
<zerver> reading feclearexcept docs makes me think it is a FPU exception and no other magic involved
<jK> I would say give it a try
<zerver> I agree it would be really good to have it on, so that kloot does not get div0 nightmares in conjunction with every release
<Kloot> engine stability uber alles at this point
<Kloot> fpe and quick hard crash >>>> nan's infecting the simulation and causing memory corruption/desync/whatever thousands of frames later
<jK> so +1 for default enabling, and +1 reverting div0 prevention in Lua & adding default NAN check in SlowUpdate even in non-debug builds
<jK> *default enabling in debugX
<zerver> Yeah, +1, although I have not enough insight in the lua issue to have an opinion there
<Kloot> also maybe each div0 in lua code should be logged by modifying the luaconf macros
<Kloot> cause I do _not_ want to have to dig into gadget code as well to trace them
<zerver> understandable, that is hell to debug
<zerver> just to find which gadget, and then, sigh, which code line in it...
<jK> In theory it can be done, but it would have to be disabled in release builds for performance reasons and then the benefit would be questionable
<Kloot> especially when the amount of gadget code in some games exceeds the engine in NLOC
<zerver> if you want to make software that is hard to debug, include some kind of interpreted language in it
<zerver> sometimes you want that... for software protections and such
<zerver> I'm AFK for 15 mins, regarding the next point, looks fine to me, although I'm unsure how much use we have for those bitwise ops in Spring

conclusion:
  • Enable SIGNAL_NANs in debug compiles, except in Lua
Include http://code.google.com/p/bitwise-enum/ ?

<jK> == include http://code.google.com/p/bitwise-enum/ ==
<jK> I wanted it to use in MoveMath.h
<jK> to make it typesafe
<jK> currently it has const static int BLOCK_MOVING = 1; etc.
<jK> because it needs bitops like BLOCK_STRUCTURE | BLOCK_MOBILE_BUSY
<jK> with that bitwise-enum template it's possible to make it typesafe and use enums
<jK> I wished boost would have such a thing, but I couldn't find anything like that
<jK> so anyone has objections or knows a diff implementation/way?
<Tobi> problem may be its GPL v3..
<jK> to be more specific it is "GNU Affero General Public License"
<Tobi> and the linked page says "OBSOLETE", so I guess we should look here instead : http://code.google.com/p/mili/wiki/BitwiseEnums
<jK> don't want to include whole MILI
<Tobi> yeah but maybe they have more up to date version of this package too?
<jK> doesn't look like, still yeah I would copy it from there
<jK> k in MiLi it is relicensed to GPL3
<Tobi> we'd have to bump whole spring to GPLv3 then
<jK> no, GPL3 should be GPL2 compatible
<jK> but GPL2 is not GPL3 compatible
<jK> so we are safe
<Tobi> yeah so 2 + 3 can not be in the same program
<jK> sure they can
<Tobi> "When we say that GPLv2 and GPLv3 are incompatible, it means there is no legal way to combine code under GPLv2 with code under GPLv3 in a single program. This is because both GPLv2 and GPLv3 are copyleft licenses: each of them says, ÔÇ£If you include code under this license in a larger program, the larger program must be under this license too.ÔÇØ There is no way to make them compatible. We could add a GPLv2-compatibility clause to GPLv3, but it wouldn't do the job, because GPLv2 would need a similar clause."
<Tobi> from http://www.gnu.org/licenses/rms-why-gplv3.html
<Tobi> since spring is v2 or later we can make spring v3 or later and then it is fine, I think
<jK> meh damn
<Tobi> yeah it sucks
<jK> I will write him an email then ...
<Tobi> ok
<Tobi> anyway technically I am +1, the goal seems good and I assume you've taken a look at the quality

conclusions:
  • Suggested library has incompatible license now
  • jK will drop the author an e-mail to see if we can arrange something
Anything else? (WVTTK)

[23:40] ** abma_irc joined the channel.
<abma> hey! :)
<Kloot> wb
<abma> did i miss anything?

Headles vs headless desyncs
<abma> any idea why headles vs headless desyncs?
<abma> imo this has to be some uninitialized var or some coding-bug...
<Kloot> let's hope the cause is as "simple" as that
<Kloot> even so, the desync location is code that is mostly the same in 82.7
<abma> ok...
<abma> let's see if some tests still bring some SIGFPE or so...
<abma> anything against making SIGNAL_NANS default in debug builds?
<abma> only bad thing is, that it will instantly quit, when such an error occurs, without backtrace or so...

Wrong GetActiveHandle called?
<abma> [LCC]jK: about https://github.com/spring/spring/commit ... 24352d0d6a (use correct function ...)
<abma> GetActiveHandle(); is called a few times more in that file
<abma> is it wrong there, too?
<abma> or was it only wrong in this place?
<jK> zerver overloaded GetActiveHandle
<jK> in the other places it expects the overloaded one
<abma> aah, ok
<jK> and if it is NULL there it really is a bug and the assert should catch it
<abma> thx
<jK> because none of that locations has a NULL check

Syncdebug output
<abma> tvo: i still don't understand the syncdebug output completely
<abma> ouch...ok
<abma> tvo: there are sometimes "holes" in the frames
<Tobi> this may be because the vars that were desynced weren't used in those frames
<abma> hmm.. ok in the latest syncdebug log frames seems to be continious
<Tobi> for example if something is desynced that is only updated in SlowUpdate, then it will appear only every 16 frames (until snowball effect kicks in ofc)
<abma> for example, frame: first frames wher 13123 and then the next frames where somewhere at 14700 or so
<Tobi> ah yeah with the first frame I do not totally trust it myself either
<abma> ok...
<abma> i took the frame from infolog.txt, thats fine?
<Tobi> it may be the case that, if two clients have different amount of assignments done, then maybe the last block in the log contains some instructions from before the history wrapped around on one of the clients
<Tobi> I don't remember if I filter that out properly in the code
<Tobi> abma: yeah should be ok
<Tobi> gn
abma
Spring Developer
Posts: 3798
Joined: 01 Jun 2009, 00:08

Re: Dev meeting minutes Monday, 1. August, 2011

Post by abma »

note: the desyncs seems to be fixed + spring seems also to sync with mac osx now.
User avatar
Tim Blokdijk
Posts: 1242
Joined: 29 May 2005, 11:18

Re: Dev meeting minutes Monday, 1. August, 2011

Post by Tim Blokdijk »

Just a little side note about GPL v.3, it has certain provisions against the use of DRM. DRM techniques might actually be useful at some point to prevent cheating. I never really studied GPL v.3 so this might not be an actual issue. And DRM failed to protect anything as far as I know. Still.. would be quite a laugh if we can't legally block cheating clients even when it's technically possible.
User avatar
Zydox
Lobby Developer
Posts: 453
Joined: 23 May 2006, 13:54

Re: Dev meeting minutes Monday, 1. August, 2011

Post by Zydox »

No new meeting minutes?
(missed the 15th, but that's still a few weeks ago...)
User avatar
aegis
Posts: 2456
Joined: 11 Jul 2007, 17:47

Re: Dev meeting minutes Monday, 1. August, 2011

Post by aegis »

big problems with DRM - clients can never cheat beyond info gathering, there are currently a multitude of ways to do that besides using a modified client, and the source is easy to compile.

maybe a separate binary anticheat software could work okay but that's still shenanigans :P
Post Reply

Return to “Meeting Minutes”