New Release Plan

New Release Plan

Discuss the source code and development of Spring Engine in general from a technical point of view. Patches go here too.

Moderator: Moderators

el_matarife
Posts: 933
Joined: 27 Feb 2006, 02:04

New Release Plan

Post by el_matarife »

Yokozar and I have been discussing what we think a "best practices" release plan for Spring would be, taking into account the lessons of Ubuntu, Wine, and other open source projects.

The first item is the current release; it has known bugs and clearly we need to go ahead and retire it by coming out with something new. We're suggesting a feature freeze in the next month, with the proposed release broken off the trunk for playtesting and bugfixing. It should be much easier to get people to playtest once we know there's going to be an actual release coming, and we could setup a "playtest schedule" once we've done the feature freeze.

Once we've made this new release, I'm proposing we put on the schedule four post-release patches. I know four patches sounds like too much, but the .79 release has gone through five releases already, and there's still a few bugs that never got fixed. If we don't need the releases, we can skip them, but if they're on the schedule there will be a lot more momentum towards actually releasing them and not just getting lazy and deciding the last few bugs aren't worth fixing.

The goal is to keep the released Spring version completely free of known bugs. We've had too many Spring releases over the past few years that sat with known bugs for months without getting fixed, which is just a toxic thing to do in terms of attracting new players, new mods, and new developers.

Finally, I think a commitment to a release every six months would do a lot for both planning and release quality. The current schedule leaves developers uncertain as to when the next version will ship, so there's more pressure to get new features into the upcoming release even if they aren't ready yet. A six month schedule would let us say "hey if X feature isn't ready, it isn't ready and it isn't a big deal because you can just get it in the next release".
YokoZar
Posts: 883
Joined: 15 Jul 2007, 22:02

Re: New Release Plan

Post by YokoZar »

The proposal is what other OSS projects with good release management do:
  • Define a feature freeze date.
  • Release a beta shortly afterwards, and focus on fixing the regressions
  • Plan on making a real release within a month of the feature freeze
  • Start the cycle again
The main idea is simple - you base the release around stopping regressions while releasing on time.

If you do this quickly enough, people don't mind if their feature slips into the next release, since that's going to come out fairly soon. Some projects do this monthly, others twice a year.

Since the beta is feature-complete and within a month of the real release, users don't mind testing as much.


It also helps to have a release manager - the lead developer is usually a good choice, but it doesn't have to be him, especially if he has his own pet features.
YokoZar
Posts: 883
Joined: 15 Jul 2007, 22:02

Re: New Release Plan

Post by YokoZar »

As Dan Kegel (Wine 1.0's release manager) put it: the job of the release manager is only to make sure we don't ship a release with embarrassing bugs.

That means doing things like making a list of release target bugs and making sure it includes all the regressions. You don't even have to be a developer to do that (though it does help you understand the time involved).


I'll note that before Dan came along, Wine went 15 years without ever making a real release. The reason is that the lead developer was essentially doing the "I'll release when I have enough features done" model - there was always something nice that could go in next. I think Spring's been in a similar position.
el_matarife
Posts: 933
Joined: 27 Feb 2006, 02:04

Re: New Release Plan

Post by el_matarife »

And as for leaving releases unpatched, I feel like I need to say a few more words beyond the mega-huge wall of text I unleashed earlier.

Leaving a release with known bugs publicly available is one of the worst mistakes we are making as a community. There's only one chance to make a first impression, and if people's first impression of Spring is "oh I tried that once and it was totally buggy. Some of the features didn't work and it was just clunky and poor quality in general" we've lost a user we may never get back, much less anyone that user may have talked to about Spring. It also discourages our mod developers, who don't see a point in building a new release around a buggy product.
Auswaschbar
Spring Developer
Posts: 1254
Joined: 24 Jun 2007, 08:34

Re: New Release Plan

Post by Auswaschbar »

el_matarife wrote:The first item is the current release; it has known bugs and clearly we need to go ahead and retire it by coming out with something new.
Thanks for pointing this out.
el_matarife wrote:We're suggesting a feature freeze in the next month, with the proposed release broken off the trunk for playtesting and bugfixing.
If we stick to the 3 / 4-month release cycle, and with the fact there are already huge changes in trunk, it should be happen in the near future, agreed. However, breaking off trunk (for the major release) is a bad idea: I tried it with 0.78, it didn't work.
el_matarife wrote:It should be much easier to get people to playtest once we know there's going to be an actual release coming, and we could setup a "playtest schedule" once we've done the feature freeze.
There even was a release candidate for 0.79 up for a week before the release. But the only people who found bugs were by people who test the devel version anyway.
el_matarife wrote:Once we've made this new release, I'm proposing we put on the schedule four post-release patches. I know four patches sounds like too much, but the .79 release has gone through five releases already, and there's still a few bugs that never got fixed. If we don't need the releases, we can skip them, but if they're on the schedule there will be a lot more momentum towards actually releasing them and not just getting lazy and deciding the last few bugs aren't worth fixing.
Throwing arbitrary numbers around isn't really helpful. It depends on the amount and seriousness of the known bugs if there is a new release or not. Reason is that we need time to find, identify and fix bugs.Making releases to earl means that maybe some bugs weren't fixed at this time, requiring a new bugfix shortly after. This might be acceptable with unsynced releases. But synced releases take some time. Packages had to be made etc.
el_matarife wrote:The goal is to keep the released Spring version completely free of known bugs. We've had too many Spring releases over the past few years that sat with known bugs for months without getting fixed, which is just a toxic thing to do in terms of attracting new players, new mods, and new developers.
Sorry to say that, but this is just not going to happen.
el_matarife wrote:Finally, I think a commitment to a release every six months would do a lot for both planning and release quality. The current schedule leaves developers uncertain as to when the next version will ship, so there's more pressure to get new features into the upcoming release even if they aren't ready yet. A six month schedule would let us say "hey if X feature isn't ready, it isn't ready and it isn't a big deal because you can just get it in the next release".
The longer a release took, the more bugs it had. 0.77 took almost a year, and it was nearly unplayable in the first versions. So a release cycle of around 3 or 4 months is a good compromise, it worked out good imho for the last 2 releases.
Auswaschbar
Spring Developer
Posts: 1254
Joined: 24 Jun 2007, 08:34

Re: New Release Plan

Post by Auswaschbar »

Spring release works like this:
  • there is a announcement that a release is planned, so that people concentrate more on bugs and less on new features (feature freeze has proven worthless, as nobody adheres to it anyway)
  • some bugs where fixed
  • there is a release candidate / public forum announcement / etc. with the request to test it
  • except the usual suspects, and maybe 2 or 3 motivated modders, nobody will actually test it
  • we force a version down everyones throat, lots of bugs discovered, rage and shitstorm all over
  • we can fix huge amounts of bugs
  • bugfix release, normality restored
YokoZar
Posts: 883
Joined: 15 Jul 2007, 22:02

Re: New Release Plan

Post by YokoZar »

Auswaschbar wrote:Spring release works like this:
  • there is a announcement that a release is planned, so that people concentrate more on bugs and less on new features (feature freeze has proven worthless, as nobody adheres to it anyway)
This is the purpose of having a release manager. It's also why it's essential that members of a project pick the release manager themselves rather than having him imposed - since, at some level, he's saying no to them ;)
  • some bugs where fixed
  • there is a release candidate / public forum announcement / etc. with the request to test it
  • except the usual suspects, and maybe 2 or 3 motivated modders, nobody will actually test it
This is a problem, of course, though I suspect one reason we lack beta testers is because it's unclear when that next release will actually come. There was an attempt to organize a few official beta test games last release, but I don't think anyone followed through.

If the release manager declared an official beta game day and invited everyone to play with him, that might make the difference.
  • we force a version down everyones throat, lots of bugs discovered, rage and shitstorm all over
  • we can fix huge amounts of bugs
  • bugfix release, normality restored
Yeah, this is what I'm hoping we can avoid. You have been very good about working like mad to restore normality though ;)
Auswaschbar
Spring Developer
Posts: 1254
Joined: 24 Jun 2007, 08:34

Re: New Release Plan

Post by Auswaschbar »

You also seem to miss there is a big difference between a wine release and a sprign release:

The wine guys only have to upload a tarball somewhere and call it the next release (ideally spoken). People will then update, or won't, will wait until their distro made a tarball or not.

For a spring major release, you have to bump the server version, which means that everyone who didn't update cannot play the game anymore.
Auswaschbar
Spring Developer
Posts: 1254
Joined: 24 Jun 2007, 08:34

Re: New Release Plan

Post by Auswaschbar »

YokoZar wrote:Yeah, this is what I'm hoping we can avoid. You have been very good about working like mad to restore normality though ;)
This is only because of the amount of feedback we get then. We could do the same before the release if there would be a decent amount of testing.
el_matarife
Posts: 933
Joined: 27 Feb 2006, 02:04

Re: New Release Plan

Post by el_matarife »

Please don't take this thread as an attack, I know I worded it aggressively but it seems like that's the only way to get stuff done around here sometimes. I'm not putting the blame of this on anyone in particular, because it isn't an anyone's fault. We have a broken process and a broken culture which forces us into making broken releases and driving off players.

I know there's no way the initial release can be "zero defects" but we should at least make a commitment to keep patching the release until we hit as close to that mark as possible. When you say normality restores itself, you're really talking about Spring going back into its normally rough and low level buggy state. I mean, look at the kind of defects we've decided are "good enough" the last few years: ground decals causing crashes, sonar causing crashes and desyncs, reclaim and resurrect broken, and I'm sure there's more I'm forgetting. Some of these bugs lingered for months and months. Is it really a good idea to leave releases like that?
Auswaschbar wrote:For a spring major release, you have to bump the server version, which means that everyone who didn't update cannot play the game anymore.
A twenty megabyte download isn't really a big barrier to adoption. I know it has to be irritating to keep reinstalling Spring, but I'd rather deal with that than known bugs. I also want to investigate the possibility of making "Update releases" essentially consisting of the changed files in a zip file or installer so that you don't have to jump a lot of hoops just to get a new EXE and a couple DLLs.
Auswaschbar
Spring Developer
Posts: 1254
Joined: 24 Jun 2007, 08:34

Re: New Release Plan

Post by Auswaschbar »

Bugs don't fix themself. And not all bugs are that simple to find, lets take the ground decals for example. I didn't cound how much commits there were saying that ground decals are fixed, neither how much time it took (and I'm sure baczek et al. didn't have a fun time with this). At such a point, we could either stop everything, or just proceed as normal, and ignore the bug for some releases. The second option is clearly the better imho.
YokoZar wrote:This is the purpose of having a release manager. It's also why it's essential that members of a project pick the release manager themselves rather than having him imposed - since, at some level, he's saying no to them ;)
Me: there was no release in the last 7 months, we really should make another one
Tobi: agreed, but I don't have time. You wanna do it?
Me: k
Tobi: good
Auswaschbar
Spring Developer
Posts: 1254
Joined: 24 Jun 2007, 08:34

Re: New Release Plan

Post by Auswaschbar »

el_matarife wrote:A twenty megabyte download isn't really a big barrier to adoption. I know it has to be irritating to keep reinstalling Spring, but I'd rather deal with that than known bugs. I also want to investigate the possibility of making "Update releases" essentially consisting of the changed files in a zip file or installer so that you don't have to jump a lot of hoops just to get a new EXE and a couple DLLs.
And every linux user just start compiling spring for themself?
el_matarife
Posts: 933
Joined: 27 Feb 2006, 02:04

Re: New Release Plan

Post by el_matarife »

Auswaschbar wrote:And every linux user just start compiling spring for themself?
Yokozar builds a DEB for Ubuntu / Debian people, I don't know who can build an RPM but the people using RedHat derivatives can always use Alien, and I'm not sure what to do about the people who really insist on building their own.
User avatar
koshi
Lobby Developer
Posts: 1059
Joined: 14 Aug 2007, 16:15

Re: New Release Plan

Post by koshi »

pushing new releases to OpenSuse Buildservice / Launchpad can be automated. For SpringLobby we already do that for OpenSuse Buildservice from our release buildbot:
http://springlobby.info/repositories/en ... ate-rpm.sh
User avatar
hoijui
Former Engine Dev
Posts: 4344
Joined: 22 Sep 2007, 09:51

Re: New Release Plan

Post by hoijui »

soo..
1. it would be good if we have auto generated linux packages for some distributions (ubuntu, opensuse, gentoo.. maybe debian?)
<- someone has to make it
2. we need more testers

the rest is fine i think, as Aus siad, he there are release candidates and testing going on before a release, and it is genreally all fine except the lack of testers. We dont need a whole new system/policy and such.
one of the main problems i see is bibim beeing the only one that really chan wokr on buildbot, as he seems to not have much time recently, and there are some things that would be nice to have there (eg. the auto linux package building). It would be good if the buildbot source could be worked on by more then one, and also the running buildbot should be maintainable by more then one.
el_matarife
Posts: 933
Joined: 27 Feb 2006, 02:04

Re: New Release Plan

Post by el_matarife »

Buildbot source should be added to the GIT repository if it isn't already. Bibim could also ask for some administrative help, I'm guessing we've probably got a few gurus around who would pitch in if given root access.
User avatar
koshi
Lobby Developer
Posts: 1059
Joined: 14 Aug 2007, 16:15

Re: New Release Plan

Post by koshi »

hoijui wrote:It would be good if the buildbot source could be worked on by more then one, and also the running buildbot should be maintainable by more then one.
Agreed, but isn't he running some very custom (perl?) buildbot on his own box?
If he, understandably, does not want to share access to that box, setting up a limited buildbot (as in http://buildbot.net) to do just linux pacakge stuff would be easy enough. I would even consider doing that on my box if it's really wanted.
User avatar
hoijui
Former Engine Dev
Posts: 4344
Joined: 22 Sep 2007, 09:51

Re: New Release Plan

Post by hoijui »

koshi:
good points, good idea!
+1 from me!

.. i can not really supply anything (not a box at least) i think, but if i can do something.. only thing i can think of is testing ubuntu packages it generates (while initially setting it up).
User avatar
clericvash
Posts: 1394
Joined: 05 Oct 2004, 01:05

Re: New Release Plan

Post by clericvash »

Auto generation of the linux packages would be a great idea and that i would support.

As for the release schedules im with the OP on this, i think they have laid out some really good ideas, the feature freeze is a good idea and many many OS projects use it.
User avatar
hoijui
Former Engine Dev
Posts: 4344
Joined: 22 Sep 2007, 09:51

Re: New Release Plan

Post by hoijui »

clericvash wrote:As for the release schedules im with the OP on this, i think they have laid out some really good ideas, the feature freeze is a good idea and many many OS projects use it.
As Auswaschbar said already, he tried it, and it does not work.

It does work wor big changes. For example, luacob, GML, or the new AI interface are thigns that will not be integrated shortly before a release, but for all the small changes, it would just be much more work for devs to sort them out in separate branches just becuase we are in prerelease time. eg, often you have fixes and small feature additions in the smae file, and you just do them in one run.. the nyou have to commit and you had to copy the file, undo all feature add changes, make sure the file still compiles, then commit to the realse candidate branch, ...
never going to happen.
Post Reply

Return to “Engine”