Pxtl wrote:Smoth, "release early, release often" doesn't mean release buggy, it means write a piece of code, test it, and release it, rather than code-code-code-code-code-test-test-test-test-test release.
Yeah, my concern though is that arbitrary dates being set where the dev team is committed to releasing every so many months WILL result in a buggy release. While I do think it is good to release a build where people can test things, some of the changes in this build and the previous were major re-factors if I recall right.
I think that spring if it had a more dedicated group of testers would have been released 2-3 months ago but they didn't have such a group. Which is another reason why I disagree. I don't think we need arbitrary dates as much as spring needs more testing.
The release early and often conversation has been had before I just didn't want to talk about it with argh, he always turns it into some circus and essentially will attack me rather than debate. Whether or not I may come through with a good point.
perhaps we should focus more on working in branches and only merging features into trunk once they've been tested so that we have greater security in any fast releases we make and we don't have to take extra time stabilizing trunk before a release.
Basically trunk would be the bug fix branch and stuff would only be merged in once its been tested and verified rather than stuff just being pushed onto trunk randomly before a release and then releases occurring when we're satisfied that it just happens to be stable.
We should also eb more willing to make bug fix releases. 0.76b2 was released and had a handful of bugs which were fixed in svn within the following 3-4 days yet no bug fix release was made despite intentions to do so being voiced by developers. There should of been a 0.76b3.
I also think we should change our naming scheme from 0.77b2 to 0.772 as the beta part is not exactly inviting.
I havent read all the answers to the initial post, but i wanted to thank Argh very much for his efforts. I totally agree with most of the points mentioned.
We should definitly release more often in the future, every 2-3 months seems to be a reasonable period for me
That is the thing AF, in the past they were always good about bug fix releases. I remember a few versions that had a bugfix within the the release. Don't get me wrong, I am not against feature releases or bug fix releases. My big thing is that we have a test build made nightly that people can test.
if we had a dedicated test team that just played on the nightly builds would that not get the needed feedback to expedite the releases?
Hmm, it would be interesting if those of us testing our projects used the nightly builds for test games. Doing something like that would be great for spring and help keep the projects on the cutting edge. Let's face it #main is full of retards and we could actually get meaningful conversation/testing done by being away from the noobs.
Argh wrote:Lastly, this debate is long overdue. Your statement that this is just some words, when you couldn't find the time for action, is duly noted.
I do not like debating with you. I spent two days telling you that the team color was an overlay, you made it long and draw out tedium and didn't even give me the courtesy of saying I was correct. Why should I debate with you? You make it a long drawn out process that is tedious and not at all to the point. Your posts are unnecessarily long and chew up much of my time. I still remember the thread where you tried to argue with me that pink was red and told me I was doing it wrong.
To af
well, we have s44, IW, gundam and pure. Each group could start testers and insist that all testing be done on the test build servers. Maybe using a weekly build selected and posted in an official thread.
Something like:
[thread] week of (10/5/08) Testing thread
Current build to use is: link
Server is: IP
Current projects testing on this build:
X
[/thread]
*edits*
Also peet had an interesting suggestion. Maybe an exp bonus for playing on the test server. Something epeen usually motivates people. Hell what if they gave testers a title on the forum. They keep the title as long as they test once a month.
I think the trick is finding a way to give people a reason to join the test server that might not even be a stable game.
I agree with the "release early, release often" development mentality.
You guys missed out on the largest testing assets available to you, the entire spring community player base.
It also covers a much wider range of people with different hardware specs, so you can make the game functional for as many users as possible.
They (including myself) might not be as pinpoint accurate and as objective as you guys are with your testing, but having the hundreds of players that play on the spring engine daily is where you will reveal most of the bugs quickly and in scenarios that you might otherwise never have tested for.
Pointing fingers at who didn't do this, or wasn't around for that doesn't solve anything either.
It doesn't hurt to ask for help from the community either, if you guys are short on testers, make a post on the main website home page and recruit some community members with some news posts instead of saying how short of testers you are.
If you start asking some of the community for help publicly, you may surprise yourself at how many will rally to do it.
Frequent releases accomplishes alot things, it makes the workload much less overall, keeps the community confident when they see things are actually getting done, allows for quick patches for features and small bugs, keeps the work flow constant, but mostly its to keep the community happy.
I see the same thing from all over the internet with development teams, Mod teams have their own lives to worry about for sure, but if you keep your community waiting for month after month with nothing to show for it, alot of people lose interest and start going off in different directions.
Thats just how the gaming market and community is.
If you keep the community happy and interested, people will flock to it, if it sits and stagnates, people will leave and just plain give up.
You guys gotta remember that there is a much broader view here, and if the spring is to grow and bring in new people who are excited to play it, and excited about contributing to it, you have to keep things short and sweet to show everyone that "hey, we're actually doing something here"
1. I don't think that "dedicated testing teams" are a realistic thing at this time. We just don't have the numbers to support it. I had a huge amount of trouble just getting people into test sessions on the test server enough to find major engine bugs... and that was with a novel game on offer as a reward, and the motivation that comes with being part of a semi-elite "beta crew". Too few people, too many timezones, basically.
2. I agree with KDR's point that the current infrastructure is inadequate. That's where the headless server comes in, I would think. We can encourage people to run their own servers, if they want to stay really conservative and advertise that "Spring is unstable, we're not"... and they can cherry-pick the most stable releases, and not update unless they're really sure that the engine's polished.
Or am I off base there? I think that's an opportunity waiting to happen, myself, if presented as a way for server operators to gain some notoriety whilst keeping the playerbase happy. If a given release is a real dog, then people will shift for a bit, then come back when they know that the main release is fixed up again.
3. I see 2-4 month cycles as feature builds, yes. We need to "train" the playerbase to see bugfix releases for the month following a feature release as a normal part of the process, not as a giant catastrophe. We shouldn't have months go by, where small bugfixes are already done, but not available- I think that hurts players a lot worse, than the minor inconvenience of installing a new version of the software, basically.
4. The goal is not to push developers into prematurely releasing something... the ultimate responsibility resides with the Release Manager, and Tobi / Auschwabar did that job correctly, imo.
The goal is to change the way that contributors see the timeframe on their releases- i.e., if you're going to write a major feature, great, get it done early in the cycle (like trepan's big commits during the early part)... if it's late in the cycle, hold off, unless you're pretty darn sure it's bulletproof. This release was unusually full of very large refactoring stuff, as Smoth pointed out elsewhere- big, nasty dirty jobs, that involved very large amounts of the total source.
This isn't always avoidable, but approaches like Git might really help keep it more manageable- giant merges are very likely to cause giant problems, so they should be timed to periods of "new feature" cycles. From my limited understanding of how Git works, it would probably really help with that.
good idea, that with the points/time boost when playing on the test server. .. some reward of any kind.
it would be the job of the lobbies to prevent confusion when the lobby server supports multiple versions. they could eg have a filter on by default, that only shows the installed versions, or greys out the otherones or something.
smoth wrote:I don't know if the players will shift a bit or outright leave.
If the server allows several versions of spring I forsee a lot of confusion for players.
I agree. We shouldn't take for granted that releases are going to be crappy. If they're buggy, they should be fixed, not released.
Of course, this is coming from a guy who's not a developer, but it's also the point of view of your target audience. I'll tell you right now that I think multiple versions of something are nothing but huge PAIN IN THE ASS, they are both annoying and confusing. Players won't know which version they should be on, and they would hate to be switching all the time. I know that I'd be very fed up if there were multiple versions going.
noobish thought : could an lobby connect you to games using different versions of spring?
that way mods could be "tied" to different versions, i.e. if you had logged on the other day you would have had the option to play mods that worked on .76, and if you wanted to try a mod that required the new version of spring, it would give you a warning message (you require a new version of spring to play this mod 0.77b2 : download now?)
then you could have different versions all lurking on your computer, and a different version could be called up as need be... possibly displaying the spring version # as one of the criteria in the room info along with host name, map name, mod name, etc
as far as testing new versions goes, if players went to join a game of a mod that was using a new release or testing version of spring, they would be aware that it's a different version of spring (if a certain version number didn't work on your computer, you could delete it, and avoid games of that version ever after)
just musing here; but it seems like it might solve the problems of needing more testers for new versions, players being unable to run newly released versions, etc etc.
Pxtl wrote:Smoth, "release early, release often" doesn't mean release buggy, it means write a piece of code, test it, and release it, rather than code-code-code-code-code-test-test-test-test-test release.
Yeah, my concern though is that arbitrary dates being set where the dev team is committed to releasing every so many months WILL result in a buggy release. While I do think it is good to release a build where people can test things, some of the changes in this build and the previous were major re-factors if I recall right.
This is the value of git - developers who want to stabilize a release can do that, and developers who don't care can keep working on their experimental changes without pushing them into the rest of the tree.
Tobi wrote:Feature branches have too much overhead in SVN. When/if we switch to git it would definitely be a good policy.
Git should be a priority, honestly. It did so much to speed Wine development.
I don't think allowing multiple spring versions on the lobby would be useful. This would split the players community which is not that big in our current single version setup.
Getting serious test session is not that hard, it only requires advertisement and organization.
I say make development model closer to that of SpringLobby , albeit with differences coz spring got sync stuff.
We use Git, not SVN . New breaking features are developed in branches. The master is regularly merged into them or re-basing is done. Branch is merged into buildbot master when branch is working. We still get occasional issues with things getting broken but not very much and most often crashbugs are fixed rather quickly.
Branches could be clearly named differently for branches that break sync and that do not. There should never be a branch with both synced and unsynced fixes if those are separable.
Multiple spring versions on lobby would be very useful for testing. THe fact that the infrastructure would allows it doesn't mean everyone should be running 10 different Spring versions and switch all the time, we could still push out updates to everyone and have server deny you access with too old versions. Having the possibility however to run with multiple Spring versions on same lobby server could streamline testing by a huge amount IMO.
