BA SVN testing Autohost: Thorium

[Beherith]

Thanks to det, we can always test the latest SVN of BA by joining the Thorium autohost with a Rapid capable client like TASClient.

Happy testing!

[klapmongool]

Nice! Does it run on 87 then? Or newer engine versions like the springies?

[Beherith]

It runs on proper .87

For the record, I do not think each game using a different, non-release version of the engine can lead to anything good. If all games adopt the 'only use 1 specific version until we find one that is better for us' will make sure new bugs in new versions wont ever get discovered, and that we wont have anything even as remotely stable as .87.

It puts in a huge delay into the bug fixing feedback loop. All that is needed is to test more. A group of 10 people testing a proper large game WILL find the bugs, and they can be reported and fixed before they go live.

[dansan]

You can also use SL with every OS if you install the rapid tool from https://github.com/tvo/rapid/ .

[very_bad_soldier]

All that is needed is to test more. A group of 10 people testing a proper large game WILL find the bugs, and they can be reported and fixed before they go live.

I agree, in theory.
But isnt that how it is supposed to work since quite a while already? The last release proved again that for the one or the other reason it is not going to work that way. And thats why I think its time to look for other ways of doing it.

I vote for a way to have BA stick to a specific engine version that is known to work good enough. The BA devs then can decide to migrate to another version they have tested at a point in time THEY chose and find it worthwhile.

[klapmongool]

All that is needed is to test more. A group of 10 people testing a proper large game WILL find the bugs, and they can be reported and fixed before they go live.

I agree, in theory.
But isnt that how it is supposed to work since quite a while already? The last release proved again that for the one or the other reason it is not going to work that way. And thats why I think its time to look for other ways of doing it.

I vote for a way to have BA stick to a specific engine version that is known to work good enough. The BA devs then can decide to migrate to another version they have tested at a point in time THEY chose and find it worthwhile.

That would create a situation in which the game list in the lobby has 10 different mods running on 10 different engine versions.

It makes more sense to have all mods just run, by default, on the official engine release. Gentlemen's agreement. Using the new/upcoming possibilities to run multiple engine versions from one lobby without having to do compiling and stuff (which players don't do obviously) it would then be A LOT easier to get test games of current release mod on non-release engine versions going.

[dansan]

I was just beginning to assemble a team of dedicated BA-testers. 8)

That team should be able to quickly run testing-versions of the engine with current or dev-versions of BA. When the times comes for a new engine or BA release it would be ready to play test-games and report.

IMO it is a timing issue: If the engine-devs announce that they are nearing a new release, they must create an rc-tag in git, that we can use to test the games (BA in our case). The engine-devs may set a deadline of 1 or 2 weeks until the rc goes release -> live.

That time should be enough to find the most annoying stuff, so that a new BA release can happen _before_ the engine release! Minor issues will prob. still be found later and produce another BA version, but that shouldn't be such a problem.

What's the difference to before? To have a _dedicated_ testing-team that feels responsible (social pressure ;).

Requirements for test-group are just two:
* being able to use rapid for BA-test and
* using a engine-dev-version. Win users can use the buildbots binaries, mac and linux users must compile from git.

MT testing is optional though desired.

[Licho]

There is no hassle for zero-k lobby users regarding different engine or mod versions.

They just join the battle and it downloads engine/mod same way it does map.

Other lobbies could in theory work the same way.

1time 20MB extra download is surely less evil than having semi broken game for weeks.

[klapmongool]

There is no hassle for zero-k lobby users regarding different engine or mod versions.

They just join the battle and it downloads engine/mod same way it does map.

Other lobbies could in theory work the same way.

1time 20MB extra download is surely less evil than having semi broken game for weeks.

20MB is not evil at all. But I don't know about the technical details or anything about linux; that is between you/koshi/devs/whoever.

This part is important though:

It makes more sense to have all mods just run, by default, on the official engine release.

Instead of using the functionality in zk lobby for temporarily fixing a engine/mod problem it would be an altogether better situation to use it for proper SVN-release testing (in combination with mod). That way bad releases (86?) don't need to be released and at the same time mods can be fixed in advance of release.

Was ZK tested by a larger group of players on the 87 release candidate prior to its release? Why not? Or was it and did devs ignore results?

[luckywaldo7]

Was ZK tested by a larger group of players on the 87 release candidate prior to its release? Why not? Or was it and did devs ignore results?

It was not, because:

1) It is a hassle for linux users, as they need to compile the dev version instead of just downloading an exe. That is the particular detail that is going down between the lobby devs.

2) No one knew that anything related to area commands was going to change and end up broken. So no one knew that testing needed to be a priority.

It makes more sense to have all mods just run, by default, on the official engine release. Gentlemen's agreement. Using the new/upcoming possibilities to run multiple engine versions from one lobby without having to do compiling and stuff (which players don't do obviously) it would then be A LOT easier to get test games of current release mod on non-release engine versions going.

I like to think this is basically how it will turn out. Currently was switched to the dev version just because 87 is so unplayable. (FWIW, I run linux, and compiling was the preferred choice for me, over playing 87)

[Licho]

Btw we also run autohosts that auto use latest dev branch build of engine and we do tests there, but you cannot really do mass scale testing without changing the official server.

At best you can hope for some 1v1 that usually wont reveal more rare issues (like chicken crashing or nanoframe targetting).

[klapmongool]

Was ZK tested by a larger group of players on the 87 release candidate prior to its release? Why not? Or was it and did devs ignore results?

It was not, because:

1) It is a hassle for linux users, as they need to compile the dev version instead of just downloading an exe. That is the particular detail that is going down between the lobby devs.

2) No one knew that anything related to area commands was going to change and end up broken. So no one knew that testing needed to be a priority.

1) that isn't stopping ZK from doing it now, is it?

2) That is kinda the purpose of testing; finding unknown bugs.

Btw we also run autohosts that auto use latest dev branch build of engine and we do tests there, but you cannot really do mass scale testing without changing the official server.

Why? Basically ZK users are testing an unreleased build right now. What am I missing here?

[Licho]

Users are actually enjoying more stable build than other mods ..

There is nothing "sacred" about official release, its just one build marked to be released .. ZKL is not constrained by it so we can pick better version.

Players ar enot testing it, we tested build and then deployed it for players we dont change it, its same version now unless we find some new bugs. We are not using every engine build this way, just pick the one that works best if official one is broken.

[klapmongool]

Users are actually enjoying more stable build than other mods ..

There is nothing "sacred" about official release, its just one build marked to be released .. ZKL is not constrained by it so we can pick better version.

Players ar enot testing it, we tested build and then deployed it for players we dont change it, its same version now unless we find some new bugs. We are not using every engine build this way, just pick the one that works best if official one is broken.

Yes, now scroll back up and read my post again...