Support for multiple engine versions

[Jasper1984]

Allowing different binaries(launchable from the lobby) sounds like a good idea. Imo it should, for one, also be used to allow people to enter binary-test servers more easily. (I suppose apt-get can be modified to update a stable and a test version.) Release early is nice, but i do not see reason to expose the entire gaming community to all of the bugs that are left when only the developers test it. Most of these users will not be effective in feedback.(Or just duplicate eachothers feedback.) I think many regular people will think 'alright, i'll join', despite of the higher risk of bugs.(And maybe you can advertise use of !bugreport in the welcome message.) Since they had a choice, the negative image impact will also be lower. People where annoyed with bugs in 0.77*, the last(later) one seems to have most bugs weeded out.

[Tobi]

Idea seems nice, though maintaining duplicate API seems pain (OTOH I suppose it wouldn't be hard to design this in reasonable clean way using some code generation here and there.)

Using SVN revision numbers as engine version doesn't work, it complicates a move to other VCS, and it reduces compatibility unneccesarily. (As we have seen in the past multiple times already..., e.g. demos, sync, etc.. IOW, IMO it needs to be decoupled from source control.)

Using strings as engine version seems to me to introduce way too much messing around with strings, if you pass it to every unitsync method. I'd rather see you pass e.g. a 'version handle' as extra argument (just integer), which you first acquired through something like AcquireVersion (this could load DLL, do checks, etc.). This could take string argument.

AF does have a point in that this way unnecessary updates are required to the version thing. While unitsync API changes aren't *that* frequent, I think unitsync API did change about every single release last few years, and then we end up with exact same situation for this version thing as we have now for unitsync.

However, as YokoZar pointed out in other thread, I think we are very close to boundary of over engineering, and need to watch out very carefully to not cross it.

For one, I think it is fine to rely on the directory name as version. On modern OS this directory wouldn't be renameable by user anyway without special permissions, and if you don't trust user on renaming directory, then on what do you trust him? He could edit lua file too. He could put unitsync of 0.77b1 in directory of 0.76b1. He could copy springcontent.sdz from 0.73b2 and put it in 0.77b4 folder. Do you want to check all that?

Also using lua for version info seems overcomplicated, any version is just a string, requiring any tool which wants to find out about engine versions to depend on lua just because we don't trust user on not renaming directory but we do trust user on not renaming/editing lua file, seems silly.

[Licho]

Do we really need support for more than two engine versions? There seems to be a lot of potential for overengineering here - all I really need is the "Spring everyone else has" and the "latest Spring that we are going to test on".

Im not sure its easier to support 2 versions than any number of versions :)
Its probably even more complicated, because the "latest" version changes rapidly (several times per day), and you have to identify which game is being hosted by what anyway and somehow notify clients that they need to update to newest version.

But I agree there should be just 2 versions used - "stable" and "latest test".

[Licho]

re Tobi
There are other ways to solve "duplicated API" problems..

1)
We could "fixate" current unitsync - simply dont change it across engine versions and keep just one unitsync for whole installation.

It would just switch to correct base/springcontent based on engine version parameter.

Drawback is that even non-API changes to unitsync workings are restricted to "full releases" of the current style.


Alternatively there would also be no need for any springsync, since lobby themselves can easilly search the "engines" folder and pass path to base as parameter to certain unitsync calls.
<- this would probably be fastest to implement solution

2)
we could let lobbies load individual unitsyncs and keep unitsync specific calls away from springsync - but imo this solution is ugliest of all, because it means lobby has to be prepared for different versions of unitsync. So imo no way to go.

[AF]

How about adding the following to my proposed EngineSync library:

Code: Select all

int GetUnitsyncAPIVersion(int engineID);

This way if the required version is not available the lobby can fallback on a minimal version that uses the core API that hasn't changed since the start of spring that has the most basic of the API in it. Either way lobbies already contend with this problem.

I suggested a lua file to act as a basic config file telling the lobbies/tools the preferred engine to be used, and that the user could modify this if they wished so there default is a non-stable or future build or past build.

Also, we're forgetting that users will want older versions of the engine to be able to play old replays! Or lets say BA gets broken in 0.78 users can carry on with 0.77 until its fixed etc etc

[Licho]

I dont really want to support multiple API in stuff that interactis with unitsync and I dont think other developers want if they can avoid it..

Whole idea was to avoid having multiple/versioned API.

[AF]

Which is why you would implement a minimal API interface and the extended interface and reject anything that doesn't fit either.

Either way even with your springsync idea you would have to deal with this only its a lot more work in springsync because you have to duplicate functionality in springsync if its missing in the target proxied library and it becomes a mess.

[Licho]

Tbh im not concerned in long term handling of interface..
I wanted to support hour by hour releases of engine to improve speed of engine development and its quality and put restrictions off from engine developers.

I never wanted support across changing unitsync api. I think if api has to change - we have to make new release.

[Tobi]

The more I think about it the more I like this idea; as Licho implies it's an utopy to think we can fix everything for always, there will always be breaking things sometimes.

Unitsync is currently probably one of the most stable things related to the spring engine. New APIs get added every now and then and sometimes semantics change (read changing checksums ), but I don't think ever arguments to call were changed or call was removed (without huuuuuuuuge switch over period as for InitArchiveScanner method).

Also, it makes sense to do this logic of picking unitsync version by this 'springsync' DLL, because otherwise the same logic would be duplicated in all lobbies. This, for one, would mean that it would be impossible to directly link e.g. a C/C++ lobby to unitsync because it needs to be able to switch unitsync on the fly. This loading on the fly may be even harder in higher level languages like C#.

However, I think giving engine argument to many unitsync functions wouldn't make sense, just make API stateful (as unitsync API is now anyway), and add method SelectSpringEngineVersion(const char* version). Very efficient implementation would be possible using this (e.g. lazy loading in SelectSpringEngineVersion), and pretty much nothing needs to change in lobbies (nothing at all if a default version is selected in this springsync DLL).

When a unitsync API changes or is added, springsync will need to be rereleased, but so will lobbies and any other apps depending on this API changes. There's no way those apps can already be forward compatible with new APIs we don't even know that they will be created at some point..

Though maybe making unitsync version agnostic makes even more sense, given that with the above it's API can not change often anyway.

[AF]

There's a core of unitsync APIs which will not change and provide the basics such as map and mod listings.

Either way I do not believe we should end up with a unitsync that has an engineID parameter on every single exported function since that would be more work. Instead if we're going to go with spring sync we need a one off Init() call passing the engine ID.

The point with C++ unitsync sorta applies to springsync too since it would be C/C++.

However, I want the flexibility to override springsync. While you have a point that it would prevent duplication in the lobbies, that's making a large assumption that the lobbies all require the same exact identical behavior, which is an unacceptable assumption. Maximum flexibility is a huge must.

In the meantime how will we determine which folder in the bin directory contains the preferred version of spring? (This has uses outside the springsync idea)

[imbaczek]

preferred version should be given by the lobby server IMHO.

[AF]

That's not flexible enough, what about people running offline or on LANs? There needs to be a mechanism by which to figure out the preferred version that doesn't involve being told by an external source.

[imbaczek]

but only the external source _really_ knows. otherwise, I think a global setting for every version should be enough.

btw settings need to get back to Spring <version> for this to work sensibly.

[Licho]

I agree both with Tobi and Imbaczek.

Though I dont like state machines, unitsync is already one and adding extra "setengine" call wont confuse things more and will make things easier for lobbies- allowing even legacy support without need to code any change (you dont set engine, default is picked).

And yeah lobby server should broadcast "stable" version to be used as default in say "host this game" dialog.

[AF]

The lobby server should do that but there needs to be a way of telling which version is the most appropriate in an offline setting. Spring may be installed on a machine that's never connected to the internet, or never connects to the lobby (or cant connect to the lobby).

For example my machine in work has the lobby ports blocked, how would springsync know which version of the engine to use for single player or LAN games when there's no lobby server or internet connection to tell it?

Regarding settings, we need to sort out a method of copying existing settings over to the new version.

[Licho]

Some releases would be marked as "stable" latest stable is default preffered version.

Settings should be shared.

[AF]

So they would have stable at the end of their foldernames?

I would prefer a lua file in each folder with the full string version name, that way we can add extra options like stability, date of release, change log, authors etc.

[Licho]

Whatever it can be lua, tdf or whatever - springsync would read it, not you ..

[Tobi]

but only the external source _really_ knows. otherwise, I think a global setting for every version should be enough.

btw settings need to get back to Spring <version> for this to work sensibly.

If so they need to be done smarter, ie. if setting isn't found yet try to get it from older version until it is either found or that version doesn't exist in the settings source. This also means it shouldn't write default values in settings source.

[Tobi]

For version names just adhere to Spring's current convention.

[0-9].[0-9][0-9]b[0-9] is stable (e.g. 0.77b4) and
[0-9].[0-9][0-9]b[0-9]\+.* is unstable (e.g. 0.77b4+, 0.77b4+svn6877, etc.)