The end of the maintenance branch - Page 8

The end of the maintenance branch

Discuss the source code and development of Spring Engine in general from a technical point of view. Patches go here too.

Moderator: Moderators

Post Reply
abma
Spring Developer
Posts: 3798
Joined: 01 Jun 2009, 00:08

Re: The end of the maintenance branch

Post by abma »

ok, i've tagged 105.0 from maintenance and deleted it. I did not downgrade the buildbot workers. so the 105.0 linux builds will require glibc >= 2.29 as i'm to demotivated to dowgrade the buildbot workers, sorry. IMHO its better to release with "high" requirements than no release.

happy new year!
ivand
Posts: 310
Joined: 27 Jun 2007, 17:05

Re: The end of the maintenance branch

Post by ivand »

I understand your frustration. One particular troll almost ruined my motivation to contribute to spring, other smaller ones spread decay here and there in small amounts to paint it all black. I'm actually surprised how lenient their treatment is. IMO moderation at springrts should be way stricter than it is.

I wish 105 was released with older deps in mind, but we have what we have. Thanks for making it happen, better late than never.

P.S. I haven't been slacking either: https://github.com/beyond-all-reason/sp ... reason:BAR
abma
Spring Developer
Posts: 3798
Joined: 01 Jun 2009, 00:08

Re: The end of the maintenance branch

Post by abma »

ivand wrote: 01 Jan 2021, 17:45 P.S. I haven't been slacking either: https://github.com/beyond-all-reason/sp ... reason:BAR
this seems to be based on the maintenance branch: how do we get this into the develop branch?

it looks like the roadmap is wrong and 106.0 will be a continuation of the maintenance branch? if not, the develop branch should merged into the BAR branch.

(sorry if i missed something, its difficult for me to follow because of many interruptions with different topics)
ivand
Posts: 310
Joined: 27 Jun 2007, 17:05

Re: The end of the maintenance branch

Post by ivand »

The transition or BAR branch as a possible bleeding edge of it is based on the maintenance indeed. The reason is simple: it's not possible to "dumb down" the develop to run on low OpenGL profiles.

Instead the maintenance is getting new GL capabilities mostly in Lua space, so games could migrate their widget/gadget code to these GL primitives. Later on the same "new GL" code can be added to the develop too so switching between the two is more or less easy.

P.S. For now I've switched to a feature branch https://github.com/beyond-all-reason/sp ... O?expand=1 that involves some additional changes to the engine code. Mostly they enable the way to completely bypass the way the maintenance/transition engine draw things and instead draw stuff from Lua in a modern way. In particular I rewrote that VBOs are now assigned per model (same way as the develop does, used to be per-piece and it was too granular), exposed model VBOs to Lua, collected the most common matrices (bind pose matrices, unit matrices, piece matrices, etc) into a GPU buffer, that is accessible from any widget/gadget. Now I'm thinking through the system to do the indexing of the matrices data. In any case even now with static indexing one can draw any model from the widget. Such drawing bypasses the engine (old style) drawing completely: no display lists, no embedded matrices stack, no gl_Vertex, etc.
Image the left robot is a unitDef drawn in that new way.

I expect to finish that branch within a few days or weeks, test it and then perhaps collect relevant commits and put together a PR.
Attachments
screen02023.png
(4.07 MiB) Not downloaded yet
ivand
Posts: 310
Joined: 27 Jun 2007, 17:05

Re: The end of the maintenance branch

Post by ivand »

Drawn in one go without any CPU roundtrip

P.S. Note it's completely different from the above. Above it was a static unitdef, but here it's a unit. But both are drawn using the same mechanism.

P.P.S. Now the code is wired to unitID/featureID/unitDefID/featureDefID.
ivand
Posts: 310
Joined: 27 Jun 2007, 17:05

Re: The end of the maintenance branch

Post by ivand »

@abma, the commits on the BAR branch keep piling up and since this branch dragged several contributors other than myself, the question of pulling the changes back becomes quite important. Have a look at the current diff: https://github.com/spring/spring/compar ... reason:BAR it's already massive enough that no sane one-step review is possible. At least I talked to @gajop and he confirmed that.

With that in mind I'd like to discuss the merge strategy (provided it's still wanted by spring/spring owners).
* First we can just merge it in one big chunk minus some minor local changes I did for BAR like removing outdated AIs. This makes review hardly a possibility but represents the easiest way as far as I'm concerned.
* The second strategy would be to review code by functional parts, but since spring is tightly coupled I may well see the scenarios where the PRs are non-atomic. E.g. they might contain parts of the unrelated code and might not even compile because of external dependencies to be covered in further PRs. Despite parts being non-atomic, the code can be sanely reviewed by senior peers.

Separately I'd like to note that I did substantial modifications to CMakeLists to get the compilation going and expose the tuning params I wanted for Github builds, so spring shepherds better make up their mind if they want to migrate builds (and issues) infra to Github or stay where they are. In the later case I cannot guarantee the code from PRs will compile the same way or at all it's getting compiled in my environment.

P.S. The code has been mostly verified, but I'll admit some parts received very limited if any testing, so it's mostly good for devs only to do first baby steps in migration.
abma
Spring Developer
Posts: 3798
Joined: 01 Jun 2009, 00:08

Re: The end of the maintenance branch

Post by abma »

i guess reviewing will be teamwork...

still the biggest issue is, that it is based on the maintenance branch.
ivand
Posts: 310
Joined: 27 Jun 2007, 17:05

Re: The end of the maintenance branch

Post by ivand »

I've never claimed it'll be based off something other than the maintenance. Because unlike the develop where the support for the old ways was specifically purged, the maintenance can run both old and the new code. Also see the very first post of this topic.

Your answer doesn't give much clarity how to approach this nor if you are willing to give up on buildbot and thus if I should include CMakeLists changes. I guess you would want to discuss it with whoever left active and come back with some guidance how to proceed.

P.S. would be nice if you were Discord accessible, even through a bridge to Matrix as we tend to discuss such matters with the rest in the real time.
abma
Spring Developer
Posts: 3798
Joined: 01 Jun 2009, 00:08

Re: The end of the maintenance branch

Post by abma »

then we have a big problem. i always claimed that the "develop" branch will and should be continued and the maintenance branch beeing dropped: kloot has/had the same stance afaik.

not sure how we can continue: suggestions welcome. suggest a roadmap please / correct the existing one.

as i read the first post, you suggest to create 106.0 from "transition" and then drop transition and continue "develop" as it is, right?
ivand
Posts: 310
Joined: 27 Jun 2007, 17:05

Re: The end of the maintenance branch

Post by ivand »

Don't think we have any problem here. :P
We discussed it extensively in this topic and in one of the github PRs and I'm sure Kloot got my intents right.

Yes, port widgets to GL4 API gradually while being on transition then switch engines (assuming the develop gets the same API, which is perfectly doable). As an example BAR now uses all old widgets, except for one I ported to GL4. Once porting is complete we can switch. The primary advantage here is that you don't need to put your game on pause for half-year or whatever time it takes to port all widgets because both old-GL and new-GL widgets remain operational.
abma
Spring Developer
Posts: 3798
Joined: 01 Jun 2009, 00:08

Re: The end of the maintenance branch

Post by abma »

can you please suggest a roadmap with details which branch will be merged where, and when a release should be made of what.

without a public plan i can't help.
ivand
Posts: 310
Joined: 27 Jun 2007, 17:05

Re: The end of the maintenance branch

Post by ivand »

The roadmap is rather trivial, I'm not even sure there is anything to document here. Anyway:

As stated "transition" stems from "maintenance" and remains compatible to "maintenance" builds such as 105.0 in its core. This means that no engine core capabilities are removed or changed, but new OpenGL primitives are added to Lua and partly to the engine as support. Currently the bleeding edge of transition is represented by BAR branch of spring at beyond-all-reason github.
I urge to merge BAR branch into transition soonest as the amount of diverging commits keep piling up. The merge is already on the verge of doable and is better be done soon or it won't happen at all.

As far as my work is concerned "develop" will remain dormant for now. I'm not sure if any game will manage the transition to new GL by means of using transition or BAR branch, therefore I see no reason to spend time on updating "develop".
That said: if a game reaches the point when transition is say 90% complete and the game runs almost all code by means of new GL (from transition/BAR), I'm gonna add the same Lua API (and necessary engine support) that exists in the "transition" to "develop".
abma
Spring Developer
Posts: 3798
Joined: 01 Jun 2009, 00:08

Re: The end of the maintenance branch

Post by abma »

ivand wrote: 24 Jan 2021, 13:04 The roadmap is rather trivial, I'm not even sure there is anything to document here. Anyway:
no its not. what do you suggest to release as 106.0? it sounds like the "transition" branch.

what will be 107.0?

having a roadmap answers questions like this: https://github.com/spring/spring/pull/5 ... -758095113 and allows other people to help. without a release plan / roadmap its difficult to update the changelog.

without a goal i guess nothing will happen.
ivand
Posts: 310
Joined: 27 Jun 2007, 17:05

Re: The end of the maintenance branch

Post by ivand »

abma wrote: 24 Jan 2021, 13:39 no its not. what do you suggest to release as 106.0? it sounds like the "transition" branch.
what will be 107.0?
I don't think the focus on release numbers is important: it took several years to get from 104 to 105, but that fact never prevented innovation in the in-between versions. It's not like spring has a lot of developers and strict product plans to follow to be focused on releases. Given the circumstances I personally consider every intermediate release (like a buildbot run) as a release.

If you still want an answer with release numbers, then ideally 106.0 should be the develop, while the transition will be something like 105+ (same way as it was with the maintenance builds).
abma wrote: 24 Jan 2021, 13:39 having a roadmap answers questions like this: https://github.com/spring/spring/pull/5 ... -758095113 and allows other people to help.
Gajop got it right there.
abma wrote: 24 Jan 2021, 13:39 without a goal i guess nothing will happen.
Is the above info enough?
User avatar
MasterBel
Posts: 271
Joined: 18 Mar 2018, 07:48

Re: The end of the maintenance branch

Post by MasterBel »

Might I suggest that before support for transition be dropped, you release a stable version for anyone who continues to make use of the 105 GL implementation? Even if that means releasing a 106.0 from Transition and 106.1 from Develop about the same time?

Also having some idea of a periodic stable release whenever a +XXXX version is deemed stable enough may be useful for the preservation of builds on the server. Not too often, say just every couple months or so. If that wouldn't take a tonne of extra effort, I think it may be good for morale and potentially even from a publicity standpoint. (I recall some places covered Spring 104, but did we get any attention for 105?)
ivand
Posts: 310
Joined: 27 Jun 2007, 17:05

Re: The end of the maintenance branch

Post by ivand »

MasterBel wrote: 24 Jan 2021, 23:39 Might I suggest that before support for transition be dropped, you release a stable version for anyone who continues to make use of the 105 GL implementation? Even if that means releasing a 106.0 from Transition and 106.1 from Develop about the same time?
Just to be on the same page: the transition is supposed to be 105.0 as a base + new OpenGL API + some minor QoL improvements and bugfixes. Those who prefer to stay with the current OpenGL in Lua (& in the engine) may well stay on 105.0. The transition hardly brings any significant value for them. I see the transition mostly as a mean to carry the game towards the develop. As I was saying with the resources we have left, only a rolling release model is doable, but if the community feels it is important to have transition as a numbered release, then I have no objections.
MasterBel wrote: 24 Jan 2021, 23:39 Also having some idea of a periodic stable release whenever a +XXXX version is deemed stable enough may be useful for the preservation of builds on the server. Not too often, say just every couple months or so. If that wouldn't take a tonne of extra effort, I think it may be good for morale and potentially even from a publicity standpoint. (I recall some places covered Spring 104, but did we get any attention for 105?)
IMO Frequent releases are not a solution to infra issues, but it might indeed be good for publicity.
abma
Spring Developer
Posts: 3798
Joined: 01 Jun 2009, 00:08

Re: The end of the maintenance branch

Post by abma »

without a release i don't see why should i spent any time on it. this is really needed for motivation and publicity: thats why i'm still waiting for an adjusted release roadmap for the transition changes.

wrt merging: the github actions and the "transition" changes to the engine should be split into at least two pull requests.

the last time i checked the github actions, they were in a state, that can't be merged as they duplicate A LOT of code. this has to be fixed first as duplicate code is really bad to maintain.
ivand
Posts: 310
Joined: 27 Jun 2007, 17:05

Re: The end of the maintenance branch

Post by ivand »

it's hard to say where things are going with the transition.

I do commit to the BAR fork regularly (https://github.com/beyond-all-reason/spring/commits/BAR), but in the same time we realize it's still a huge effort to migrate everything to the develop and the develop itself needs lots of enhancements (which I also do slowly https://github.com/beyond-all-reason/sp ... mmits/BARd). By now both branches (BAR and BARd) have common transition API and widgets written with this common API work cross-engine. There are still some issues with things like font rendering, but that is not the point. The point is that update work is not moving due to being still too heavy.

Perhaps we will do it the other way around and port some of the develop parts into the current engine, while removing outdated things like stone age shading paths and alike.

We release engines around once in two weeks and call it progress. I understand you want some planning, but I don't have any concrete plans. Things change as we go.

As far as GH Actions I agree with your dup code assessment. Feel free to propose the improvements and I'll implement them. Despite my by now significant experience in building the engine, I realize you know much more.

Ideally I want something like "make portable" that would collect the files and prepare the builddir to be zipped and shipped as minimal-portable engine. Right now the preparation requires a lot of manual steps, which should be CMake'd ideally.
abma
Spring Developer
Posts: 3798
Joined: 01 Jun 2009, 00:08

Re: The end of the maintenance branch

Post by abma »

hm, so the roadmap is: "when its ready..."

i guess its not useful for other projects to start migrating to sth. which isn't working: ideally a transition release can be skipped :-)

ivand wrote: 20 Mar 2021, 11:16 Ideally I want something like "make portable" that would collect the files and prepare the builddir to be zipped and shipped as minimal-portable engine. Right now the preparation requires a lot of manual steps, which should be CMake'd ideally.
IMHO cmake's cpack 7zip generator can and should be used: i didn't use it yet, but it should be straight forward. this possible breaks some custom stuff in the existing scripts: but very likely most customization in the build scripts can be silently dropped: i.e. the spring installer already was removed. Only thing idk if its easily possible with cpack is, the debug symbols. not sure if cmake/cpack can do the same without ugly hacks: https://github.com/spring/spring/blob/d ... dle.sh#L28

so generally speaking, the scripts for creating the archives should be replaced with cmake so the build target(s) can be directly used for github actions and the buildbot.
ivand
Posts: 310
Joined: 27 Jun 2007, 17:05

Re: The end of the maintenance branch

Post by ivand »

abma wrote: 23 Mar 2021, 04:21 hm, so the roadmap is: "when its ready..."
As far as "main" spring is concerned probably yes, but within the community I'm at right now we have different planning horizon and roadmap items come and go in weeks.
abma wrote: 23 Mar 2021, 04:21 i guess its not useful for other projects to start migrating to sth. which isn't working: ideally a transition release can be skipped
I guess we are not on the same page regarding migration. Realistically only one maybe two games can lead GL4 transition, because the effort is quite huge. I'm talking to these guys directly, so there is not much point to bringing the code to the wider community since it practically won't be used.

Thanks for cpack ref. Never heard of it, will have a look. My question however was more near-sighted. You mentioned the code duplicated the existing build routines and I was referring to these code parts and what they can be substituted with.
Post Reply

Return to “Engine”