The end of the maintenance branch

The end of the maintenance branch

Discuss the source code and development of Spring Engine in general from a technical point of view. Patches go here too.

Moderator: Moderators

User avatar
ivand
Posts: 155
Joined: 27 Jun 2007, 17:05

The end of the maintenance branch

Post by ivand »

Hi there!

I propose to end the maintenance branch (104+) as we know it. It uses a lot of legacy graphics code that:
  • Slow. The newer rendering approaches are at least a magnitude faster.
  • Totally deprecated. The older approach required that drivers hold huge state and handle peculiar 20+ years legacy interactions. The newer drivers tend to introduce bugs here and there, this bugs will never be fixed, because no commercial games run old OpenGL anymore.
  • It significantly holds back development of new graphical features for spring, game/Lua and engine sides as well. Even if something is developed, the developer needs to implement at least three rendering paths to make universally compatible. No one wants to do that.
  • We are in mid of 2020, even embedded Intels support the "new" OpenGL, let alone GPU/APU from NVidia/AMD. Each day the more legacy hardware gets phased out and the new one gets purchased. If you have no money or desire to change your 10+ years old laptop or GFX card, we are sorry, but that's not something engine or game developers can help you with.
  • Last, but not least. For those unaware we have had no lead engine dev since last year, merely part time contributors. The reason is that no one wants to fix or work on the current "maintenance" engine.
My proposal is to end the development and support for the maintenance branch, signified by releasing the next stable. Call it 104.1 or 105.0. Whatever desired. I fully understand that many spring games have little man-power or motivation to move forward, for them 104.1/105.0 would be the release to target for to stay stable. The current maintenance nightly builds have been run by Zero-K and BAR to name a few, I'm aware of, and are relatively stable and have no known fatal bugs. The biggest advancement here is a fixed ROAM/LOAM code by Beherith, which improved performance of terrain rendering and performance significantly. Again the latest nightly of maintenance is decent enough and requires relatively marginal changes if you migrate from 104.0. My proposal is to name https://springrts.com/dl/buildbot/defau ... -g89bb8e3/ the next stable and the end of the maintenance era release. AFAIK @GoogleFrog and @Floris agree it's a good release.

Moving on, my plan is to modify the Lua graphics API of the current "maintenance" such that it's API compatible with the other branch of the engine called "develop". "Develop" uses fully updated OpenGL code and runs several times faster as far as rendering is concerned, that said the current "develop" doesn't even use the extended acceleration capabilities of new OpenGL to full extent, meaning that it can be made even faster once it gets the deserved attention. There is a big likelihood that not-only the current "maintenance" engine will need to move towards "develop", but also that "develop" will need to be readjusted to ease the transition and stay compatible with hardware/drivers of as many as reasonable.

Once the "maintenance" or better call it "transition" is extended with new API, I expect that developers of BAR, ZK, Chobby/Chilli will lead the transition on the Lua side. A lot of widgets and gadgets will need to be rewritten so they use new common API. Gradual introduction of the new API and gameside changes allow for uninterrupted game development (in areas other than graphics) and for almost unaffected user experience (since we talk about the process similar to continuous integration each misstep on the game or engine side could be easily reverted). The final step would be to fully switch over to "develop" as a "transition" branch serves only as a mean to bridge the gap towards the future spring ("develop"). The developers of other games interested in moving on are more than welcome to join the endeavor or follow on on the results.

I was advised to get approval of the plan above from spring project wardens: @Kloot, @hokomoko, @gajop, @abma and @SilentWings. Please speak up.
Last edited by ivand on 18 Jun 2020, 14:37, edited 1 time in total.
10 x

User avatar
Silentwings
Moderator
Posts: 3660
Joined: 25 Oct 2008, 00:23

Re: The end of the maintenance branch

Post by Silentwings »

+1 from me.
2 x

User avatar
IceXuick
Posts: 510
Joined: 14 Mar 2006, 01:46

Re: The end of the maintenance branch

Post by IceXuick »

I fully/100% support this proposal and hope we can start working on this ASAP.
1 x

User avatar
Beherith
Moderator
Posts: 4969
Joined: 26 Oct 2007, 16:21

Re: The end of the maintenance branch

Post by Beherith »

Ive seen generally excellent stability from latest maintenance, and agree that it is a good closing point for old OpenGL paths.

GFX hardware/drivers have evolved to such a point, as witnessed by the magnificence of what I've witnessed from Kloot's work in develop that I feel the time has come to leave 10+ year old hardware behind, especially considering the gains to anyone who has something even a bit recent.

Cheers to everyone pushing heavily into 2020, and an enormous thank you to all current and past engine devs who have made our work possible!

It would be helpful if ivand got commit rights into engine, so we can test and iterate quickly.
2 x

User avatar
Floris
Balanced Annihilation Developer
Posts: 606
Joined: 04 Jan 2011, 20:00

Re: The end of the maintenance branch

Post by Floris »

yes please!
2 x

User avatar
AF
AI Developer
Posts: 20678
Joined: 14 Sep 2004, 11:32

Re: The end of the maintenance branch

Post by AF »

+1
2 x

User avatar
FLOZi
MC: Legacy & Spring 1944 Developer
Posts: 6188
Joined: 29 Apr 2005, 01:14

Re: The end of the maintenance branch

Post by FLOZi »

Fwiw, +1 from me.

It took ~10 minutes to update MCL (untouched since 2017) for maintenance-1510 from 104.0, and would have been quicker had I opened the changelog and looked first. :roll:
1 x

Ares
Balanced Annihilation Developer
Posts: 368
Joined: 19 Mar 2011, 13:43

Re: The end of the maintenance branch

Post by Ares »

Do not do this. Ivand couldn't care less about the future of Spring. This is just a political move to put the engine into the hands of BAR.

Image

BA was already sacrificed on a whim just so he could add some cell shader effect to BA10. If you keep following his suggestions in the end Spring will only run on 64 bit Windows.

User was warned for this post. Felony 1. Please avoid ad hominem.
-- FLOZi
Attachments
Untitledaa.png
(28.72 KiB) Not downloaded yet
1 x

User avatar
IceXuick
Posts: 510
Joined: 14 Mar 2006, 01:46

Re: The end of the maintenance branch

Post by IceXuick »

@Ares: This is 180 degrees different in how i have learned to know ivand. He cares very much and is willing to progress/push forward/optimize/work together. (and many more strong points besides these).

Furthermore he has shown the drive/passion/motivation to see this through.
He has talked about these plans for months (if not years) and i really appreciate a persons persistence to go 'beyond'.

And for the sake of BAR - it's that in the past year we've had so many ideas/things we want to and can do, though only when moving forward and leaving old tech behind. This is holding back tremendous power and possibilities, perhaps already even since we re-enlightened BAR ±2 years ago. I think that's a big waste of potential.
1 x

User avatar
FLOZi
MC: Legacy & Spring 1944 Developer
Posts: 6188
Joined: 29 Apr 2005, 01:14

Re: The end of the maintenance branch

Post by FLOZi »

IceXuick wrote:
18 Jun 2020, 16:57
@Ares: This is 180 degrees different in how i have learned to know ivand. He cares very much and is willing to progress/push forward/optimize/work together. (and many more strong points besides these).

Furthermore he has shown the drive/passion/motivation to see this through.
He has talked about these plans for months (if not years) and i really appreciate a persons persistence to go 'beyond'.

And for the sake of BAR - it's that in the past year we've had so many ideas/things we want to and can do, though only when moving forward and leaving old tech behind. This is holding back tremendous power and possibilities, perhaps already even since we re-enlightened BAR ±2 years ago. I think that's a big waste of potential.
Though positive, please can we keep this topic about technical aspects and not personal ones.
2 x

Ares
Balanced Annihilation Developer
Posts: 368
Joined: 19 Mar 2011, 13:43

Re: The end of the maintenance branch

Post by Ares »

Why do I get warned when Ivand is the one who literally called you a cancerous limb? If I provide a list of personal attacks he's made against Spring devs, will you remove my warning please?

I get 9 fps zooming in on a tree in BAR, so having more opengl calls at your disposal seems inversly proportional to frame rate and will be bad for the health of Spring Engine.

If you let this happen the obvious consequence will be every game apart from BAR will end up broken and unsupported and BAR will do whatever it pleases with the engine. This is my appeal to the people in charge not to let that happen.
Last edited by Ares on 18 Jun 2020, 17:46, edited 3 times in total.
0 x

MasterBel
Posts: 205
Joined: 18 Mar 2018, 07:48

Re: The end of the maintenance branch

Post by MasterBel »

Ares wrote:
18 Jun 2020, 17:13
I get 9 fps zooming in on a tree in BAR
Very sure Ivand would love to hear details about this so he can do something to fix it.

Also I (and other Mac people) would be very happy if the transition took us through an OpenGL core profile <= 4.1 (And stayed there? :wink: )

Again many thanks for all the work you've done, @ivand
0 x

User avatar
ivand
Posts: 155
Joined: 27 Jun 2007, 17:05

Re: The end of the maintenance branch

Post by ivand »

Well, I regret the exact choice of words, I wrote that being frustrated by the lack of feedback and you caught me writing in the bad mood. I won't be explaining what I meant behind the rude words publicly here, but I can explain to anyone interested privately. I have never been shy to tell what I think straight, but again I apologize for the language.

I'm obviously not begging for any spring commit rights. If people decide it's reasonable, I wouldn't mind having them, if not - I'll be happy to help someone else who is willing to execute on the outlined plan with the rights I have.

Now exclude myself from the equation and please discuss the proposed plan.
0 x

matyhaty
Posts: 11
Joined: 22 May 2014, 15:12

Re: The end of the maintenance branch

Post by matyhaty »

I dont normally get caught up in this stuff, but I will urge some caution

BAR is not out.
The only real games which are happening on the official right now is TA.

So while I support the move to focus on future engines and accept things must move forwards - there needs to be a clear understanding and acceptance that TA might not be able to achieve 105 support for some time.

What you cant have is the spring lobbies forcing 105 when there isnt viable games.
The transition to BAR (if its a transition at all) may or may not happen, but you have to assume it will take time for player to consider moving to a new 'game'

We cant have more players leave Spring
We are all ready fragmented.

Perhaps as part of the move to 105 engine devs could offer support to game mods like TA, BA and whateer ones there are to help them move to 105 swiftly.

Please keep this in mind
0 x

User avatar
FLOZi
MC: Legacy & Spring 1944 Developer
Posts: 6188
Joined: 29 Apr 2005, 01:14

Re: The end of the maintenance branch

Post by FLOZi »

If games work on 104.0 there should not be that much to do for 104 maintenance 1510 and ergo 105.0.

I offer my help in this regard. It's little more than a few changed lua API calls.

If they don't work on 104.0 (released circa 2017) yet, you have a bigger problem of not having anyone maintaining the game you want to play.



Engine dev help will be needed for those looking to move past 105 into develop.


I do have concerns that 1510 has some 'off by 1' indexing errors for unit pieces in scripts in S44 and CEG emission in MCL. Need to check changelog to see if anything relevant in there. hoko was surprised by the former.
0 x

Ares
Balanced Annihilation Developer
Posts: 368
Joined: 19 Mar 2011, 13:43

Re: The end of the maintenance branch

Post by Ares »

If they don't work on 104.0 yet, you have a bigger problem
Moving engine is trivial, the reason games cannot migrate is because the same people asking for this request previously decided it was more important to add new opengl shaders than letting players who had played Spring games for 10 years keep playing their favourite games. First they came for the Mac players and next they will come for you.
If you have no money or desire to change your 10+ years old laptop or GFX card, we are sorry, but that's not something engine or game developers can help you with.
Apart from Flozi everyone agreeing is a BAR contributor, and this is just a power grab by BAR to hijack the engine for their own purposes, if you care about open source and the future of Spring outside of BAR you must decline this request.
0 x

abma
Spring Developer
Posts: 3601
Joined: 01 Jun 2009, 00:08

Re: The end of the maintenance branch

Post by abma »

ivand wrote:
18 Jun 2020, 12:57
Moving on, my plan is to modify the Lua graphics API of the current "maintenance" such that it's API compatible with the other branch of the engine called "develop".
@ivand:
why not vice versa, why not add some backwards compat code to the develop branch? i don't have code insights: i just want to know, if you have a reason to add the opengl code to maintenance. adding more code to maintenance makes it to live longer i guess. without investigation/code insights i would suggest to:

- release current maintenance as 105.0
- drop maintenance branch
- add compat code to develop
- release develop as 106.0

still +1 from me in general for the plan and thanks a lot for your efforts!
4 x

User avatar
ivand
Posts: 155
Joined: 27 Jun 2007, 17:05

Re: The end of the maintenance branch

Post by ivand »

abma wrote:
18 Jun 2020, 23:53
Moving on, my plan is to modify the Lua graphics API of the current "maintenance" such that it's API compatible with the other branch of the engine called "develop".
why not vice versa, why not add some backwards compat code to the develop branch? i don't have code insights: i just want to know, if you have a reason to add the opengl code to maintenance. adding more code to maintenance makes it to live longer i guess.
I'll gladly explain. First off we should define terminology:
  • Fixed function pipeline (FFP) - the older pre-shader era method of rendering, the main one used in spring-maintenance
  • Compatibility OpenGL profile - the implementation of OpenGL that tries to marry together FFP and more advanced concepts, that were developed after FFP. spring-maintenance profile in use
  • Programmable pipeline - the modern implementation that deprecated most FFP functions, other redundant functions with very little exception. All the removed functions are now implemented in GPU programs - shaders. Used in spring-develop.
  • Core OpenGL profile - the implementation of OpenGL that strictly denies the use of all outdated and removed functions.
Now the actual explanation. The core profile simply doesn't have several dozens of FFP functions, that compat profile used to have. They are either deprecated as concepts or implemented with shaders. You cannot just for example do gl.AlphaTest() from Lua, because its former functionallity is nowadays implemented in GLSL, which lives in domain separate from Lua so to speak. The compat profile was designed to accommodate new and old concepts together, therefore maintenance would be the right target to extend develop API into and not the other way around.

There is one extra important bit here. The current games run on the maintenance. Even if we went by the path of "downgrading" develop, such engine would not be useful, until the downgrade is complete. I'm a believer in the continuous non-disruptive development and only extension of the maintenance allow for that approach.

Around 9 months ago I tried to come up with the common rendering API, mimicking maintenance OpenGL API that would have "develop" class implementation. I was even able to render some widgets with it, but then I stumbled with something I couldn't bridge, not to mention that Lua part of it was becoming to look like a monster from reading and performance perspective. I'm afraid the only right way is to implement things "properly" according to modern best practices.
abma wrote:
18 Jun 2020, 23:53
still +1 from me in general for the plan!
Much appreciated.
0 x

Ares
Balanced Annihilation Developer
Posts: 368
Joined: 19 Mar 2011, 13:43

Re: The end of the maintenance branch

Post by Ares »

But your track record when it comes to perfomance is reducing my fps to 10 with 1 unit in play. By comparison I get 50 fps with 10k units in BA, so clearly technical ability is lacking. For the good of all, lets not give engine access to someone whose only measurable impact on performance is to decrease it. Instead I counter nominate Mando to take over engine development, he is an excellent coder with a heart of gold, he can do it in webGl, run ba in a browser and get better perfomance than Ivand and would never leave a single person behind.

Attached image of 55fps with 10k units in BA, deathstar @ 24 fps on his single core 10 year old rig 10k units no lag

Image

Attached image of 33fps with 1 unit in BAR due to new opengl calls. I tried to use my normal 4k resolution and zoom closer to show it going to 9fps but due to the graphically intensive nature of one unit and one tree my pc ran out of memory and then Spring crashed.

Image


User was warned for this post. Felony 2. Please only constructive criticism.
Side note: Mando is banned, has never created a pull request or a bug report and he didn't try to follow our rules.
-- abma
Attachments
1_35fps.png
(719.04 KiB) Not downloaded yet
10k 50fps.png
(2.14 MiB) Not downloaded yet
0 x

abma
Spring Developer
Posts: 3601
Joined: 01 Jun 2009, 00:08

Re: The end of the maintenance branch

Post by abma »

ivand wrote:
19 Jun 2020, 00:13
Now the actual explanation. The core profile simply doesn't have several dozens of FFP functions, that compat profile used to have. They are either deprecated as concepts or implemented with shaders. You cannot just for example do gl.AlphaTest() from Lua, because its former functionallity is nowadays implemented in GLSL, which lives in domain separate from Lua so to speak. The compat profile was designed to accommodate new and old concepts together, therefore maintenance would be the right target to extend develop API into and not the other way around.

There is one extra important bit here. The current games run on the maintenance. Even if we went by the path of "downgrading" develop, such engine would not be useful, until the downgrade is complete. I'm a believer in the continuous non-disruptive development and only extension of the maintenance allow for that approach.

Around 9 months ago I tried to come up with the common rendering API, mimicking maintenance OpenGL API that would have "develop" class implementation. I was even able to render some widgets with it, but then I stumbled with something I couldn't bridge, not to mention that Lua part of it was becoming to look like a monster from reading and performance perspective. I'm afraid the only right way is to implement things "properly" according to modern best practices.
thank you for explaining, so a +1 from me for that plan!
0 x

Post Reply

Return to “Engine”