Autodeletion of builds & types of buildbot (split from end of maintenance branch)

Autodeletion of builds & types of buildbot (split from end of maintenance branch)

Post just about everything that isn't directly related to Spring here!

Moderator: Moderators

gajop
Moderator
Posts: 3051
Joined: 05 Aug 2009, 20:42

Autodeletion of builds & types of buildbot (split from end of maintenance branch)

Post by gajop »

abma wrote: 19 Dec 2020, 18:47 ok, i can do this. Thats create 105.0 from maintenance + drop the maintenance branch, right?

I'll try to so on monday: break-day! 8) :lol:

Can you please also prevent the script from deleting old builds? At the very least keep the latest 100 builds or so.

Right now it's deleting them based on date, which is a huge issue due to a lack of activity. We recently ended up not having any maintenance builds left.
User avatar
Silentwings
Posts: 3720
Joined: 25 Oct 2008, 00:23

Re: The end of the maintenance branch

Post by Silentwings »

You simply won't find any better candidate to release as 105/stable.
+1 from me, and GL update gets renamed 106.
abma
Spring Developer
Posts: 3798
Joined: 01 Jun 2009, 00:08

Re: The end of the maintenance branch

Post by abma »

gajop wrote: 20 Dec 2020, 05:43 Can you please also prevent the script from deleting old builds? At the very least keep the latest 100 builds or so.
due to lack of time, thats atm unrealistic. creating a release of springlobby + 105.0 is IMHO much more important.

also maintenance 104.0.1-1563-g66cad77 exists which should be tested as it was compiled with a new environment (gcc 10).
User avatar
bibim
Lobby Developer
Posts: 952
Joined: 06 Dec 2007, 11:12

Re: The end of the maintenance branch

Post by bibim »

raaar wrote: 19 Dec 2020, 18:08While new maintenance builds are still broken, please restore the 1553 maintenance build files on the https://springrts.com/dl/buildbot/default/maintenance/.
Yeah, the new static builds will fail on a lot of systems because they require GLIBC 2.29, so I guess it would be a good option to put back build 1553 online as afaik it is the last build which doesn't require GLIBC 2.29.
raaar
Metal Factions Developer
Posts: 1094
Joined: 20 Feb 2010, 12:17

Re: The end of the maintenance branch

Post by raaar »

bibim wrote: 20 Dec 2020, 16:26
raaar wrote: 19 Dec 2020, 18:08While new maintenance builds are still broken, please restore the 1553 maintenance build files on the https://springrts.com/dl/buildbot/default/maintenance/.
Yeah, the new static builds will fail on a lot of systems because they require GLIBC 2.29, so I guess it would be a good option to put back build 1553 online as afaik it is the last build which doesn't require GLIBC 2.29.
Is that an issue related to getting spads to use the new engine build?
User avatar
FLOZi
MC: Legacy & Spring 1944 Developer
Posts: 6240
Joined: 29 Apr 2005, 01:14

Re: The end of the maintenance branch

Post by FLOZi »

gajop wrote: 20 Dec 2020, 05:43
abma wrote: 19 Dec 2020, 18:47 ok, i can do this. Thats create 105.0 from maintenance + drop the maintenance branch, right?

I'll try to so on monday: break-day! 8) :lol:

Can you please also prevent the script from deleting old builds? At the very least keep the latest 100 builds or so.

Right now it's deleting them based on date, which is a huge issue due to a lack of activity. We recently ended up not having any maintenance builds left.
+9001
due to lack of time, thats atm unrealistic. creating a release of springlobby + 105.0 is IMHO much more important.
:lol: :|





:(
User avatar
bibim
Lobby Developer
Posts: 952
Joined: 06 Dec 2007, 11:12

Re: The end of the maintenance branch

Post by bibim »

raaar wrote: 20 Dec 2020, 17:30
bibim wrote: 20 Dec 2020, 16:26the new static builds will fail on a lot of systems because they require GLIBC 2.29, so I guess it would be a good option to put back build 1553 online as afaik it is the last build which doesn't require GLIBC 2.29.
Is that an issue related to getting spads to use the new engine build?
I don't know, what issue are you referring to exactly ? If your system supports the new Spring engine builds (in particular, if it has Glibc >= 2.29), then SPADS should work fine (maybe post on SPADS forum or open an issue on Github to avoid getting offtopic here).
abma
Spring Developer
Posts: 3798
Joined: 01 Jun 2009, 00:08

Re: The end of the maintenance branch

Post by abma »

bibim wrote: 20 Dec 2020, 16:26 Yeah, the new static builds will fail on a lot of systems because they require GLIBC 2.29, so I guess it would be a good option to put back build 1553 online as afaik it is the last build which doesn't require GLIBC 2.29.
glibc 2.29 was released 2019-01-31, ubuntu 20.04 has glibc 2.31, debian 10 is to old, debian 11 with an up to date glibc will be released in early 2021. On which distribution / version did you currently test?

the build system isn't touched this often: thats why i want to have a fresh system there. putting back 1553 online should do someone else or i will do it when i find some time.
ivand
Posts: 310
Joined: 27 Jun 2007, 17:05

Re: The end of the maintenance branch

Post by ivand »

I'm actually exploring the same issue with my builds. We build on Ubuntu 18.04 worker and today someone popped up trying to run it on 16.04. Obviously it didn't run. I don't want to switch workers to 16.04 (though probably can). I tried to quickly hack the wpring ELF file to reference lower version of GLIBC_2.* to no avail.

From what I read static linking of glibc is discouraged and difficult to do practically. So @abma if you find some solution, please share.
Last edited by ivand on 20 Dec 2020, 18:51, edited 1 time in total.
User avatar
bibim
Lobby Developer
Posts: 952
Joined: 06 Dec 2007, 11:12

Re: The end of the maintenance branch

Post by bibim »

abma wrote: 20 Dec 2020, 18:30 glibc 2.29 was released 2019-01-31, ubuntu 20.04 has glibc 2.31, debian 10 is to old, debian 11 with an up to date glibc will be released in early 2021. On which distribution / version did you currently test?
Tbh my system is irrelevant, I don't personally need the static build of Spring engine to work on my system. But it is a fact that current Debian stable for example is widely used on headless servers (whether you find it suitable or not), and afaik the new static builds won't work (easily) on these systems.
abma wrote: 20 Dec 2020, 18:30the build system isn't touched this often: thats why i want to have a fresh system there.
Yes I totally understand that, thank you for working on this!
abma wrote: 20 Dec 2020, 18:30putting back 1553 online should do someone else or i will do it when i find some time.
But who has the access rights to do so ?
abma
Spring Developer
Posts: 3798
Joined: 01 Jun 2009, 00:08

Re: The end of the maintenance branch

Post by abma »

bibim wrote: 20 Dec 2020, 18:48
abma wrote: 20 Dec 2020, 18:30the build system isn't touched this often: thats why i want to have a fresh system there.
Yes I totally understand that, thank you for working on this!
abma wrote: 20 Dec 2020, 18:30putting back 1553 online should do someone else or i will do it when i find some time.
But who has the access rights to do so ?
afaik hokomoko, turboss, jk and dansan.
abma
Spring Developer
Posts: 3798
Joined: 01 Jun 2009, 00:08

Re: The end of the maintenance branch

Post by abma »

ivand wrote: 20 Dec 2020, 18:37 I'm actually exploring the same issue with my builds. We build on Ubuntu 18.04 worker and today someone popped up trying to run it on 16.04. Obviously it didn't run. I don't want to switch workers to 16.04 (though probably can). I tried to quickly hack the wpring ELF file to reference lower version of GLIBC_2.* to no avail.

From what I read static linking of glibc is discouraged and difficult to do practically. So @abma if you find some solution, please share.
you could try to use some alternative to glibc like musl, but IMHO then its more reasonable to create a flatpak with the spring engine.
abma
Spring Developer
Posts: 3798
Joined: 01 Jun 2009, 00:08

Re: The end of the maintenance branch

Post by abma »

"my" roadmap to 106.0:

https://springrts.com/wiki/Roadmap

questions/suggestions?
ivand
Posts: 310
Joined: 27 Jun 2007, 17:05

Re: The end of the maintenance branch

Post by ivand »

abma wrote: 20 Dec 2020, 21:08 questions/suggestions?
Short term I'm concerned with glibc version issue.
It might affect the adoption rate of the future stable as I envisioned 105 to be the engine for everyone.
Ideally 105 should be built with old build nodes, then after 105 is released, additional breaking changes could be imposed (such as glibc min version and deprecation of x32 builds). Actually x32 builds can probably be dropped even in 105.

Long term roadmap is not very clear, but I don't think any game can go straight to the develop without the transition phase, when new develop-compatible features are added game-side, but old ones keep working as before.
gajop
Moderator
Posts: 3051
Joined: 05 Aug 2009, 20:42

Re: The end of the maintenance branch

Post by gajop »

After 105 I'd like to move to GH actions for engine builds.

I found myself unable to fix anything despite having some permissions.

I managed to force builds but with no working builders that didn't work.
I managed to ssh into springfiles and witness various issues but I couldn't do anything due to lack of root permissions.

I'm also going to do something about springfiles and prd. Right now both are quite unstable and a constant source of problems. More troubling, they're built in outdated tech which needs a facelift - perhaps a rewrite?
I dislike that they also don't have clear instructions in how to setup (springfiles) or make builds (distributing prd should also be done via GH actions).
I think abma also shares some of my grievances regarding springfiles, and I applaud his idea of decoupling the frontend and backend (at least that's how I parsed his latest point: "make springfiles.com static").
I'd also further decouple the login/upload part from just serving data. Simplifying hosting should make it easy to spin up data hosting mirrors without having to care about setting up Drupal or whatever it is springfiles uses
abma
Spring Developer
Posts: 3798
Joined: 01 Jun 2009, 00:08

Re: The end of the maintenance branch

Post by abma »

meh, thanks bibim, i forgot the autohosts: having to upgrade to an "unstable" linux distribution for the autohosts isn't possible in most cases, so back to start: seems i have to temporary downgrade linux32/linux64 buildslaves for the 105.0 release. i've adjusted the roadmap.

wrt github actions: is this doable at all to create a linux static build as it currently is? the build environment is VERY custom because of the static libs and compiling all libs from scratch takes a lot of time and is very instable when downloads have to be made from multiple servers.

@gajop: wrt your plans, i guess we need to talk directly.
ivand
Posts: 310
Joined: 27 Jun 2007, 17:05

Re: The end of the maintenance branch

Post by ivand »

abma wrote: 21 Dec 2020, 14:03 wrt github actions: is this doable at all to create a linux static build as it currently is? the build environment is VERY custom because of the static libs and compiling all libs from scratch takes a lot of time and is very instable when downloads have to be made from multiple servers.
Yes it's possible.
* I have one GH actions script compiling linux static libs on 3 versions of Ubuntu OS and puts results into github repo.
* Another script creates win64, win32 and lin64 distribution very similar to what buildbot produces.
I'm not entirely happy about the script, because I had to do certain modifications to CMakeLists and because of the way the final zip is assembled, but it does its job: people play games with that engine and SL64 can recognize its unitsync.
abma
Spring Developer
Posts: 3798
Joined: 01 Jun 2009, 00:08

Re: The end of the maintenance branch

Post by abma »

ivand wrote: 21 Dec 2020, 14:16 * I have one GH actions script compiling linux static libs on 3 versions of Ubuntu OS and puts results into github repo.
* Another script creates win64, win32 and lin64 distribution very similar to what buildbot produces.
I'm not entirely happy about the script, because I had to do certain modifications to CMakeLists and because of the way the final zip is assembled, but it does its job: people play games with that engine and SL64 can recognize its unitsync.
links to the repos please!

so, the transition branch is history / could be dropped? (for historical reasons i would keep it)
ivand
Posts: 310
Joined: 27 Jun 2007, 17:05

Re: The end of the maintenance branch

Post by ivand »

https://github.com/beyond-all-reason/spring-static-libs
https://github.com/beyond-all-reason/spring
abma wrote: 21 Dec 2020, 14:29 so, the transition branch is history / could be dropped? (for historical reasons i would keep it)
No, not at all. BAR branch (the one above) is the testing ground for the transition API/implementation.

P.S. Here's the example of what result github actions produce: https://github.com/beyond-all-reason/sp ... 4-g321b911

P.P.S I also wanted to do similar thing for mingw libs, but was unable to in my first attempt and never had enough time and motivation later on.
abma
Spring Developer
Posts: 3798
Joined: 01 Jun 2009, 00:08

Re: The end of the maintenance branch

Post by abma »

ivand wrote: 21 Dec 2020, 15:00 No, not at all. BAR branch (the one above) is the testing ground for the transition API/implementation.
whats the plan with this branch then? should it be merged to the development branch at some point? (just trying to complete the "roadmap")
Locked

Return to “Off Topic Discussion”