refactoring repos.springrts.com (rapid repositories)
Moderator: Moderators
refactoring repos.springrts.com (rapid repositories)
Ive tried to refactor repos.springrts.com:
old:
http://ba.repo.springrts.com/
http://zk.repo.springrts.com/
...
new:
http://repos.springrts.com/ba/
http://repos.springrts.com/zk/
http://repos.springrts.com/tchobby/
...
also packages.springrts.com and repos.springrts.com was split, before it pointed to the same stuff.
atm only tchobby inside http://repos.springrts.com/repos.gz points to the new url schema to test if clients can handle it.
this change doesn't hopefully break anything: if so, please report here!
thanks!
you may ask why:
each subdomain requires a new dns request + http connection to be made. this can be painfully slow. when all repos are at the same subdomain, only one request has to be made and this should be much faster + less error-prone. also this allows to add / remove repos without reconfiguring apache and easily allows to use https.
also WTF: 42 git repos are hosted on springrts.com!
old:
http://ba.repo.springrts.com/
http://zk.repo.springrts.com/
...
new:
http://repos.springrts.com/ba/
http://repos.springrts.com/zk/
http://repos.springrts.com/tchobby/
...
also packages.springrts.com and repos.springrts.com was split, before it pointed to the same stuff.
atm only tchobby inside http://repos.springrts.com/repos.gz points to the new url schema to test if clients can handle it.
this change doesn't hopefully break anything: if so, please report here!
thanks!
you may ask why:
each subdomain requires a new dns request + http connection to be made. this can be painfully slow. when all repos are at the same subdomain, only one request has to be made and this should be much faster + less error-prone. also this allows to add / remove repos without reconfiguring apache and easily allows to use https.
also WTF: 42 git repos are hosted on springrts.com!
Re: Experimenting with refactoring repos.springrts.com
But what was the question again - you know, the one question, the question - to the meaning of life, the answer to everything- you know the question
Re: Experimenting with refactoring repos.springrts.com
meh, found a bug: streamer users DOCUMENT_ROOT: https://github.com/spring/pr-downloader ... er.cpp#L98
but it should use DOCUMENT_ROOT + SCRIPT_NAME i guess...
but it should use DOCUMENT_ROOT + SCRIPT_NAME i guess...
Re: Experimenting with refactoring repos.springrts.com
Why are we using apache? Why not nginx?
Re: Experimenting with refactoring repos.springrts.com
Why can't we use url:s like https://repos.springrts.com/zk/ instead?abma wrote: ↑06 Mar 2019, 20:13
new:
http://repos.springrts.com/ba/
http://repos.springrts.com/zk/
http://repos.springrts.com/tchobby/
...
also WTF: 42 git repos are hosted on springrts.com!
- Forboding Angel
- Evolution RTS Developer
- Posts: 14673
- Joined: 17 Nov 2005, 02:43
Re: Experimenting with refactoring repos.springrts.com
Yeah, personally I don't see any benefit to using subdomains for this. Subfolders make a lot more sense.
- Silentwings
- Posts: 3720
- Joined: 25 Oct 2008, 00:23
Re: Experimenting with refactoring repos.springrts.com
I think you will be able to use https, just not yet.Why can't we use url:s like https://repos.springrts.com/zk/ instead?
Re: Experimenting with refactoring repos.springrts.com
yes, thats the plan! i don't want to break all at once, this is why https isn't enabled yet.
Re: Experimenting with refactoring repos.springrts.com
ok, fixed a blocking bug. https://github.com/spring/pr-downloader/issues/113
downloading should now work via http://repos.springrts.com/tchobby at least. can you please test it if you can download "tchobby:test", too?
when i hear no problem/bug reports, i'll change all repos in repos.gz to the new urls.
downloading should now work via http://repos.springrts.com/tchobby at least. can you please test it if you can download "tchobby:test", too?
when i hear no problem/bug reports, i'll change all repos in repos.gz to the new urls.
Re: Experimenting with refactoring repos.springrts.com
nothing heard -> https://github.com/spring/pr-downloader ... e-repos.py
all urls in repos.gz now point to http://repos.springrts.com/
all urls in repos.gz now point to http://repos.springrts.com/
Re: Experimenting with refactoring repos.springrts.com
Does rapid work for anyone?
Is the streamer streaming itself?
Code: Select all
/opt/springrts.com/spring/104.0/bin/pr-downloader --filesystem-writepath /data/users/s44/spads/game_data --rapid-download s44:test
pr-downloader tarball
[Info] Using filesystem-writepath: /data/users/s44/spads/game_data
[Info] Free disk space: 10902 MB
[Info] Found 42 repos in /data/users/s44/spads/game_data/rapid/repos.springrts.com/repos.gz
[Download] Spring: 1944 test-5448-ae935de
[Info] Using rapid
[Info] http://repos.springrts.com/s44/streamer.cgi?debae699faac80db2422860cefacee09
[Progress] 2% [ ] 14271/925208 ^M[Progress] 100% [==============================] 925208/925208 ^M[Info] Missing file: /data/users/s44/spads/game_data/pool/9a/0b8dce4cf805ddcaedb0a6a9341736.gz
[Info] compare failed(): 96c51cd2ee6c2a7dc2c0717afc336974 569168b394351c7eda9442de43f7af4f
[Info] Removing invalid file: /data/users/s44/spads/game_data/pool/56/9168b394351c7eda9442de43f7af4f.gz
[Info] Missing file: /data/users/s44/spads/game_data/pool/38/2f9e03e4de37425f4ab5a4a18b8a2e.gz
[Info] Missing file: /data/users/s44/spads/game_data/pool/38/2f9e03e4de37425f4ab5a4a18b8a2e.gz
wget -O /tmp/test 'http://repos.springrts.com/s44/streamer.cgi?debae699faac80db2422860cefacee09'
file /tmp/test
/tmp/test: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 2.6.32, BuildID[sha1]=568ee4e3ef19f67c55f9d32b3e7f8165bf4bbed2, with debug_info, not stripped
SHA1 b147f1862236bf696a1eda0bcf4fdb0cec496e2c /tmp/test
Is the streamer streaming itself?
Re: Experimenting with refactoring repos.springrts.com
ugh, weird, when testing it worked for me
Re: Experimenting with refactoring repos.springrts.com
should be fixed now! apache config was invalid, sorry
the regex was wrong: https://github.com/spring/pr-downloader ... e.conf#L18
-
- Posts: 12
- Joined: 14 Aug 2014, 00:12
Re: Experimenting with refactoring repos.springrts.com
Edit: ZK has its own downloader that failed due to this update.
-
- Moderator
- Posts: 2464
- Joined: 12 Oct 2007, 09:24
Re: Experimenting with refactoring repos.springrts.com
It looks like singleplayer was broken for 6 hours, during which we received about 60 crash reports. It would be nice if repository names didn't change around.
Re: Experimenting with refactoring repos.springrts.com
could you provide more details please? the names did not change, the url of the individual repos did change. When the rapid client is implemented probably, it should have worked without any changes. It was announced and a test repo was added also the old urls still work, too.Google_Frog wrote: ↑28 Mar 2019, 13:06 It looks like singleplayer was broken for 6 hours, during which we received about 60 crash reports. It would be nice if repository names didn't change around.
the entry url did not change: http://repos.springrts.com/repos.gz which contains the (new) prefixes to the individual repos.
-
- Moderator
- Posts: 2464
- Joined: 12 Oct 2007, 09:24
Re: Experimenting with refactoring repos.springrts.com
I don't know details. Apparently this change fixes it: https://github.com/ZeroK-RTS/Zero-K-Inf ... d5a512be17
Re: Experimenting with refactoring repos.springrts.com
improving / fixing these:
abma wrote: ↑06 Mar 2019, 20:13 each subdomain requires a new dns request + http connection to be made. this can be painfully slow. when all repos are at the same subdomain, only one request has to be made and this should be much faster + less error-prone. also this allows to add / remove repos without reconfiguring apache and easily allows to use https.
was not possible without changing the repos urls.
- Silentwings
- Posts: 3720
- Joined: 25 Oct 2008, 00:23
Re: refactoring repos.springrts.com (rapid repositories)
Which urls should we now use to get hold of the sdz built by rapid for major versions?
I don't see any (or even a blank list) on http://repos.springrts.com/ba/, all I get there is a link to the rapid wiki page
I don't see any (or even a blank list) on http://repos.springrts.com/ba/, all I get there is a link to the rapid wiki page