rapid update: streamer.cgi is now from rapidtools

[abma]

finally i found the time to replace / completely migrate to RapidTools written by det. This replaces the "old" code (written in ocaml) with fresh c++ code.

all rapid downloads should be slightly faster, also the code is now written in c++ and should be much easier to maintain / can be easily integrated/reused in other tools. also setting up an own repo should be much easier, too.

i hope rapid downloads still works, if not, please report!


upstream project:

https://github.com/spring/RapidTools
wiki: http://springrts.com/wiki/RapidTools

Thanks a lot, det!

[jK]

why is the repo named RapidTools instead of RapidServer?

[abma]

as it contains 4 tools, "Streamer" is the only server-side only tool, other programs are (maybe) useful on client:

MakeZip for example can be used to create a zip from a .sdp.



https://github.com/spring/RapidTools/tree/master/Tools

[jK]

So how should someone who has a basic idea of what Rapid is, understand all those different repos, forks and tools?

I mean it's impossible except for you & det atm to even find the corresponding files, or to setup an own rapid server, or to fix bugs, ...

99.9% won't even know this topic means, they may know what streamer.cgi is, but not what rapidtools is nor what it changed (you don't link a commit or nothing).

For me and likely many others, Rapid is a blackbox same as Plasma, Springie, Spads, ... are. You may find a few examples here and there on the wiki, but you won't even find the actual used server code and so have zero possibilities to check the examples or use semi-documented features.

Also please, paste all spring fundamental tools on http://springrts.com/wiki/Links .

[abma]

So how should someone who has a basic idea of what Rapid is, understand all those different repos, forks and tools?

no, but feel free to ask. most people heard that "rapid" is for downloading games. currently the critism only motivates me to not post such a major change next time to the forum as its not constructive. how should i explain this? either someone is interested and tries to understand or ...not.

I mean it's impossible except for you & det atm to even find the corresponding files, or to setup an own rapid server, or to fix bugs, ...

i wouldn't say impossible, yes, its pretty hard. very few docs exist & tools have a small command line help.

99.9% won't even know this topic means, they may know what streamer.cgi is, but not what rapidtools is nor what it changed (you don't link a commit or nothing).

true, but this is the development forum. i wrote what was changed. the tools were completely rewritten in c++.

For me and likely many others, Rapid is a blackbox same as Plasma, Springie, Spads, ... are. You may find a few examples here and there on the wiki, but you won't even find the actual used server code and so have zero possibilities to check the examples or use semi-documented features.

Also please, paste all spring fundamental tools on http://springrts.com/wiki/Links .

done, but i dislike this wiki page in general, as it easily gets out of date. there should be a single page about one tool with links to it.
edit: fixed a bunch of links, most of them pointed to the old github repo url. changed links to link to wiki page where link is up to date.

actual server code is this: https://github.com/spring/RapidTools
old one is this: github.com/det/BuildRapid

[Anarchid]

how should i explain this? either someone is interested and tries to understand or ...not.

I think jK is campaigning towards the construction of some sort of a better doc.

Currently the README on that repo just contains a licence block and a wiki url.

[abma]

Currently the README on that repo just contains a licence block and a wiki url.

whats wrong with that? it doesn't make sense to copy the bad wiki site to the readme, or does it? feel free to improve the wiki page.

i know the content of the wiki page is bad, it doesn't improve the content by saying its bad again and again.

atm nobody seems to be interested in it / have questions related to it, so why solve a problem that doesn't exist? my todo-list is ways to long, i have much more important stuff to do than improving this very specific wiki page which should already contain many information.

please, only constructive critism! atm this is only bawing.

what should be improved, how and why?
also if you have questions to it, please ask specific. i can't explain the world.

[zwzsg]

atm nobody seems to be interested in it / have questions related to it, so why solve a problem that doesn't exist?

I'd have questions, but I'm so lost I don't even know where to begin.

- What is plasma?
- What is rapid?
- What is pr-downloader?
- Anything related?
- Which is outdated, which is up-to-date?
- On the client side, what files do I need?
- Where do I need to put them? Can it work without a Spring install?
- What's the commandline to list all files / tag / .. related to a mod, knowing only its name or abbreviation?
- Where are the files downloaded and stored in my computer?
- Is there a commandline to turn "file dust" (= innumerable files with indecipherable names that cannot be opened by anything but spring, lobbies, and downloader) into a zip, a 7z, or a sdd?


- When I update KP, do I still have to upload it to http://packages.springrts.com/ ?
- Does it still only accept sdz?
- How do I check the file I put there was properly upload and categorized?
- How do I download it back to check it worked?
- How do I mark it as the next stable?

... and that is only from a user point of view. From a dev pov like jK, it's prolly way more undocumented and confusing.

[abma]

- What is plasma?

http://springrts.com/wiki/Plasma

- What is rapid?

http://springrts.com/wiki/Rapid

- What is pr-downloader?

http://springrts.com/wiki/pr-downloader

- Anything related?

http://springrts.com/wiki/Category:Download_Systems

- Which is outdated, which is up-to-date?

see the wikipages. plasma is mainly used by zero-k. i personally would avoid zero-k services (=plasma) as devs mostly ignore feature/bug reports if they don't affect zero-k. all "download systems" are currently used, so all is up-to date.

- On the client side, what files do I need?

question is to unspecific. what do you want to do? you "need" no file on client side.

- Where do I need to put them? Can it work without a Spring install?

at least i can guess now what you want to do. with pr-downloader you can download engine/game/map, springlobby can do the same. With Makezip from rapid-tools you can create a .zip from a downloaded package from rapid.

- What's the commandline to list all files / tag / .. related to a mod, knowing only its name or abbreviation?

see the wiki

- Where are the files downloaded and stored in my computer?

should be in wiki as well. depends on the used tool.

- Is there a commandline to turn "file dust" (= innumerable files with indecipherable names that cannot be opened by anything but spring, lobbies, and downloader) into a zip, a 7z, or a sdd?

answered above

- When I update KP, do I still have to upload it to http://packages.springrts.com/ ?

if kernelpanic is inside an svn-repository it can be automaticly updated, see http://springrts.com/wiki/Rapid

for setting it up on server side you have to ask me or det.

- Does it still only accept sdz?

yes, nothing was changed, only the programs were replaced with a c++ implementation.

- How do I check the file I put there was properly upload and categorized?

download it? manual download http://packages.springrts.com/versions.gz and check its contents?

- How do I download it back to check it worked?

try download the package you've uploaded.

- How do I mark it as the next stable?

idk, i didn't write the upload thing. when using svn it works like this:
http://springrts.com/wiki/Rapid#creating_a_release

... and that is only from a user point of view. From a dev pov like jK, it's prolly way more undocumented and confusing.

i think that was clearly a game-dev pov. also next time please at least try to read a bit in the wiki before you ask.

[jK]

PLEASE guys I am not raging. I want to understand stuff and I don't want to annoy others with questions, so I want to be able to find the answers myself, which isn't possible atm even when you have vague ideas. Neither do I request any time consuming work by others at all.

Still the failures are repated many times in Spring's sub-projects. And those failure are extremely minor ones (like namings) but makes patches by others impossible.

So how should someone who has a basic idea of what Rapid is, understand all those different repos, forks and tools?

no, but feel free to ask. most people heard that "rapid" is for downloading games. currently the critism only motivates me to not post such a major change next time to the forum as its not constructive. how should i explain this? either someone is interested and tries to understand or ...not.

It seems you don't see, that no one understood what your topic means. It is in your interest ppl understand what you did and can so grant you the kudos you obviously deserve. And a link to commit is like "a picture paints a thousand words".

I mean it's impossible except for you & det atm to even find the corresponding files, or to setup an own rapid server, or to fix bugs, ...

i wouldn't say impossible, yes, its pretty hard. very few docs exist & tools have a small command line help.

how should i explain this? either someone is interested and tries to understand or ...not.

I think jK is campaigning towards the construction of some sort of a better doc.
Currently the README on that repo just contains a licence block and a wiki url.

No, I don't want any README or time consuming tutorials full-blown wiki pages and such crap. Users may want that but this is dev topic.
I want open & public links to sources of the tools used by infrastructure that runs this project. And those sources should be more or less readable.
Neither is true for Rapid atm, i.e. when opening the RapidTools repo you don't understand anything:

1. it says `tools`, what are tools? I would except apps to upload sdz or checking checksum integrity, but the server itself isn't a tool!
So why name the repo not just `Rapid`, then you can combine tools & server in the same repo w/o creating a confusion.
2. the server is in tools/Stream.cpp, again the server is no tool! it is the core of the whole thing!
When starting a server/client based infrastructure, you start with 3 folders src/server, src/client, src/common.
And later you can add src/tools for additional tools.

See I don't want that anyone invests more time, I just want that ppl invest it wisely. AND YES, I know this isn't your fault abma, you just maintain code others started, so these failures were made earlier.

Also please, paste all spring fundamental tools on http://springrts.com/wiki/Links .

done, but i dislike this wiki page in general, as it easily gets out of date. there should be a single page about one tool with links to it.
edit: fixed a bunch of links, most of them pointed to the old github repo url. changed links to link to wiki page where link is up to date.

As you say yourself wiki pages outdate w/o doing, a link to the source NEVER outdates. Also having a central page to find all important stuff that is used by this project is invaluable, esp. when there are so many sub-projects.

- Which is outdated, which is up-to-date?

see the wikipages. plasma is mainly used by zero-k. i personally would avoid zero-k services (=plasma) as devs mostly ignore feature/bug reports if they don't affect zero-k. all "download systems" are currently used, so all is up-to date.

It's more than the old deprecated torrent system, afaik it has interfaces that allow you to access map stats & minimap w/o downloading them etc. But all that stuff isn't documented nor is the source findable -> blackbox

- Where do I need to put them? Can it work without a Spring install?

at least i can guess now what you want to do. with pr-downloader you can download engine/game/map, springlobby can do the same. With Makezip from rapid-tools you can create a .zip from a downloaded package from rapid.
...

- What's the commandline to list all files / tag / .. related to a mod, knowing only its name or abbreviation?

see the http://springrts.com/wiki/Rapid_Tags

The old python rapid client had all that already integrated. Also the RapidTools repo contains c++ code for that, maybe I will make my first pull request for pr-downloader soon.

[abma]

It seems you don't see, that no one understood what your topic means. It is in your interest ppl understand what you did and can so grant you the kudos you obviously deserve. And a link to commit is like "a picture paints a thousand words".

did you read my post? there is no single commit. WHOLE STUFF was rewritten in C++, bevore it was OCAML. so every commit would count.

1. it says `tools`, what are tools? I would except apps to upload sdz or checking checksum integrity, but the server itself isn't a tool!
So why name the repo not just `Rapid`, then you can combine tools & server in the same repo w/o creating a confusion.

because there are 4 Programs (=Tools), AddZip, BuildSvn, MakeZip and Streamer.

2. the server is in tools/Stream.cpp, again the server is no tool! it is the core of the whole thing!
When starting a server/client based infrastructure, you start with 3 folders src/server, src/client, src/common.
And later you can add src/tools for additional tools.

det made that file structure, i didn't change that. "client" is maybe pr-downloader, but merging/cleaning up/refactoring wasn't done yet.
+1 for that change.

imo the rapid code can/should be used in spring, too. it could simplify many stuff, like make spring reading/understand rapid tags.

As you say yourself wiki pages outdate w/o doing, a link the source NEVER outdates. Also having a central page to find all important stuff that is used by this project is invaluable, esp. when they are so many sub-projects.

yep, true. but its more difficult to maintain when you've the same info multiple times in the wiki. so don't copy links to a single page, keep them on the dedicated page related to the tool/stuff. (links to the dedicated are fine, good and welcome, but a list of external links imo should be avoided if possible)

It's more than the old deprecated torrent system, afaik it has interfaces that allow you to access map stats & minimap w/o downloading them etc. But all that stuff isn't documented nor is the source findable -> blackbox

exactly! also its very likely it can/will change without any notice. sadly some functionality was dropped, but the python rapid clients code wasn't (re)usable in spring / for other download-tools or lobbies.

i know the docs/wiki pages related to rapid suck, but they contain information which should already explain most stuff. if you don't understand or don't find needed information, please ask.

[bibim]

For me and likely many others, Rapid is a blackbox same as Plasma, Springie, Spads, ... are. You may find a few examples here and there on the wiki, but you won't even find the actual used server code and so have zero possibilities to check the examples or use semi-documented features.

Can you explain exactly what makes you see SPADS as a blackbox?

[abma]

spads is on the wiki too:

http://springrts.com/wiki/SPADS

most stuff is on the wiki, this changed heavily recently. very few stuff has no wiki page, i guess some people missed that

[jK]

ohh sorry then, still it was missing on the central links page.

Have to correct then: SPADS is very very well documented :D