Page 1 of 1

Primer on "pool" vfs used for downloading mods

Posted: 28 Dec 2013, 07:46
by MajBoredom
I use linux. I noticed that when springlobby downloads a new mod, it doesn't put the .sdz file under my datadir in /mods or /games. Instead, it seems to break these files up into some kind of mysterious VFS/DB structure under the "pool" directory. (Is this correct?) This always gets put in ~/.spring even though I have a different datadir for each engine (run in portable mode).

In order for any of the engines to use the files auto-downloaded by springlobby, this seems to mean that I need ~/.spring as a read-only path for every engine.

Before I roll up my sleeves and find out how the pool directory works, can someone provide a primer or basic explanation of how this directory serves files to the engine? Who owns this code? Would it be very difficult to make springlobby use a different "pool" for each of its engines? Or configure each engine to use its own read/write datadir? This seems to be the intended usage for portable mode (man spring) but springlobby seems to assume a single datadir for all engines. It would be favourable if there was an option to use a global datadir or a specific one for each engine. But I should put this into the tracker.

Re: Primer on "pool" vfs used for downloading mods

Posted: 28 Dec 2013, 09:25
by Forboding Angel
Use symlinks to make the locations be where ever you like?

Re: Primer on "pool" vfs used for downloading mods

Posted: 29 Dec 2013, 03:23
by SirMaverick
MajBoredom wrote:I noticed that when springlobby downloads a new mod, it doesn't put the .sdz file under my datadir in /mods or /games. Instead, it seems to break these files up into some kind of mysterious VFS/DB structure under the "pool" directory. (Is this correct?)
Yes. This download system is called rapid. It was invented to reduce download sizes. Instead of downloading the whole sdz file you just download the difference to the old version. Those parts are stored in the pool/package directory. Package for index files (which files does a game contain, pool for the actual game files).
This always gets put in ~/.spring even though I have a different datadir for each engine (run in portable mode).

In order for any of the engines to use the files auto-downloaded by springlobby, this seems to mean that I need ~/.spring as a read-only path for every engine.
Not sure how to configure the download destination. As mentioned above symlink to the download directory or have all other engines use this directory as additional datadir.

More info about rapid can be found here:
http://springrts.com/wiki/Rapid
http://springrts.com/phpbb/viewtopic.php?f=67&t=22077

Re: Primer on "pool" vfs used for downloading mods

Posted: 29 Dec 2013, 03:42
by smoth
MajBoredom wrote:man spring
Damn straight!

Re: Primer on "pool" vfs used for downloading mods

Posted: 29 Dec 2013, 12:13
by abma
the pool folder is used for the "rapid" download system.

using a dedicated "pool" folder for each engine breaks the purpose of it as it shares many files between different game versions. for example an update from BA 7.89 to 7.90 requires only the changed files to be downloaded. when using .sdz the whole archive needs to be downloaded, in numbers thats something like dl size of 20MB vs 20KB. (also the difference in local storage used is the same)

if you split up the pool folder to each engine version, every time you need the whole pool "archive" which is slightly bigger than a .sdz. also download is much slower as only one mirror exists. locally much more space is used as well. you really don't want this.


atm there are three solutions for your problem:

1. move the pool/packages/<wanted game version>.sdp to the specific engine folder
2. use console pr-downloader --filesystem-writepath ~/.spring/engine/<version> <game to download>
3. implement a way in springlobby to set the filesystem-writepath. as already said this will very likely break the purpose of "rapid"
4. change your mind that having a "global" pool folder is bad. why do you want to change that? maybe some other solution exists for your problem.