Primer on "pool" vfs used for downloading mods

Primer on "pool" vfs used for downloading mods

Discuss development of lobby clients, server, autohosts and auto-download software.

Moderators: Moderators, Lobby Developers

Post Reply
MajBoredom
Posts: 17
Joined: 24 Nov 2013, 09:14

Primer on "pool" vfs used for downloading mods

Post by MajBoredom »

I use linux. I noticed that when springlobby downloads a new mod, it doesn't put the .sdz file under my datadir in /mods or /games. Instead, it seems to break these files up into some kind of mysterious VFS/DB structure under the "pool" directory. (Is this correct?) This always gets put in ~/.spring even though I have a different datadir for each engine (run in portable mode).

In order for any of the engines to use the files auto-downloaded by springlobby, this seems to mean that I need ~/.spring as a read-only path for every engine.

Before I roll up my sleeves and find out how the pool directory works, can someone provide a primer or basic explanation of how this directory serves files to the engine? Who owns this code? Would it be very difficult to make springlobby use a different "pool" for each of its engines? Or configure each engine to use its own read/write datadir? This seems to be the intended usage for portable mode (man spring) but springlobby seems to assume a single datadir for all engines. It would be favourable if there was an option to use a global datadir or a specific one for each engine. But I should put this into the tracker.
User avatar
Forboding Angel
Evolution RTS Developer
Posts: 14673
Joined: 17 Nov 2005, 02:43

Re: Primer on "pool" vfs used for downloading mods

Post by Forboding Angel »

Use symlinks to make the locations be where ever you like?
SirMaverick
Posts: 834
Joined: 19 May 2009, 21:10

Re: Primer on "pool" vfs used for downloading mods

Post by SirMaverick »

MajBoredom wrote:I noticed that when springlobby downloads a new mod, it doesn't put the .sdz file under my datadir in /mods or /games. Instead, it seems to break these files up into some kind of mysterious VFS/DB structure under the "pool" directory. (Is this correct?)
Yes. This download system is called rapid. It was invented to reduce download sizes. Instead of downloading the whole sdz file you just download the difference to the old version. Those parts are stored in the pool/package directory. Package for index files (which files does a game contain, pool for the actual game files).
This always gets put in ~/.spring even though I have a different datadir for each engine (run in portable mode).

In order for any of the engines to use the files auto-downloaded by springlobby, this seems to mean that I need ~/.spring as a read-only path for every engine.
Not sure how to configure the download destination. As mentioned above symlink to the download directory or have all other engines use this directory as additional datadir.

More info about rapid can be found here:
http://springrts.com/wiki/Rapid
http://springrts.com/phpbb/viewtopic.php?f=67&t=22077
User avatar
smoth
Posts: 22309
Joined: 13 Jan 2005, 00:46

Re: Primer on "pool" vfs used for downloading mods

Post by smoth »

MajBoredom wrote:man spring
Damn straight!
abma
Spring Developer
Posts: 3798
Joined: 01 Jun 2009, 00:08

Re: Primer on "pool" vfs used for downloading mods

Post by abma »

the pool folder is used for the "rapid" download system.

using a dedicated "pool" folder for each engine breaks the purpose of it as it shares many files between different game versions. for example an update from BA 7.89 to 7.90 requires only the changed files to be downloaded. when using .sdz the whole archive needs to be downloaded, in numbers thats something like dl size of 20MB vs 20KB. (also the difference in local storage used is the same)

if you split up the pool folder to each engine version, every time you need the whole pool "archive" which is slightly bigger than a .sdz. also download is much slower as only one mirror exists. locally much more space is used as well. you really don't want this.


atm there are three solutions for your problem:

1. move the pool/packages/<wanted game version>.sdp to the specific engine folder
2. use console pr-downloader --filesystem-writepath ~/.spring/engine/<version> <game to download>
3. implement a way in springlobby to set the filesystem-writepath. as already said this will very likely break the purpose of "rapid"
4. change your mind that having a "global" pool folder is bad. why do you want to change that? maybe some other solution exists for your problem.
Post Reply

Return to “Lobby Clients & Server”