Best practices for multi-hosting

Best practices for multi-hosting

SpringRTS Perl Autohost for Dedicated Server

Moderators: Moderators, Lobby Developers, SPADS AutoHost

Post Reply
User avatar
Jools
XTA Developer
Posts: 2816
Joined: 23 Feb 2009, 16:29

Best practices for multi-hosting

Post by Jools »

I also wonder what is the best practice if you want to host two instances of spads, ie two autohosts. Can they use same /etc directory but different /var? Or can the use same var directory and therefore share things like saved mapboxes etc?

What about same plugins folder? Should plugins then be written so that they use the varDir for storing and retrieving things and not the pluginsdir?
User avatar
bibim
Lobby Developer
Posts: 952
Joined: 06 Dec 2007, 11:12

Re: SPADS AutoHost

Post by bibim »

Jools wrote:I also wonder what is the best practice if you want to host two instances of spads, ie two autohosts. Can they use same /etc directory but different /var? Or can the use same var directory and therefore share things like saved mapboxes etc?

What about same plugins folder? Should plugins then be written so that they use the varDir for storing and retrieving things and not the pluginsdir?
The only directory which has to be different between 2 autohosts is the var instanceDir (*) directory, all other files can be shared between several instances (binary files, conf files, plugins etc.). So you don't need to install SPADS several times to host multiple autohosts of course.

Plugins should indeed store their persistent data into the var instanceDir (*) directory, like other SPADS persistent data, so that they won't conflict when used on several instances at the same time.

(*)
Since SPADS 0.12+, the directory used to store instance-specific dynamic data is "instanceDir" instead of "varDir" .
The "varDir" directory is now used to store dynamic data shared between all running instances.
User avatar
ThinkSome
Posts: 387
Joined: 14 Jun 2015, 13:36

Re: Best practices for multi-hosting

Post by ThinkSome »

I have a couple of questions:

1) Have you changed demo storage location from springDataDir/demos-server to varDir/demos-server?


2) As you said in PM, I can do

Code: Select all

spads.pl /path/to/spads.conf NAME=blabla
and it will replace all instances of %NAME% with blabla. Is it possible to pass a file full of these definitions to spads.pl? Furthermore, is inline math possible? e.g. if I have 3 autohosts, I want to do:

Code: Select all

./spads.pl /path/to/shared/spads.conf pathtodefintionsfile INSTANCE=3
and then something like

Code: Select all

SHAREDNAME=WorldAtWarII_rapid
BASEPORT=9000
BASEAHPORT=10000
NAME=%SHAREDNAME + INSTANCE% #as string concatenation
PORT=%BASEPORT + INSTANCE% # as integer addition
AUTOHOSTPORT=%BASEAHPORT + INSTANCE % # integer addition
in the definitions file. Is this possible? Does spads implement its own "templating" or does it use something from perl (not familiar here)? If the latter, is there a tutorial on its capabilities?

3) Finaly, can updating game_data (e.g. git repo) while a game is running from old checkout somehow negatively affect it?
User avatar
Jools
XTA Developer
Posts: 2816
Joined: 23 Feb 2009, 16:29

Re: Best practices for multi-hosting

Post by Jools »

Spads has its own config stuff, here is the source: https://github.com/Yaribz/SPADS/blob/ma ... adsConf.pm
User avatar
bibim
Lobby Developer
Posts: 952
Joined: 06 Dec 2007, 11:12

Re: Best practices for multi-hosting

Post by bibim »

ThinkSome wrote:Have you changed demo storage location from springDataDir/demos-server to varDir/demos-server?
Not directly, this is a consequence of this change Spring read-write dir is now set to SPADS var dir in Spads 0.11.26.

The read/write directory is used by Spring engine to write logs, unitsync cache, springsettings.cfg configuration file, demo files etc. (the "demos-server" sub-directory is actually created by the Spring server binary itself, not by SPADS).

The reason for this change is that multiple SPADS autohosts may use same Spring data directory, and in this case logs could be overwritten, especially when 2 Spring servers using same data directory are running at the same time. So this directory has to be unique for each autohost, which is the purpose of the SPADS varDir instanceDir (*) directory (instance-specific variable data directory).

(*)
Since SPADS 0.12+, the directory used to store instance-specific dynamic data is "instanceDir" instead of "varDir" .
The "varDir" directory is now used to store dynamic data shared between all running instances.
User avatar
bibim
Lobby Developer
Posts: 952
Joined: 06 Dec 2007, 11:12

Re: Best practices for multi-hosting

Post by bibim »

ThinkSome wrote:As you said in PM, I can do

Code: Select all

spads.pl /path/to/spads.conf NAME=blabla
and it will replace all instances of %NAME% with blabla. Is it possible to pass a file full of these definitions to spads.pl? Furthermore, is inline math possible? e.g. if I have 3 autohosts, I want to do:

Code: Select all

./spads.pl /path/to/shared/spads.conf pathtodefintionsfile INSTANCE=3
and then something like

Code: Select all

SHAREDNAME=WorldAtWarII_rapid
BASEPORT=9000
BASEAHPORT=10000
NAME=%SHAREDNAME + INSTANCE% #as string concatenation
PORT=%BASEPORT + INSTANCE% # as integer addition
AUTOHOSTPORT=%BASEAHPORT + INSTANCE % # integer addition
in the definitions file. Is this possible? Does spads implement its own "templating" or does it use something from perl (not familiar here)?
It's not possible to pass a file full of these definitions and inline math is not possible, but it's still possible to do something very similar to what you want.

SPADS implements its own minimal templating system based on 2 functionalities that I call "configuration macros" and "configuration includes":
  1. "Configuration macros" are the fact of passing definitions as command line parameters when launching SPADS. For instance:

    Code: Select all

    perl spads.pl etc/spads.conf macro1=value1 macro2=value2 ...
    In this case, SPADS will replace any occurrence of "%macro1%", "%macro2%"(...) by values "value1", "value2"(...) in all its configuration files before processing them. These "%macro%" strings can be placed really everywhere (in setting names, in setting values, in preset names...).
  2. "Configuration includes" are the fact of including configuration files inside other configuration files, by using a line respecting "{<subConfigFileName>}" syntax. For instance:

    Code: Select all

    ...
    mapList:all
    {test.conf}
    map:DeltaSiegeDry
    ...
    In this case, SPADS will include the content of the "test.conf" file in lieu of the "{test.conf}" line before processing the configuration file. File names can be absolute, or relative to the path of the configuration file containing the include.
The interesting part is that these 2 functionalities can be combined. For instance if you launch SPADS like this:

Code: Select all

perl spads.pl etc/spads.conf pool=2 instance=3
And you have a config file containing this:

Code: Select all

{pool_%pool%.conf}
autoHostPort:8%pool%%instance%4
And another config file named "pool_2.conf" lying in the same directory containing this:

Code: Select all

lobbyLogin:secondPool%instance%
Then SPADS will parse it as:

Code: Select all

lobbyLogin:secondPool3
autoHostPort:8234
Unless I missed something in your question, I guess you should be able to apply this method to do exactly what you want with minimal macro definitions provided in command line arguments.
User avatar
bibim
Lobby Developer
Posts: 952
Joined: 06 Dec 2007, 11:12

Re: Best practices for multi-hosting

Post by bibim »

ThinkSome wrote:can updating game_data (e.g. git repo) while a game is running from old checkout somehow negatively affect it?
By "game_data" you mean files in Spring data directories? At SPADS side it will only trigger an automatic reload of archives if configured so, and an automatic rehost when empty if current mod has been updated and if configured so, but this won't have any impact on the running game. At Spring server side I don't think it has any negative effect neither, but this question should go to engine devs.
User avatar
ThinkSome
Posts: 387
Joined: 14 Jun 2015, 13:36

Re: Best practices for multi-hosting

Post by ThinkSome »

Yes, springDataDir
User avatar
bibim
Lobby Developer
Posts: 952
Joined: 06 Dec 2007, 11:12

Re: Best practices for multi-hosting

Post by bibim »

What is written below concerning concurrent SPADS instances sharing same SPADS binaries (i.e., started from same SPADS installation directory) is no longer true starting with SPADS version 0.12.12: it is no longer a recommendation to have the autoUpdateRelease setting set on only one of these SPADS instances. SPADS has become smarter: if several instances are configured to auto-update the same installation directory then it will dynamically and continuously select one and only one instance which will be responsible for the auto-updates. However if several instances have the autoUpdateRelease setting set, they still need to use the same value for this setting of course, otherwise each instance will fight for updating the installation directory to its specific desired release during the auto-update-at-startup process.

Another important point concerning multi-hosting with SPADS, which I explained to some autohost admins but forgot to write here it seems:

When you share same SPADS binaries between multiple SPADS instances (i.e., using same SPADS installation directory to launch SPADS multiple times), only one of these SPADS instances should have the autoUpdateRelease setting set. That way, only one instance is auto-updating the shared SPADS binaries. The other instances will benefit from the SPADS updates automatically anyway, provided you kept the autoRestartForUpdate setting enabled for them.

Having the autoUpdateRelease set on multiple instances sharing the same SPADS binaries can lead to race conditions when several instances try to update the same files at the same time. It can be even worse if these SPADS instances have different values of autoUpdateRelease of course...
Post Reply

Return to “SPADS AutoHost”