SPADS AutoHost

[nixtux]

OMG OMG thank you, you deserve some cookies for this

[FabriceFABS]

Excellent... Very nice job.
Merci beaucoup et encore merci pour le support.

Fabrice

[bibim]

You're welcome, don't hesitate to ask if you need help with the upgrade process and new version configuration.

[Jools]

Same here. Vielen Dank für das ausständige Arbeit!

[Forboding Angel]

Your upgrade instructions are hard to follow, especially for anyone running multiple spads instances.

I made them marginally easier to parse: https://pastebin.com/FQsu1sbx

[Forboding Angel]

PSA: Atm it is impossible to get the 32bit version of activeperl (Which is what I've always used) from the ActiveState site. I don't know if it will stay like that, but in case anyone is worried, strawberry seems to work 100% perfectly.

[bibim]

PSA: Atm it is impossible to get the 32bit version of activeperl (Which is what I've always used) from the ActiveState site. I don't know if it will stay like that, but in case anyone is worried, strawberry seems to work 100% perfectly.

Indeed, ActivePerl download page seems to be missing 32bit versions currently. They are still available from official ActiveState download site though (ActivePerl 5.24 Win32 x86, ActivePerl 5.26 Win32 x86). Thanks for the information.

Anyway, as you said SPADS should work as well with Strawberry Perl as with ActivePerl.

[bibim]

Here are two diagrams to help autohost administrators understand how SPADS settings and presets are organized in the 3 main configuration files: spads.conf, hostingPresets.conf and battlePresets.conf. It is important to understand these 3 files contain different settings, you can't move settings from spads.conf to battlePresets.conf for example.

SPADS settings and presets structure
SPADS presets dependencies

The presets structures are just examples of course, lots of things are possible...

[bibim]

Experimental support for TLS has been added in SPADS 0.12.6 (only in "unstable" release for now).
You can try it by adding "lobbyTls=on" as command line parameter when launching SPADS.

[Jools]

Do we need out our certificates? I know that Let's Encrypt provides that...

[bibim]

Not sure if this answers your question, but afaik currently only (self-signed) server side certificates are used in lobby protocol encryption, so you don't need your own certificates to run a SPADS instance.
SPADS does the same as SpringLobby: it validates the server certificate based on a trusted certificate fingerprint.

[bibim]

Starting with SPADS version 0.12.7 (only in unstable SPADS release for now), TLS is enabled by default for lobby server connection if it is supported by your Perl environment. You can still disable TLS for lobby server connection if you want by adding "lobbyTls=off" as command line parameter when you launch SPADS.

TLS support requires a specific Perl module named IO::Socket::SSL.

On Windows, this module is already included by default in your Perl environment so you shouldn't have anything to do to enable TLS.

On Linux, you may have to install this module manually, depending on your distribution. On main Linux distributions, the installation of this module can be performed through the standard system package manager. For example for Debian based distributions (Ubuntu...):

Code: Select all

apt-get install libio-socket-ssl-perl

For RPM-based distributions (RedHat, CentOS...):

Code: Select all

yum install perl-IO-Socket-SSL
	(or)
dnf install perl-IO-Socket-SSL

More generally, it is also possible to install this module without using the system package manager, by using the Perl "cpan" tool:

Code: Select all

cpan IO::Socket::SSL

[bibim]

A small update concerning multi-hosting (concurrent SPADS instances sharing same binaries, i.e. started from same installation directory):

Starting with SPADS version 0.12.12 (only in unstable SPADS release for now) it is no longer a recommendation to have the autoUpdateRelease setting set on only one of these SPADS instances. SPADS has become smarter: if several instances are configured to auto-update the same installation directory then it will dynamically and continuously select one and only one instance which will be responsible for the auto-updates. However if several instances have the autoUpdateRelease setting set, they still need to use the same value for this setting of course, otherwise each instance will fight for updating the installation directory to its specific desired release during the auto-update-at-startup process.

[Jools]

Would it be possible to build a version of spads also for Arm (armhf)? Did I already ask?

[bibim]

Would it be possible to build a version of spads also for Arm (armhf)? Did I already ask?

If you have all the requirements listed in the first post of this thread on your system, there is no reason SPADS shouldn't work.
Did you try ?

[bibim]

I have introduced 2 new functionalities in SPADS 0.12.17 to ease configuration management: configuration settings overrides and preset inheritance. I will explain them in following posts.

[bibim]

Configuration settings overrides

TL;DR: It is now possible to override settings configured in SPADS configuration files by passing command line parameters when starting SPADS. For example to override the "lobbyLogin" global setting and the "port" hosting setting: "perl spads.pl etc/spads.conf set:lobbyLogin=myAutohost hSet:port=9452"

1. Reminder: the already existing configuration macro system
   
   The new "configuration setting override" functionality is based on SPADS "configuration macro" system, so first here is a reminder regarding this system. The configuration macro system is a basic templating system which allows you to put placeholders (strings between "%") anywhere in SPADS configuration files. These placeholders will then be replaced by the actual values provided as command line parameters when launching SPADS.
   
   For example, instead of putting "lobbyLogin:myAutohost" in the spads.conf file, you could put "lobbyLogin:myAutohost%TEST%" and start SPADS with following extra command line parameter: "TEST=3".
   SPADS will then behave as if "lobbyLogin:myAutohost3" was written in spads.conf.
   
   Of course, you can use as many configuration macros as you like, and these configuration macros are persistent over configuration reloads (!reloadConf) and restarts (!restart). Additionally, when calling !reloadConf or !restart command, you can optionally redefine these configuration macros (refer to !reloadConf and !restart commands documentation).
   
   This configuration macro system is especially useful for multi-instance mode (i.e. when several instances of SPADS are launched from same SPADS installation directory), because when you launch several SPADS instances simultaneously a few configuration settings need to be set to different values between these instances ("lobbyLogin" and "autoHostPort" for example). Instead of duplicating SPADS configuration files entirely just to set these few settings to the specific values of each instance, it's much more efficient to share the same configuration file and use configuration macros to provide the specific parts as command line parameters. See this post for further examples.
    
2. The new configuration setting override functionality
   
   The configuration macro system presented above may lack some flexibility for some usages:
   • if you start putting placeholders for configuration macros in your configuration files, then you can no longer start SPADS without providing the corresponding macro definitions as command line parameters
   • reciprocally, if you didn't plan in advance to put a placeholder for a specific setting in your configuration file, then you can't use the configuration macro system to redefine it on the fly (you need to perform modifications in the configuration file and then !reloadConf or !restart SPADS with the correct macro definition)
   
   The configuration setting override functionality has been introduced in SPADS 0.12.17 to overcome these limitations: it provides a way to bypass entirely some configuration values set in SPADS configuration files. Configuration settings overrides are just like normal configuration macro declarations (i.e. "<name>=<value>" strings provided as additional command line parameters) but with specific prefixes in the <name> part:
   • the "set:" prefix allows you to override global settings (defined in spads.conf)
   • the "hSet:" prefix allows you to override hosting settings (defined in hostingPresets.conf).
   
   Of course you can use both functionalities (configuration macros and settings overrides) at the same time. For example, let's say the spads.conf file contains following declarations:

   Code: Select all

   lobbyLogin:oldName[%ID%]
   autoHostPort:100%ID%0
   defaultPreset:team

   Then if you launch SPADS like this "perl spads.pl etc/spads.conf ID=1", it will behave as if spads.conf contained:

   Code: Select all

   lobbyLogin:oldName[1]
   autoHostPort:10010
   defaultPreset:team

   But if you launch SPADS like this "perl spads.pl etc/spads.conf ID=2 set:lobbyLogin=newName set:defaultPreset=duel", then it will behave as if spads.conf contained:

   Code: Select all

   lobbyLogin:newName
   autoHostPort:10020
   defaultPreset:duel

   
   The configuration setting override system is clearly not a replacement for the standard configuration macro system, both have their benefits and limitations, and both have their typical use cases.
   For instance the configuration setting override system is particularly useful for SPADS plugins managing other SPADS instances, because in this case they can control the configuration of the other running instances in a non-invasive way (without requiring to change the configuration of the currently running instance).

[bibim]

Preset inheritance

TL;DR: It is now possible to make a preset inherit settings declarations from other presets, just by adding the list of inherited presets between "< >" in the preset declaration line in the configuration file. For example, for a "duelProOnly" preset inheriting from "proOnly", "duel" and "default" presets: "[duelProOnly<proOnly,duel,default>]"

1. Reminder: SPADS configuration settings and presets
   
   SPADS configuration settings are organized in several layers:
   • The global settings are the settings defined in the first part of the spads.conf file. They are declared only once and their values cannot be changed while SPADS is running. One of these global settings is "defaultPreset", which indicates which global preset is loaded by default on startup.
   • The global presets are declared after the global settings in spads.conf. Each preset is declared by a "[presetName]" line followed by the list of the settings declarations contained in this preset. Each preset can contain the same settings but with different values. Such settings contained in global presets are called "preset settings", they can be changed while SPADS is running. One of these preset settings is "hostingPreset", which indicates which hosting preset is loaded by default by this global preset. Another one of these preset settings is "battlePreset", which indicates which battle preset is loaded by default by this global preset.
   • The hosting presets are declared in the hostingPresets.conf file in the same way as global presets. The settings contained in hosting presets are called hosting settings and are related to the battleroom opened in the lobby to host the game (maximum number of players in battleroom, battleroom password...)
   • The battle presets are declared in the battlePresets.conf file, in the same way as global presets. The settings contained in battle presets are called battle settings and are related to the game and map settings to configure the battle (start position type, modoptions, mapoptions ...)
   For more information regarding SPADS settings and presets, refer to these diagrams:
   SPADS settings and presets structure
   SPADS presets dependencies
   
   An important point regarding SPADS configuration is that all the settings need to be declared in the configuration files, i.e. there is no by-default value: SPADS raises an error if a setting is missing in its configuration files.
   This means that the default preset must contain the full list of the preset settings declarations. Such a preset is called a "full preset". But this is not a requirement for the other presets. The non-default presets can contain only a subset of the preset settings, basically only the settings which needs to be adjusted when switching to this preset. Such presets are called "partial presets" because they only contain a partial list of the possible settings.
   
   For example let's say the spads.conf file contains following declarations:

   Code: Select all

   ...
   defaultPreset:team
   ...
   [team]
   description:Team preset
   preset:team|duel
   ...
   ...(all other preset settings declarations)
   ...
   [duel]
   description:Duel preset
   preset:duel|team
   nbTeams:2
   teamSize:1
   

   As the "team" preset is the default preset, it must contain all the preset settings declarations: it is a full preset. In this example the other preset "duel" only contains the desired setting changes to implement a duel mode: it is a partial preset.
   
   Which such a configuration, when the players change the preset from "team" to "duel", it will keep the oher settings which don't necessarily need to be changed, as the current map for example. However, when the players switch back to the "team" preset, all the settings will be restored to the ones defined in the "team" preset, as it is a full preset which contains all the settings declarations. If they have made changes to the current settings according to their needs, they will be lost.
   
   This may or may not be the desired behavior, depending on the use case. However both types of presets are useful:
   • a full preset is required in any case as the default preset. But full presets can also be useful for other presets as well, because players might expect the entire configuration to be reset when they apply some main presets.
   • a partial preset is useful when it makes sense to apply this preset on top of other presets. For example, one could implement a "silentMode" preset which would change the values of the "noSpecChat", "noSpecDraw" and "forwardLobbyToGame" settings all together. This would make perfect sense to make this preset applicable on top of the "team" and "duel" presets. In this case it's much easier to make a "silentMode" partial preset compared to making 2 new full presets "teamSilentMode" and "duelSilentMode".
    
2. The preset inheritance functionality
   
   As explained above, it is often useful to have multiple full presets declared in spads.conf, if only to be able to easily change the default preset without having to reorganize the configuration entirely.
   
   The problem with having multiple full presets declared in spads.conf is that it implies having lots of duplicate settings declarations, as each full preset redeclares all the preset settings. Most of these duplicate declarations serve no purpose, as usually only a small part is different between the full presets. Additionally it makes SPADS configuration maintenance much harder because when one setting needs to be updated it needs to be updated in all the full presets...
   
   Until now, the only way to workaround this issue was to use the "configuration include" functionality, which is explained in this post. Basically the idea with configuration includes is to remove the settings declarations common to all full presets from spads.conf and put them in a separate file, which can then be included in each full preset declared in spads.conf. Configuration includes make it possible to factorize the configuration and solves the problem of duplicate declarations, but it has some drawbacks:
   • it only allows factoring settings declarations which are identical in all full presets
   • it breaks the configuration into several files, which is still not ideal regarding maintenance
   The preset inheritance functionality has been introduced in SPADS 0.12.17 to overcome these limitations: it provides a way to make a preset inherit all the settings declared in other specific preset(s) (full or partial). Preset inheritances are specified in the preset declaration line ("[presetName]"), just by adding the list of inherited presets after the preset name, between "< >". For example, to declare the "duel" preset as inheriting from the "default" preset: "[duel<default>]". Another example for a "duelSilentMode" preset inheriting both from the "silentMode" and "default" presets: "[duelSilentMode<silentMode,default>]" (note that the order of inheritance is important, as any setting already defined by a preset will not be inherited from the remaining inherited presets).
   
   Let's take our previous example (from the "Reminder" section above) with the "team" and "duel" presets. The "team" preset was the default preset, it was the only full preset. The "duel" preset was a partial preset. Let's say we now want the "duel" preset to be the default preset. For this we need to make it a full preset. This can easily be done with preset inheritance like this:

   Code: Select all

   ...
   defaultPreset:duel
   ...
   [team]
   description:Team preset
   preset:team|duel
   ...
   ...(all other preset settings declarations)
   ...
   [duel<team>]
   description:Duel preset
   preset:duel|team
   nbTeams:2
   teamSize:1
   

   Let's say we now want to have both the "duel" preset as a partial preset (so that players can switch to duel mode while keeping the customizations they have made to other settings), but also a "duelReset" preset that is a full preset which will reset all the settings while switching to duel mode. Once again this can easily be done with preset inheritance like this:

   Code: Select all

   ...
   defaultPreset:team
   ...
   [team]
   description:Team preset
   preset:team|duel|duelReset
   ...
   ...(all other preset settings declarations)
   ...
   [duel]
   description:Duel preset
   preset:duel|team|duelReset
   nbTeams:2
   teamSize:1
   
   [duelReset<duel,team>]
   description:Duel preset (full)
   preset:duelReset|team|duel
   

[bibim]

Experimental support for shared dynamic data has been added in SPADS 0.12.19 (only in "unstable" release for now), for following data:

• Map start boxes
• Bans
• Trusted lobby certificates

When shared data are used, the data files are stored in the "varDir" directory (shared between multiple instances) instead of the "instanceDir" (specific to each instance). If there is no data file in the "varDir" directory, then it is automatically initialized from the data file of the "instanceDir" if any (from the first instance started in shared data mode).

This makes it possible to share saved map boxes for example between multiple instances of SPADS, so that when you do !saveBoxes on one instance, the new boxes are automatically used by the other instances too. You can try it by adding "sharedData=savedBoxes" as command line parameter when launching SPADS. This will make map start boxes shared between all your instances that use the same "varDir" directory.

Bans can be shared also, by adding "sharedData=bans" when launching SPADS. This way any ban that is added/removed on one instance will also be automatically applied on the other instances.

Finally, trusted lobby certificates (managed with the --tls-cert-trust|revoke|list commands) can also be shared by adding the "sharedData=trustedLobbyCertificates" parameter when launching SPADS.

Only one "sharedData" parameter can be given on the command line, so if you want to share multiple data you need to provide a comma separated list, for example to share all currently supported data: "sharedData=savedBoxes,bans,trustedLobbyCertificates"

[bibim]

Since SPADS 0.12.20, it is now also possible to share preferences data between multiple SPADS instances, by adding the "preferences" keyword to the list of shared data (see previous post).

For example, to share all supported data you need to start SPADS like this:

Code: Select all

perl spads.pl etc/spads.conf sharedData=bans,preferences,savedBoxes,trustedLobbyCertificates ...

(replace "..." by your specific configuration macro definitions for multi-instance)

Sharing preferences data helps both autohost administrators and users:

• Autohost administrators no longer need to be afraid of having heterogeneous configurations regarding !chskill and !chrank values on their autohosts, all their autohosts will always be consistent and use the same values.
• Players no longer need to set their "clan" preference for example on each autohost, they just need to set it once on one autohost and all the other autohosts of the group will use the same value

SPADS uses SQLite to share its preferences data, which means the Perl "DBD::SQLite" module from CPAN is a prerequisite to use the preferences data sharing functionality.
On Windows, main Perl distributions (ActivePerl and Strawberry) already include this module in their installer, so nothing needs to be done.
On Linux systems, this module may need to be installed. It can be done easily either with your standard package manager (for exampe on Debian-based systems: "apt-get install libdbd-sqlite3-perl"), either with a perl module manager such as cpanminus: "cpanm DBD::SQLite"