Headless Client (spring without GUI and Sound)

[hoijui]

am i right when i assume that the dedicated server does only relay network traffic, and does not have a simulation going on locally?

while working on the Lua AI related changes we discussed yesterday (with BrainDamage and Auswaschbar), i had to include a file from ExternalAI in GameSetup.
as GameSetup is linked into the dedicated server, i found out that the dedicated server is not linked against anything under rts/ExternalAI/,
and therefore would not be able to run AIs. so i linked in all of rts/ExternalAI/, just to find that it was missing tons of symbols after that (like DamageArray, gu, weaponDefHandler, teamHandler, CFeatureHandler, ...)

i though the dedicated server is a server without GUI, and it could be used to run AI vs AI games faster and on hardware witout graphics for example, which would be usefull when training them with the help of learning algorithms.

so...
i propose a second version of a dedicated server, which includes a whole simulation, just no GUI stuff.
i assume this is a relatively big task, and should be done after the release, but we can already discuss it anyway.

please correct me if the server already runs a simulation. it does not appear to do so to me.

[imbaczek]

what you ask is something different than a dedicated server; it's a headless client. zerver's been doing some progress on that while working on multithreaded spring - basically, you need to isolate all gfx calls and move them all to unsynced code, then you can (theoretically) compile just the synced part.

the idea of the dedicated server is to only use the netcode part, ideally; it tries to include as little other code as possible, so you can serve multiple games at neglible cpu usage.

[hoijui]

i guessed the naming would need refinement (two things with the same name -> not good).

so zerver...
this is this headless client something you aim for, or you really just want the multi-threaded thing... or would we ultimately end up with either none or both of these anyway?
is this.. the GML stuff, or something else?
GML is integrated into master aready, just not enabled by default when compiling, right?
would it make sense to have a single branch for multi-threading and headless client?
would there be a problem with any Lua code too, if it does not separate logic and drawing? or could we just make the draw calls noops engine internally?
... short: lets duiscuss a bit, please :D

[zerver]

The original reason for the multithreading is to gain performance. Sim runs in a separate thread. Draw runs in another thread. GML makes it possible to use some extra rendering "helper" threads.

This is accomplished by defining wrappers for all relevant gl* functions. Depending on which thread makes a call, different things can happen. If it is the main Draw thread, it calls directly into OpenGL. If it is a helper thread, it is enqueued for processing later by the main Draw thread. If it is the Sim thread, an error is output on the info console.

This existing design with wrappers would make it relatively simple to NOP all GL calls if that is what you want. Like you said, some mods may otherwise try to make LUA GL calls from the Sim thread. This is a very common problem with many widgets, in for example CA.

Ideally, GML can boost performance a lot. However, the limitation of a single "real" Draw thread is a major bottleneck. Right now, the biggest gain comes not from GML, but from the separate Sim thread. GML performs best when the rendering code is computationally intensive and the percentage of GL calls is fairly low, which is not very common.

[Auswaschbar]

OpenGL calls are mostly asyncronous anyway, so I'm afraid swapping out the opengl-calls with noops won't give you any speed improvement.

[zerver]

I don't think speed improvement was the key here, but rather to make a headless client that does not malfunction when a mod makes a GL call from Sim. The headless client may not have initialized GL at all, so I don't know what would happen. Crash maybe.

[Auswaschbar]

could be used to run AI vs AI games faster

[SpliFF]

I think a lot of threading and headless issues would be solved at once if the Sim and Synced Lua were prohibited from performing any *direct* GL or audio functions. Any rendering/sound commands should be passed to a UI thread which runs unsynced. Is this what GML does or is GML still synced?

I know this is how Supreme Commander does it. All os/audio/gfx tasks are run from an unsynced "User" thread. Although UI events can be triggered from the Sim via a callback system, they don't run _in_ the Sim. Results (if any) from UI code are passed back as an event on the next frame. The Sim and User states are actually entirely seperate Lua states.

[hoijui]

my main interest on this is to make it possible to have a server farm run spring games with AIvsAI on eg. 10x or 20x speed or so.. clustering style, and do training.
it is not feasable for many things, but for some it is, especially for research, even if it wont produce useable AIs. if it is not much faster without graphics, its ok for me, still serves the main purpose.

as i understand it, this could easily be done without affecting spring development. it woudl not even need a separate branch. just create tools/HeadlessClient/CMakeLists.txt, do not build this by default, and then start adding #ifdef HEADLESS here and there.
right? :D
the downside of a separate branch is, that all the work would have to be done by a single person, most likely (i know thats not the case techincally, but in practise thats the case).

[aegis]

removing the renderer might not decrease the cycles used by the main thread, but it *will* remove any cpu requirements of the actual rendering

[Auswaschbar]

have a server farm run spring games with AIvsAI

Do you have one of those?

[aegis]

I have a decent amount of unused processing power locally if anyone wanted to run such a project.

[AF]

Whatever happened to autohosts and hwo they run spring? I remember sander bakke asking me about this so he could run batch runs of spring automatically for academic purposes and he managed to get it running via command line parameters which started spring minimized. I also remember svn commits making spring not bother to render while minimized.

[imbaczek]

starting minimized and starting without graphics whatsoever is something that makes a difference.

[zerver]

Yep, and if it's still too slow the draw loop can easily be stripped a bit further with some #ifdefs.

[BaNa]

my main interest on this is to make it possible to have a server farm run spring games with AIvsAI on eg. 10x or 20x speed or so.. clustering style, and do training..

Am I the only one getting skynet vibes here?

[zerver]

I'm not sure they employed the neural networks yet. But if they did, it becomes self-aware pretty soon.

[Neddie]

Sentience is unlikely to be duplicated in silicon due to the large amount of indirect relationships involved in thought and the completely contextual and imprecisely indexed method of data storage.

[lurker]

Sentience is unlikely to be duplicated in silicone

I agree.

[Neddie]

Averaging one spelling mistake in 24,000 words at last count, Neddie!