Why need loading the whole game when connecting as spec?

[Pithikos]

Why is it currently so that a person has to go through the whole game when connecting as a spec?
Is it meant to be like that for a specific reason or is it because of implementation difficulties?

[Pxtl]

What do you mean, you mean the fast-forward if you late-join to spectate, or the fact that all the gamedata is loaded?

Spring operates on a synced state. All the players send to each other is orders. All the unit movement and physics and everything are calculated locally, and the math is totally deterministic. That means if every player-order is handled by every machine at the exact same timeframe, then they all experience the exact same game - if somebody's processor mucks up the math or somebody has different starting state, then the game is borked and you get a desync.

There is no mechanism for sending the current state of the game, so the spectators need to run the full simulation from the very beginning of the game.

[Cheesecan]

In plain english it's like if I told you:
Go ten steps forward.
Then go twenty steps left.
Then after that go three steps forward and you're there.

And you only hear the last two instructions and not the first, then you will end up in the wrong place.

[SinbadEV]

In plain english it's like if I told you:
Go ten steps forward.
Then go twenty steps left.
Then after that go three steps forward and you're there.

And you only hear the last two instructions and not the first, then you will end up in the wrong place.

for the same reason that you can download DwarfFortress in 8MB and then use it to procedurally generate a hugely complicated world that takes up hundreds of MB of ram when it executes... it's faster (network wise) for your computer to use the simulation commands to recreate the game state then it would be to dump the state of the hosts ram and send it to you.

[zerver]

Problem is the game state can be very big. You would definitely have to download much more than the few MB of data contained in the typical 8v8BADSD demo. Transfer of game state also means an increased risk for obscure desync bugs since the game state will be assembled using a different method compared to the other players.

[AdCr]

A way to make the loading shorter, would be to disable some graphics effects that are slowing the video rendering until the game is synced...
I don't know if this is realist to implement but i think it could be nice.

[knorke]

A way to make the loading shorter, would be to disable some graphics effects that are slowing the video rendering until the game is synced...
I don't know if this is realist to implement but i think it could be nice.

I think it already draws less frames when fastforwarding during rejoin.
If you skip forward in replays (using /skip X ie /skip 60 -> forward one minute) it does not draw any graphic at all, just a progress bar.
It does not seem to be that much faster on my computer, I guess because the limit is cpu (how fast it can run the simulation etc) and not the graphics. Not sure what it is like on faster computers.

[Pithikos]

In plain english it's like if I told you:
Go ten steps forward.
Then go twenty steps left.
Then after that go three steps forward and you're there.

And you only hear the last two instructions and not the first, then you will end up in the wrong place.

Well what if I can get the current position of x object? I don't see the reason to load all the previous things. Can't you just load every position of every unit and their states(command given, attacking etc) at the current state in game? That doesn't seem like that much information to me. Though I have never been in game development so I am not that sure how it would be in practise.

[SinbadEV]

In plain english it's like if I told you:
Go ten steps forward.
Then go twenty steps left.
Then after that go three steps forward and you're there.

And you only hear the last two instructions and not the first, then you will end up in the wrong place.

Well what if I can get the current position of x object? I don't see the reason to load all the previous things. Can't you just load every position of every unit and their states(command given, attacking etc) at the current state in game? That doesn't seem like that much information to me. Though I have never been in game development so I am not that sure how it would be in practise.

I'm not a dev but:

Look at how much "memory" (ram and PF) spring uses when it is running... this approximates the size of the file you'd need to send to transmit the state, and this state would be outdated by the time it arrived and was loaded on your machine so you would need to run sim from that point on to catch up anyway... and the host would likely need to pause ITS sim to export this data to you.

To get an idea of what going on here is a short incomplete list of what would need to be "saved and sent" (All of this data needs to be transferred in a 32bit float with no rounding):
- The location and orientation of every piece of every unit.
- The current state of all players synced Lua including the current state of each unit's script
- The current height of each pixel of the height map (due to terrain deformation)
- Some synced stuff that's only available in the GPU/GRAM that would be difficult to extract
-

[knorke]

Look at how much "memory" (ram and PF) spring uses when it is running... this approximates the size of the file you'd need to send to transmit the state

i think most of ram usage is graphics, textures, 3d models, sound etc.

- The current height of each pixel of the height map (due to terrain deformation)

only sending differences might work too.

But with fast computers it only takes some few minutes to rejoin and since that system works it is unlikely that somebody will make a new system. In previous threads (rejoining+load/save (which is kind of related) comes up all the time) some devs said the real problem was that its complicated to get all needed data for save games because it is scattered across the engine. or something like that.

[Pithikos]

- The current height of each pixel of the height map (due to terrain deformation)

I think you worry about things that you don't have to. There must be a way to know that an explosion took place at some point so that any cpu can calculate on itself how the deformation will look like.

[Pxtl]

Zero-K and certain Kernel Panic maps deform the heightmap without using explosions. You've just desynced their players. See why this is a hard problem to solve?

Now obviously the heightmap deformation data transfer could be compressed by various ways... but playing-back a history of modifications is basically a small subset of *what the engine does now* and nicely demonstrates *why* it does it that way.

[SirMaverick]

- The current height of each pixel of the height map (due to terrain deformation)

I think you worry about things that you don't have to. There must be a way to know that an explosion took place at some point so that any cpu can calculate on itself how the deformation will look like.

That point was for the potential mode where you send the current state instead of the total network traffic (as done atm). Only that will be send, so you don't know what happened before (that might imply problems with stats and replay).

[zerver]

I think you worry about things that you don't have to. There must be a way to know that an explosion took place at some point so that any cpu can calculate on itself how the deformation will look like.

Yes, and this is exactly what it does when it fast forwards the sim during rejoin. Sounds a bit like you are contradicting your initial suggestion to transfer the game state instead.

[Pithikos]

I think you worry about things that you don't have to. There must be a way to know that an explosion took place at some point so that any cpu can calculate on itself how the deformation will look like.

Yes, and this is exactly what it does when it fast forwards the sim during rejoin. Sounds a bit like you are contradicting your initial suggestion to transfer the game state instead.

I am not sure how you see some contradiction there. At the current state the game goes through the whole replay. If there is an explosion that took place at time 50:00 then it will still start from the beginning and replay the whole game until 50:00. So what I said is why can't you just skip directly to the 50:00? That's just a little example to show you that there's no contradiction. My initial suggestion was on skipping directly to the current state or some seconds before the current state.

I mean there is no reason for the user to see everything happening from the beginning of the game until the current state. You could calculate the map deformation and show the map only when your calculations are done, right?

[Pithikos]

Is there by the way some documentation on how the simulation/save/load work in detail and where in the source code someone should look?

[Satirik]

i guess the answer is "because of implementation difficulties"

getting the state of everything, sending it, and apply them is a lot of work and would introduce a lot of desync problems, it's not just position, orientation etc, spring is full of lua

and ... it's not a bug it's a feature, so you can see the game at high speed to understand what happenned

[SirMaverick]

I am not sure how you see some contradiction there. At the current state the game goes through the whole replay.

Yes.

If there is an explosion that took place at time 50:00 then it will still start from the beginning and replay the whole game until 50:00. So what I said is why can't you just skip directly to the 50:00? That's just a little example to show you that there's no contradiction. My initial suggestion was on skipping directly to the current state or some seconds before the current state.

You don't know that there was an explosion. Neither current simulation state (apart from the fact that the map does not have the original height at some point anymore) nor network traffic tells you that.

Network traffic just tells you initial conditions (map, mod, options, start points...), when game frames happen and commands given. For example the event that an unit is created or destroyed is never send. You only know this when you run the simulation (e.g. a build command is given, factory is ready and starts building). Explosions are just secondary effects from commands. E.g. they happen by weapon fire (attack command, units on patrol) or unit explosions (unit died).
Again, all this you don't know. Just that there were some commands send, like build command, move command and attack/patrol/fight command. None of them tell you if/when/where explosions happen.

I mean there is no reason for the user to see everything happening from the beginning of the game until the current state.

Just for joining the game, yes.

You could calculate the map deformation and show the map only when your calculations are done, right?

See above.

[SirMaverick]

Is there by the way some documentation on how the simulation/save/load work in detail and where in the source code someone should look?

On client side there is no change needed for that. It just receives the traffic as usual and processes it. The game is running faster, because the traffic is sent at once: the simulation will not process just 30 GameFrames/s, but as much as possible.

Server side: GameServer.cpp
Server buffers all outgoing/broadcasted traffic. The buffer content is send to all connection clients so that they can catch up.