Indeterminacy, Pt. 2 - Page 2

Indeterminacy, Pt. 2

Discuss Lua based Spring scripts (LuaUI widgets, mission scripts, gaia scripts, mod-rules scripts, scripted keybindings, etc...)

Moderator: Moderators

eriatarka
Posts: 67
Joined: 26 Jan 2008, 18:50

Re: Indeterminacy, Pt. 2

Post by eriatarka »

Argh wrote:Ok, here's my test (I'll update the script, above).

First results, "live":

First Iteration, 19781
Second Iteration, 19781, 9843
First Iteration, 14030
Second Iteration, 14030, 18346
First Iteration, 6126
Second Iteration, 6126, 8308

Second results, "Replay Running 1"

First Iteration, 19781
Second Iteration, 19781, 9843
First Iteration, 14030
Second Iteration, 14030, 18346
First Iteration, 6126
Second Iteration, 6126, 8308

Hmm. It's exactly the same. So, that ain't it, the seeds must be matching exactly... back to the drawing board.
Well, AFAIK there are two RNGs in Spring, a synced one and an unsynced one, right? Now if unsynced Lua code uses the synced RNG, then of course everything would break down because the unsynced code doesn't run identically on every client. I would think that, if this is the case, this could also desync multiplayer games.


EDIT: Well, it seems LuaSyncedHandler sets everything up so that the correct RNG gets called. Argh, is the issue reproducible? Maybe you could post a replay (as short and simple as possible) demonstrating the problem?
User avatar
Argh
Posts: 10920
Joined: 21 Feb 2005, 03:38

Re: Indeterminacy, Pt. 2

Post by Argh »

Argh, is the issue reproducible?
Well, there's the rub- I hadn't seen this issue before. However, I usually don't look at replays a lot, either. I can't think of anything in CA's code that relies on indeterminacy, offhand, that isn't unsynced, so therefore, I'll just have to write some unsynced code to test this out, and see what transpires:

Code: Select all

function gadget:GetInfo()
   return {
      name = "Random Verifier",
      desc = "Verifies integrity of math.random within replays.",
      author = "Argh",
      date = "",
      license = "GPL, v.2 or later, 2008",
      layer = 1,
      enabled = true,
   }
end

if (gadgetHandler:IsSyncedCode()) then
--------------------------------------------------------------------------------
--  SYNCED
--------------------------------------------------------------------------------
function gadget:GameFrame(f)

if f > 0 and f < 5 then

local Random = math.random (1,20000)
Spring.Echo ("First Iteration",Random)
local Random2 = math.random (1,20000)
Spring.Echo ("Second Iteration",Random,Random2)

end

if (f == 5) then Spring.Echo("Removing Randomizer Gadget") gadgetHandler.RemoveGadget("Random Verifier")  end

end
else

function gadget:DrawWorld()

local Random3 = math.random (1,20000)
Spring.Echo ("Third Iteration",Random3)
local Random4 = math.random (1,20000)
Spring.Echo ("Fourth Iteration",Random3,Random4)

end
end
And here are the results, "live".

Code: Select all

First Iteration, 3739
Second Iteration, 3739, 11717
First Iteration, 5776
Second Iteration, 5776, 11855
First Iteration, 7995
Second Iteration, 7995, 16451
First Iteration, 456
Second Iteration, 456, 314

Code: Select all

Third Iteration, 19608
Fourth Iteration, 19608, 8783
Third Iteration, 9352
Fourth Iteration, 9352, 1799
Third Iteration, 11623
Fourth Iteration, 11623, 18757
Third Iteration, 8348
Fourth Iteration, 8348, 8215
Ok, let's see a replay, and make sure they match...

Code: Select all

First Iteration, 19781
Second Iteration, 19781, 9843
First Iteration, 18625
Second Iteration, 18625, 9561
First Iteration, 6126
Second Iteration, 6126, 8308
First Iteration, 6137
Second Iteration, 6137, 19525

Code: Select all

Third Iteration, 12945
Fourth Iteration, 12945, 12442
Third Iteration, 12777
Fourth Iteration, 12777, 14417
Third Iteration, 19269
Fourth Iteration, 19269, 5835
Third Iteration, 9663
Fourth Iteration, 9663, 18394
Hmm. It appears I've just reproduced this problem.

To eliminate the possibility that it was just the nature of the predrawworld stuff, I checked my replay results against the "live", expecting to find the same set of numbers, maybe just in a different place (i.e., same seed, just called differently). Didn't happen. It appears that we have a genuine problem after all, unless somebody sees a big hole in the way this experiment was set up.
Kloot
Spring Developer
Posts: 1867
Joined: 08 Oct 2006, 16:58

Re: Indeterminacy, Pt. 2

Post by Kloot »

Not reproducable in SVN Spring:

Code: Select all

function gadget:GetInfo()
	return {
		name = "RNG",
		desc = "",
		author = "",
		date = "",
		license = "",
		layer = 1,
		enabled = true,
	}
end

if (gadgetHandler:IsSyncedCode()) then
	function gadget:GameFrame(f)
		if (f >= 450 and f < 480) then
			local r = math.random(1, 20000)
			Spring.Echo("(  synced) frame=" .. f .. "\tnum=" .. r .. "\n")
		end

		if (f == 480) then
			gadgetHandler.RemoveGadget("RNG")
		end
	end
else
	local lastFrame = 0

	function gadget:DrawWorld()
		local frame = Spring.GetGameFrame()

		if (frame >= 450 and frame < 480 and frame ~= lastFrame) then
			lastFrame = frame
			local r = math.random(1, 20000)
			Spring.Echo("(unsynced) frame=" .. frame .. "\tnum=" .. r .. "\n")
		end
	end
end
Live output:

Code: Select all

(unsynced) frame=450	num=10306
(  synced) frame=450	num=13208
(unsynced) frame=451	num=11173
(  synced) frame=451	num=19560
(unsynced) frame=452	num=16123
(  synced) frame=452	num=14879
(unsynced) frame=453	num=9388
(  synced) frame=453	num=5797
(unsynced) frame=454	num=1172
(  synced) frame=454	num=2419
(unsynced) frame=455	num=11995
(  synced) frame=455	num=4426
(unsynced) frame=456	num=15886
(  synced) frame=456	num=2782
(unsynced) frame=457	num=10175
(  synced) frame=457	num=14588
(unsynced) frame=458	num=2781
(  synced) frame=458	num=18545
(unsynced) frame=459	num=17552
(  synced) frame=459	num=8549
(unsynced) frame=460	num=5548
(  synced) frame=460	num=12912
(unsynced) frame=461	num=987
(  synced) frame=461	num=16700
(unsynced) frame=462	num=7305
(  synced) frame=462	num=14710
(unsynced) frame=463	num=12206
(  synced) frame=463	num=12556
(unsynced) frame=464	num=18542
(  synced) frame=464	num=14038
(  synced) frame=465	num=13385
(unsynced) frame=466	num=18200
(  synced) frame=466	num=9656
(unsynced) frame=467	num=2899
(  synced) frame=467	num=12717
(unsynced) frame=468	num=19921
(  synced) frame=468	num=7909
(unsynced) frame=469	num=17484
(  synced) frame=469	num=9207
(unsynced) frame=470	num=6224
(  synced) frame=470	num=4688
(unsynced) frame=471	num=12552
(  synced) frame=471	num=121
(unsynced) frame=472	num=7921
(  synced) frame=472	num=8195
(unsynced) frame=473	num=16648
(  synced) frame=473	num=850
(unsynced) frame=474	num=11374
(  synced) frame=474	num=12259
(unsynced) frame=475	num=13592
(  synced) frame=475	num=9909
(unsynced) frame=476	num=958
(  synced) frame=476	num=1332
(unsynced) frame=477	num=11923
(  synced) frame=477	num=13939
(unsynced) frame=478	num=10871
(  synced) frame=478	num=15494
(unsynced) frame=479	num=449
(  synced) frame=479	num=12705
(  synced) frame=480	num=7517
Demo output:

Code: Select all

(  synced) frame=450	num=13208
(unsynced) frame=451	num=10306
(  synced) frame=451	num=19560
(unsynced) frame=452	num=11173
(  synced) frame=452	num=14879
(unsynced) frame=453	num=16123
(  synced) frame=453	num=5797
(unsynced) frame=454	num=9388
(  synced) frame=454	num=2419
(unsynced) frame=455	num=1172
(  synced) frame=455	num=4426
(  synced) frame=456	num=2782
(unsynced) frame=457	num=11995
(  synced) frame=457	num=14588
(  synced) frame=458	num=18545
(unsynced) frame=459	num=15886
(  synced) frame=459	num=8549
(unsynced) frame=460	num=10175
(  synced) frame=460	num=12912
(unsynced) frame=461	num=2781
(  synced) frame=461	num=16700
(unsynced) frame=462	num=17552
(  synced) frame=462	num=14710
(unsynced) frame=463	num=5548
(  synced) frame=463	num=12556
(  synced) frame=464	num=14038
(  synced) frame=465	num=13385
(unsynced) frame=466	num=987
(  synced) frame=466	num=9656
(  synced) frame=467	num=12717
(unsynced) frame=468	num=7305
(  synced) frame=468	num=7909
(unsynced) frame=469	num=12206
(  synced) frame=469	num=9207
(unsynced) frame=470	num=18542
(  synced) frame=470	num=4688
(unsynced) frame=471	num=18200
(  synced) frame=471	num=121
(unsynced) frame=472	num=2899
(  synced) frame=472	num=8195
(unsynced) frame=473	num=19921
(  synced) frame=473	num=850
(unsynced) frame=474	num=17484
(  synced) frame=474	num=12259
(  synced) frame=475	num=9909
(unsynced) frame=476	num=6224
(  synced) frame=476	num=1332
(unsynced) frame=477	num=12552
(  synced) frame=477	num=13939
(unsynced) frame=478	num=7921
(  synced) frame=478	num=15494
(unsynced) frame=479	num=16648
(  synced) frame=479	num=12705
(  synced) frame=480	num=7517
(Note that the unsynced demo sequence runs one
random() call behind the unsynced live sequence.)
Last edited by Kloot on 10 Feb 2008, 19:49, edited 1 time in total.
User avatar
Argh
Posts: 10920
Joined: 21 Feb 2005, 03:38

Re: Indeterminacy, Pt. 2

Post by Argh »

Um, I just spot-checked your output... looks like it reproduced, to me...

Live:

Code: Select all

(unsynced) frame=470   num=6224
(  synced) frame=470   num=4688
Demo:

Code: Select all

(unsynced) frame=470   num=18542
(  synced) frame=470   num=4688
Live:

Code: Select all

(unsynced) frame=477   num=11923
(  synced) frame=477   num=13939
Demo:

Code: Select all

(unsynced) frame=477   num=12552
(  synced) frame=477   num=13939
Shouldn't both unsynced and synced results match each other in demos and live? IIRC, some synced code has gotta be relying on unsynced code's output. Lemme take a look at KDR's code in his area-attack Widget...

Nope, that's a dead end. No unsynced code there.
Kloot
Spring Developer
Posts: 1867
Joined: 08 Oct 2006, 16:58

Re: Indeterminacy, Pt. 2

Post by Kloot »

The whole point of this exercise was to show that
the synced RNG uses the same seed for demo's
as for the games from which those demo's were
recorded, the unsynced RNG was not in question.

Also:
(Note that the unsynced demo sequence runs one
random() call behind the unsynced live sequence.)
User avatar
Peet
Malcontent
Posts: 4384
Joined: 27 Feb 2006, 22:04

Re: Indeterminacy, Pt. 2

Post by Peet »

Argh wrote:Shouldn't both unsynced and synced results match each other in demos and live? IIRC, some synced code has gotta be relying on unsynced code's output.
Image
tombom
Posts: 1933
Joined: 18 Dec 2005, 20:21

Re: Indeterminacy, Pt. 2

Post by tombom »

Argh wrote:Shouldn't both unsynced and synced results match each other in demos and live? IIRC, some synced code has gotta be relying on unsynced code's output. Lemme take a look at KDR's code in his area-attack Widget...
You seem to be completely mislead on the difference between unsynced and synced results. Unsynced isn't intended to be the same between computers and runs; synced code that relies on unsynced would cause sync errors quickly. A synced result should definitely always be the same between systems and replays.
User avatar
Argh
Posts: 10920
Joined: 21 Feb 2005, 03:38

Re: Indeterminacy, Pt. 2

Post by Argh »

The whole point of this exercise was to show that
the synced RNG uses the same seed for demo's
as for the games from which those demo's were
recorded, the unsynced RNG was not in question.
I agree, this appears to be the case. I will attempt to duplicate the results reported in KDR's thread, as I have not seen a problem with desync in any of the stuff in World Builder in the demos (and if it should be happening anywhere, it should be there).
You seem to be completely mislead on the difference between unsynced and synced results. Unsynced isn't intended to be the same between computers and runs; synced code that relies on unsynced would cause sync errors quickly. A synced result should definitely always be the same between systems and replays.
Um, but shouldn't it use the same seed, per computer and per run? I mean, why make it different for everybody, when there's no real need?

I mean, that could have all sorts of unintended consequences for gameplay. Take, for example, a shader that randomly makes "smoke" on the battlefield, and uses unsynced code to completely hide any Units that are beneath it (by culling them or drawing the particles in a higher layer, etc.), making it difficult for players to select a given Unit. If it runs differently on each machine, it's effectively doing something different to everybody's real experience of the game. Is this a good idea, when it's totally unnecessary? I mean, there aren't any negative consequences of everybody sharing the same seed...
User avatar
lurker
Posts: 3842
Joined: 08 Jan 2007, 06:13

Re: Indeterminacy, Pt. 2

Post by lurker »

Let's use that same example. You have a smoke shader that uses a unsynced generator that starts the same everywhere. But then someone isn't looking at the smoke. You still need the numbers? What if they run a widget that calls for a random number. You want to run a separate one for each widget and gadget? What if someone needs to have their smoke shader use fewer larger particles to avoid crippling their computer?
User avatar
Argh
Posts: 10920
Joined: 21 Feb 2005, 03:38

Re: Indeterminacy, Pt. 2

Post by Argh »

Sorry, again I'm guilty of being unclear here. I'm not at all concerned with Widgets. I'm concerned with Gadgets, where the game designer expects the same outcome for all parties. I couldn't care less if Widgets do whatever they feel like. Per this example:
What if someone needs to have their smoke shader use fewer larger particles to avoid crippling their computer?
Then that's something that should be done through a Gadget, so that game designers control the outcome, and it should result in exactly the same amount of blindness that other players experience, if the game designer's doing their job correctly.
eriatarka
Posts: 67
Joined: 26 Jan 2008, 18:50

Re: Indeterminacy, Pt. 2

Post by eriatarka »

Ok, Argh had a misconception regarding synced/unsynced mode, but his experiment where he compared synced and unsynced random numbers seems to indicate that there is a problem in .76b1 (where I assume he did these tests), doesn't it? Maybe it's something that was since fixed in SVN?
eriatarka
Posts: 67
Joined: 26 Jan 2008, 18:50

Re: Indeterminacy, Pt. 2

Post by eriatarka »

Argh wrote:Sorry, again I'm guilty of being unclear here. I'm not at all concerned with Widgets. I'm concerned with Gadgets, where the game designer expects the same outcome for all parties. I couldn't care less if Widgets do whatever they feel like.
I'm not sure if the distinction is relevant here. Whatever runs synced must be consistent across clients (and also across live/demo), and what runs unsynced will usually not match up. That's the whole point of this concept. This implies that synced code must never rely on the results of unsynced code.
User avatar
Argh
Posts: 10920
Joined: 21 Feb 2005, 03:38

Re: Indeterminacy, Pt. 2

Post by Argh »

I was testing with SVN revision 5455. Kloot's results and mine differed considerably- I let the unsync code run unsync'd, whereas he tied it to the value of gameframe, and I ended up with considerably different results... I never saw the same numbers come up again, in my test, whereas Kloot demonstrated matching sets. My best guess is that Kloot's test was programmed correctly, mine must be flawed in some way.
I'm not sure if the distinction is relevant here. Whatever runs synced must be consistent across clients (and also across live/demo), and what runs unsynced will usually not match up. That's the whole point of this concept. This implies that synced code must never rely on the results of unsynced code.
I understand your point, and Lurker's. Maybe if I want anything to render that might have gameplay effects, it'll have to be done in synced code, period. That imposes some limitations, but nothing too odious.
Kloot
Spring Developer
Posts: 1867
Joined: 08 Oct 2006, 16:58

Re: Indeterminacy, Pt. 2

Post by Kloot »

eriatarka wrote:Ok, Argh had a misconception regarding synced/unsynced mode, but his experiment where he compared synced and unsynced random numbers seems to indicate that there is a problem in .76b1 (where I assume he did these tests), doesn't it? Maybe it's something that was since fixed in SVN?
You can run the gadget I posted with .76b1 to verify that yourself. ;)
Argh wrote:I was testing with SVN revision 5455. Kloot's results and mine differed considerably- I let the unsync code run unsync'd, whereas he tied it to the value of gameframe, and I ended up with considerably different results... I never saw the same numbers come up again, in my test, whereas Kloot demonstrated matching sets.
The only thing that affects is the frequency of unsynced random()
calls (once per sim-frame like the synced portion of the script vs.
once per rendered frame).

Live SVN output, where line 33 now reads

Code: Select all

if (frame >= 0 and frame < 480) then

Code: Select all

(  synced) frame=181	num=17973
(  synced) frame=182	num=4017
(unsynced) frame=183	num=10306
(unsynced) frame=183	num=11173
(  synced) frame=183	num=5259
(unsynced) frame=184	num=16123
(unsynced) frame=184	num=9388
(unsynced) frame=184	num=1172
(  synced) frame=184	num=3408
(unsynced) frame=185	num=11995
(unsynced) frame=185	num=15886
(unsynced) frame=185	num=10175
(  synced) frame=185	num=3336
(unsynced) frame=186	num=2781
(unsynced) frame=186	num=17552
(unsynced) frame=186	num=5548
(unsynced) frame=186	num=987
(  synced) frame=186	num=425
(unsynced) frame=187	num=7305
(unsynced) frame=187	num=12206
(unsynced) frame=187	num=18542
(unsynced) frame=187	num=18200
(  synced) frame=187	num=18533
(unsynced) frame=188	num=2899
(unsynced) frame=188	num=19921
(unsynced) frame=188	num=17484
(  synced) frame=188	num=16086
(unsynced) frame=189	num=6224
(unsynced) frame=189	num=12552
(unsynced) frame=189	num=7921
(  synced) frame=189	num=7802
(unsynced) frame=190	num=16648
(unsynced) frame=190	num=11374
(unsynced) frame=190	num=13592
(  synced) frame=190	num=19530
(unsynced) frame=191	num=958

Demo SVN output:

Code: Select all

(  synced) frame=181	num=17973
(  synced) frame=182	num=4017
(  synced) frame=183	num=5259
(unsynced) frame=184	num=10306
(unsynced) frame=184	num=11173
(  synced) frame=184	num=3408
(unsynced) frame=185	num=16123
(unsynced) frame=185	num=9388
(  synced) frame=185	num=3336
(unsynced) frame=186	num=1172
(unsynced) frame=186	num=11995
(unsynced) frame=186	num=15886
(  synced) frame=186	num=425
(unsynced) frame=187	num=10175
(unsynced) frame=187	num=2781
(unsynced) frame=187	num=17552
(  synced) frame=187	num=18533
(unsynced) frame=188	num=5548
(unsynced) frame=188	num=987
(unsynced) frame=188	num=7305
(  synced) frame=188	num=16086
(unsynced) frame=189	num=12206
(unsynced) frame=189	num=18542
(unsynced) frame=189	num=18200
(unsynced) frame=189	num=2899
(  synced) frame=189	num=7802
(unsynced) frame=190	num=19921
(unsynced) frame=190	num=17484
(  synced) frame=190	num=19530
(unsynced) frame=191	num=6224
(unsynced) frame=191	num=12552
(unsynced) frame=191	num=7921
(  synced) frame=191	num=3721
eriatarka
Posts: 67
Joined: 26 Jan 2008, 18:50

Re: Indeterminacy, Pt. 2

Post by eriatarka »

Kloot wrote:
eriatarka wrote:Ok, Argh had a misconception regarding synced/unsynced mode, but his experiment where he compared synced and unsynced random numbers seems to indicate that there is a problem in .76b1 (where I assume he did these tests), doesn't it? Maybe it's something that was since fixed in SVN?
You can run the gadget I posted with .76b1 to verify that yourself. ;)
Hm, the point is probably moot anyway since Argh tested with Rev 5455. I mean of course there's a slight chance something happened between 5455 (Tuesday) and now, haven't been following all changes, but somehow I doubt it.

Well, I'm stumped ;)
User avatar
AF
AI Developer
Posts: 20687
Joined: 14 Sep 2004, 11:32

Re: Indeterminacy, Pt. 2

Post by AF »

Peet wrote:
Argh wrote:Shouldn't both unsynced and synced results match each other in demos and live? IIRC, some synced code has gotta be relying on unsynced code's output.
...
woo this ones a keeper

http://www.pastebin.com making horrendous threads readable
eriatarka
Posts: 67
Joined: 26 Jan 2008, 18:50

Re: Indeterminacy, Pt. 2

Post by eriatarka »

I just recalled there was a report of replays becoming desynced: http://spring.clan-sy.com/phpbb/viewtop ... 62#p255262. So this whole thing must hold some water.
User avatar
Zpock
Posts: 1218
Joined: 16 Sep 2004, 23:20

Re: Indeterminacy, Pt. 2

Post by Zpock »

Think about what happens if a widget is used to give units commands using (unsynced) random numbers? Such as an area attack widget.

The commands given would be different in the replay and so it gets borked.
User avatar
AF
AI Developer
Posts: 20687
Joined: 14 Sep 2004, 11:32

Re: Indeterminacy, Pt. 2

Post by AF »

Not really, imo the widget should not be able to interfere with the commands saved in the replay. Remember when playing back a replay you're reading in saved network traffic, your not resimulating everythign from scratch so if lua gadgets are runnign and actively issuing commands then they're adding to the saved replay traffic which imo is a bug in itself.

May not apply to thread, btu just my pennies worth with golden trim and silver rims, with a little moustache on the queens head and big dangly hoop earings.
eriatarka
Posts: 67
Joined: 26 Jan 2008, 18:50

Re: Indeterminacy, Pt. 2

Post by eriatarka »

Zpock wrote:Think about what happens if a widget is used to give units commands using (unsynced) random numbers? Such as an area attack widget.

The commands given would be different in the replay and so it gets borked.
In the thread I linked to, commands were given from a gadget, not a widget, specifically from synced code; and synced Lua code always uses the synced RNG, unless there's some bug I haven't spotted so far.
Post Reply

Return to “Lua Scripts”