2025-07-02 07:46 CEST
  

View Issue Details [ Jump to Notes ] [ Issue History ] [ Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0001019Spring engineGeneralpublic2008-08-22 23:122008-09-04 23:15
Reporterimbaczek 
Assigned ToAuswaschbar 
PrioritynormalSeverityblockReproducibilityalways
StatusresolvedResolutionfixed 
Product Version0.76b1+svn 
Target VersionFixed in Version 
Summary0001019: major desyncs in all mods
Descriptionright from the start.
TagsNo tags attached.
Checked infolog.txt for Errors
Attached Files

-Relationships
related to 0001017closed some commands don't get transmitted in CA r2697 
+Relationships

-Notes

~0002486

jK (developer)

2008-08-22 23:13

 r6303 + ca 2697

~0002488

imbaczek (reporter)

2008-08-22 23:27

 the replays are kinda broken, too; only one player on each team works, and there's error spam about invalid startpos messages right from the start.

~0002502

bibim_ (reporter)

2008-08-23 12:54

 I tried replaying my demo with r6308. Now all players are here, but the demo doesn't match what I saw and did in game when playing (lots of commands skipped :/)

~0002504

imbaczek (reporter)

2008-08-23 13:11

 I can't reproduce this reliably on lan with 4 players and 2 machines, with or without 6243+6262. It only happened once, when not needed...

Maybe latency is an issue here?

~0002512

Auswaschbar (reporter)

2008-08-24 18:49

 I did some testing:
Not reproducable with two instances of the same build. Even if I add much latency and packet loss (networking works fine).

I can 100% reproduce desyncs with:
Run a self-compiled and a windows version from buildserv with BA
-> build an core anti-air-tower
-> build some fighters (every plane works)
-> let them flew over aa gun
-> desync

~0002513

imbaczek (reporter)

2008-08-24 21:11

 reproduced with two exactly same builds, with much less elaborate setup; sometimes, you don't need to do _anything_, it desyncs on frame 1 (CA r2697). Not sure if there's any useful information in replays, but posting them anyway...

~0002514

Kloot (developer)

2008-08-24 21:36

 I've also had it desync with identical builds on frame 1 (before any user commands are ever transmitted), so we're probably / hopefully just looking at uninitialized data reads.

~0002515

Kloot (developer)

2008-08-25 13:53

Last edited: 2008-08-25 13:57

 I traced the source of the desync in your (imbaczek) latest demos to a crucial difference in the setup scripts:


Client:
    [TEAM0]
    {
        TeamLeader=0;
        AllyTeam=0;
        RGBColor=0.07843 0.07843 0.96078;
        Side=ARM;
        Handicap=0;
    }
    [TEAM1]
    {
        TeamLeader=1;
        AllyTeam=1;
        RGBColor=0.78431 0.00000 0.00000;
        Side=ARM; <======================== !
        Handicap=0;
    }


Server:
    [TEAM0]
    {
        TeamLeader=0;
        AllyTeam=0;
        RGBColor=0.07843 0.07843 0.96078;
        Side=ARM;
        Handicap=0;
    }
    [TEAM1]
    {
        TeamLeader=1;
        AllyTeam=1;
        RGBColor=0.78431 0.00000 0.00000;
        Side=CORE; <======================== !!
        Handicap=0;
    }


Ironically I didn't notice this by just watching them, I had to hack the sync checker to find that the point of divergence was actually somewhere in CCommanderScript::GameStart() (which didn't leave much room for interpretation). Bottom line: always compare the scripts when faced with a
very early sync error like this!

~0002517

imbaczek (reporter)

2008-08-25 14:00

Last edited: 2008-08-25 14:00

 well duh! /me fails. sorry for your wasted time. (commit synccheker hacks, tho ;p)

maybe implement script.txt (or just sycned team/player data) transfer and/or checksumming? something to consider after release.

~0002521

Kloot (developer)

2008-08-25 19:07

Last edited: 2008-08-25 23:14

 Yeah, good idea. Only problem with checksumming the scripts directly is that they can differ wrt. case (ex. "Arm" vs "ARM" vs "arm") and formatting (ex. "[GAME] {" vs "[GAME]\n{"), it would need to be done over the parsed key/value pairs.

I've also just examined the demos of that 3v3 game on SmallDivide with a self-compiled build of r6316, all of them play out exactly the same way as far as I can tell (locally calculated simframe checksums are equal across each demo for the first 5 minutes at least). I don't think we're going to extract any useful information out of them, it's impossible to know why fi. Crayfish desynced as early as frame 32 from just the original warning message (maybe it is build-specfic like Auswaschbar's post suggests).

~0002531

Auswaschbar (reporter)

2008-08-29 01:05

 I can now 100% reproduce a desync with two different builds, every time in frame 32. No orders are given, it jsut desyncs while running.

I have the suspicion that in frame 32 (31) something reads an unsynced variable and writs it to synced. Anyone have an idea what runs in frame 32?

~0002539

imbaczek (reporter)

2008-08-29 09:29

 Isn't that the first SlowUpdate?

~0002541

Kloot (developer)

2008-08-29 13:43

 Not for units; they follow a staggered update pattern in which the first (activeUnits.size() / 16) units are SlowUpdate()'d when !(gs->frameNum & 15) == true (so every 16th frame, starting at frame 0), then the second (activeUnits.size() / 16) units on the next frame, etc. CUnit::SlowUpdate() in turn SlowUpdate()'s the unit's commandAI, movetype, and weapons.

Teams are SlowUpdate()'d every 32 frames though (the only class in Spring that is).

~0002549

Auswaschbar (reporter)

2008-09-01 16:52

 It's gone away. I tried it with buildserv-build from wine and my local build, but no desync any more. Anyone here still getting sync errors?

~0002550

Kloot (developer)

2008-09-01 17:17

 I've unfortunately never been able to trigger it with any trunk revision >= r6320 (r6320 was the earliest I tested with, my own gcc 4.1.3 Linux build vs. bibim's crosscomp buildserv-build run in wine), so I can't confirm that the desync is really gone (just that I haven't seen it). It would be interesting to know which exact rev fixed things though, especially because of the 100% reproducability (there were very few synced code changes recently, so should be doable).

~0002553

imbaczek (reporter)

2008-09-03 00:09

 i cannot reproduce a desync between two boxes, one compiled with gcc 4.3.2, another with bibim's build. needs further testing in 4+ player matches.

~0002558

Auswaschbar (reporter)

2008-09-04 23:15

 After some testing, we could finally track this down and resolve this one
+Notes

-Issue History
Date Modified Username Field Change
2008-08-22 23:12 imbaczek New Issue
2008-08-22 23:13 jK Note Added: 0002486
2008-08-22 23:14 imbaczek File Added: 20080822_224112_SmallDivide_0.76b1+.sdf
2008-08-22 23:14 jK File Added: 20080822_224133_SmallDivide_0.76b1+.sdf
2008-08-22 23:18 imbaczek File Added: desu-smalldivide1.sdf
2008-08-22 23:19 imbaczek File Added: fatality-20080822_224132_SmallDivide_0.76b1+.sdf
2008-08-22 23:27 imbaczek Note Added: 0002488
2008-08-22 23:51 imbaczek Summary major desyncs in CA => major desyncs in all mods
2008-08-22 23:52 imbaczek Severity minor => block
2008-08-23 12:49 bibim_ File Added: bibim-20080822_225057_SmallDivide_0.76b1+.sdf
2008-08-23 12:54 bibim_ Note Added: 0002502
2008-08-23 13:11 imbaczek Note Added: 0002504
2008-08-24 16:02 imbaczek Relationship added related to 0001017
2008-08-24 18:49 Auswaschbar Note Added: 0002512
2008-08-24 21:11 imbaczek Note Added: 0002513
2008-08-24 21:12 imbaczek File Added: host_20080824_210838_SmallDivide_0.76b1+.sdf
2008-08-24 21:12 imbaczek File Added: client_20080824_210843_SmallDivide_0.76b1+.sdf
2008-08-24 21:36 Kloot Note Added: 0002514
2008-08-25 13:53 Kloot Note Added: 0002515
2008-08-25 13:53 Kloot Status new => feedback
2008-08-25 13:55 Kloot Note Edited: 0002515
2008-08-25 13:57 Kloot Note Edited: 0002515
2008-08-25 14:00 imbaczek Note Added: 0002517
2008-08-25 14:00 imbaczek Note Edited: 0002517
2008-08-25 19:07 Kloot Note Added: 0002521
2008-08-25 19:11 Kloot Note Edited: 0002521
2008-08-25 22:18 Kloot Note Edited: 0002521
2008-08-25 22:31 Kloot Note Edited: 0002521
2008-08-25 23:14 Kloot Note Edited: 0002521
2008-08-25 23:14 Kloot Note Edited: 0002521
2008-08-29 01:05 Auswaschbar Note Added: 0002531
2008-08-29 09:29 imbaczek Note Added: 0002539
2008-08-29 13:43 Kloot Note Added: 0002541
2008-09-01 16:52 Auswaschbar Note Added: 0002549
2008-09-01 17:17 Kloot Note Added: 0002550
2008-09-03 00:09 imbaczek Note Added: 0002553
2008-09-04 23:15 Auswaschbar Status feedback => resolved
2008-09-04 23:15 Auswaschbar Resolution open => fixed
2008-09-04 23:15 Auswaschbar Assigned To => Auswaschbar
2008-09-04 23:15 Auswaschbar Note Added: 0002558
+Issue History