2019-08-23 18:19 CEST

View Issue Details Jump to Notes ]
IDProjectCategoryView StatusLast Update
0006163Spring engineLuapublic2019-06-30 10:47
ReporterGoogle_Frog 
Assigned ToKloot 
PrioritynormalSeveritycrashReproducibilityhave not tried
StatusresolvedResolutionfixed 
Product Version104.0 +git 
Target VersionFixed in Version104.0 +git 
Summary0006163: 104.0.1-1060-g7f3541f maintenance VFS seemed to misplace files inside a sdz during a game
DescriptionI received some desync reports caused by a missing file. The surprising thing is that this missing file should cause the game to desync immediately, yet it did not. katastrophe had the missing file and, luckily, sent a report: https://github.com/ZeroK-RTS/CrashReports/issues/20580

Some sort of buffering occurs during the game. It initially fails to find some files:

[f=0029249] [~CBufferedArchive][name=C:\Users\Katastrophe\Documents\My Games\Zero-K\games\c6384de84018c64982268a7a399db121.sdz] 294516711 bytes cached in 3697 files
[f=0029415] <snd_noises.lua>: Error file Sounds/reply/windmill.WAV doesn't exist.
[f=0029475] <snd_noises.lua>: Error file Sounds/reply/hovercraft_select.WAV doesn't exist.

Later this occurs:

[f=0030045] Error: [LuaRules::RunCallInTraceback] error=2 (LUA_ERRRUN) callin=UnitCreated trace=[Internal Lua error: Call failure] Include() could not load 'LuaRules/Configs/customcmds.h.lua'
stack traceback:
    [C]: in function 'sp_CallAsUnit'
    [string "LuaRules/Gadgets/unit_script.lua"]:311: in function 'CallAsUnitNoReturn'
    [string "LuaRules/Gadgets/unit_script.lua"]:737: in function 'UnitCreated'
    [string "LuaRules/gadgets.lua"]:1432: in function <[string "LuaRules/gadgets.lua"]:1427>
    (tail call): ?

The game would be unplayable if 'LuaRules/Configs/customcmds.h.lua' was missing for the entire game, so VFS seemed to lose the file during the game.

Here are reports from some other players in the game, gathered because the missing file caused desync:
 - https://github.com/ZeroK-RTS/CrashReports/issues/20572
 - https://github.com/ZeroK-RTS/CrashReports/issues/20574
 - https://github.com/ZeroK-RTS/CrashReports/issues/20575
 - https://github.com/ZeroK-RTS/CrashReports/issues/20577
 - https://github.com/ZeroK-RTS/CrashReports/issues/20587

The same thing seemed to occur to Godde in a separate game, but I don't have his report. All I have is this desync report from another player in the game: https://github.com/ZeroK-RTS/CrashReports/issues/20559
TagsNo tags attached.
Checked infolog.txt for lua ErrorsYes
Attached Files

-Relationships
+Relationships

-Notes

~0019837

Kloot (developer)

There is a config option (VFSCacheArchiveFiles) to turn off buffering, set that to 0 when you push the next ZK update. This may explain many past desyncs.

~0019838

Google_Frog (reporter)

https://github.com/ZeroK-RTS/Chobby/commit/02a62484d172cc485273639946167f7b450dd496

VFSCacheArchiveFiles is not in https://springrts.com/wiki/Springsettings.cfg, but I found it in the sources as a springsetting, so that is how I have set it.

~0019860

Google_Frog (reporter)

Here is the issue on 104.0.1-1085-gfc6d80e, but I am unsure whether VFSCacheArchiveFiles requires a restart to apply.
 * https://github.com/ZeroK-RTS/CrashReports/issues/20672
 * https://github.com/ZeroK-RTS/CrashReports/issues/20674

~0019861

Kloot (developer)

It does, but 2067{2,4} seem to be duplicates of 0006157.

~0019865

Google_Frog (reporter)

Here is another, but it occurred 14 hours ago so there may have been insufficient restarts for the settings update.
 * https://github.com/ZeroK-RTS/CrashReports/issues/20686 - "Just as I observed this happening, the unit commands UI panel also started rendering white."
 * https://github.com/ZeroK-RTS/CrashReports/issues/20687
 * https://github.com/ZeroK-RTS/CrashReports/issues/20688
The first one is from the player who had the VFS issue, and commented on the ticket that UI panel textures were lost as well.

~0019870

Kloot (developer)

Last edited: 2019-03-17 21:19

View 2 revisions

from the ~CBufferedArchive lines in darloth's log (20686) we can infer the settings change hadn't percolated yet.

textures vanishing is a sign of memory corruption.

~0019901

Google_Frog (reporter)

I have seen two recent instances of the issue, so it is possibly not resolved.

The first occurrence has someone lose customcmds, which would definitely have crashed the game earlier if it was missing from the start of the game. The affected player didn't submit a report though.
 * https://github.com/ZeroK-RTS/CrashReports/issues/21081
 * https://github.com/ZeroK-RTS/CrashReports/issues/21082
 * https://github.com/ZeroK-RTS/CrashReports/issues/21084
[f=0014943] [Game::ClientReadNet][LOGMSG] sender="Hellaciouss" string="[Internal Lua error: Call failure] Include() could not load 'LuaRules/Configs/customcmds.h.lua'
stack traceback:
    [C]: in function 'sp_CallAsUnit'
    [string "LuaRules/Gadgets/unit_script.lua"]:317: in function 'CallAsUnitNoReturn'
    [string "LuaRules/Gadgets/unit_script.lua"]:756: in function 'UnitCreated'
    [string "LuaRules/gadgets.lua"]:1398: in function <[string "LuaRules/gadgets.lua"]:1393>
    (tail call): ?"
[f=0014943] Sync error for Hellaciouss in frame 14925 (got 5ff22da1, correct is e508c608)

The second occurrence may be unrelated as the player is unable to load the model for which the error later occurs.
 * https://github.com/ZeroK-RTS/CrashReports/issues/21091 - Report for the affected player (Reapo), has VFSCacheArchiveFiles = 0.
[f=0021312] game_message: Metal Extractor is under attack
[f=0021500] Error: [LuaRules::RunCallInTraceback] error=2 (LUA_ERRRUN) callin=UnitCreated trace=[Internal Lua error: Call failure] [string "scripts/turretgauss.lua"]:4: piece not found: Concrete
stack traceback:
    [C]: in function 'sp_CallAsUnit'
    [string "LuaRules/Gadgets/unit_script.lua"]:317: in function 'CallAsUnitNoReturn'
    [string "LuaRules/Gadgets/unit_script.lua"]:756: in function 'UnitCreated'
    [string "LuaRules/gadgets.lua"]:1398: in function <[string "LuaRules/gadgets.lua"]:1393>
    (tail call): ?
[f=0021511] [Game::ClientReadNet][LOGMSG] sender="Reapo" string="[Internal Lua error: Call failure] [string "scripts/turretgauss.lua"]:4: piece not found: Concrete
stack traceback:
    [C]: in function 'sp_CallAsUnit'
    [string "LuaRules/Gadgets/unit_script.lua"]:317: in function 'CallAsUnitNoReturn'
    [string "LuaRules/Gadgets/unit_script.lua"]:756: in function 'UnitCreated'
    [string "LuaRules/gadgets.lua"]:1398: in function <[string "LuaRules/gadgets.lua"]:1393>
    (tail call): ?"
[f=0021521] Sync error for Reapo in frame 21500 (got d84785b3, correct is 9c68bbdc)

~0019904

Kloot (developer)

The customcmds.h.lua error always originates from unit_script which performs some include caching trickery not typically seen in other gadgets, unlikely to be a coincidence. Knowing under what exact conditions it triggers would help.

~0019939

Kloot (developer)

I need debug output from 1209-gcef61f0 or later.

~0019962

Kloot (developer)

No further response and a version rollback, UTR.

~0019963

Google_Frog (reporter)

There will be a response once we manage to update. When I last tried it became apparent that the ZK infrastructure would update the engine for some players, but not the default game version. I found out because the changes to GetTeamInfo broke compatibility, resulting in too many fatal crashes for my liking.

~0020013

Kloot (developer)

This is either fixed or traceable to the root cause in 1273-ge1c1143, updating would be appreciated.
+Notes

-Issue History
Date Modified Username Field Change
2019-03-13 05:31 Google_Frog New Issue
2019-03-13 14:14 Kloot Note Added: 0019837
2019-03-13 14:14 Kloot Status new => feedback
2019-03-13 14:55 Google_Frog Note Added: 0019838
2019-03-13 14:55 Google_Frog Status feedback => new
2019-03-16 11:05 Google_Frog Note Added: 0019860
2019-03-16 12:30 Kloot Note Added: 0019861
2019-03-17 08:10 Google_Frog Note Added: 0019865
2019-03-17 21:19 Kloot Note Added: 0019870
2019-03-17 21:19 Kloot Note Edited: 0019870 View Revisions
2019-03-31 00:48 Google_Frog Note Added: 0019901
2019-03-31 03:12 Kloot Note Added: 0019904
2019-04-14 12:20 Kloot Note Added: 0019939
2019-04-14 12:20 Kloot Status new => feedback
2019-04-27 10:43 Kloot Status feedback => closed
2019-04-27 10:43 Kloot Resolution open => unable to reproduce
2019-04-27 10:43 Kloot Note Added: 0019962
2019-04-27 11:28 Google_Frog Status closed => feedback
2019-04-27 11:28 Google_Frog Resolution unable to reproduce => reopened
2019-04-27 11:28 Google_Frog Note Added: 0019963
2019-06-30 10:47 Kloot Assigned To => Kloot
2019-06-30 10:47 Kloot Status feedback => resolved
2019-06-30 10:47 Kloot Resolution reopened => fixed
2019-06-30 10:47 Kloot Fixed in Version => 104.0 +git
2019-06-30 10:47 Kloot Note Added: 0020013
+Issue History