View Issue Details [ Jump to Notes ] | [ Issue History ] [ Print ] | ||||||||
ID | Project | Category | View Status | Date Submitted | Last Update | ||||
---|---|---|---|---|---|---|---|---|---|
0001033 | Spring engine | General | public | 2008-09-05 00:12 | 2008-10-14 15:09 | ||||
Reporter | imbaczek | ||||||||
Assigned To | Auswaschbar | ||||||||
Priority | normal | Severity | block | Reproducibility | always | ||||
Status | resolved | Resolution | fixed | ||||||
Product Version | 0.76b1+svn | ||||||||
Target Version | Fixed in Version | ||||||||
Summary | 0001033: desync in 2v2 CA | ||||||||
Description | links to demos: jk: http://home.arcor.de/jkei/20080904_234212_CometCatcherRedux_0.76b1+_jk.sdf satirik: http://filebin.ca/bkyoqf bibim: http://planetspring.free.fr/tmp/20080904_234214_Comet%20Catcher%20Redux_0.76b1+.sdf me (used a different build of same revision): attached first desync of me on frame 6154, then satirik on frame 20504. r6365. | ||||||||
Tags | No tags attached. | ||||||||
Checked infolog.txt for Errors | |||||||||
Attached Files |
|
![]() |
|||||||||||
|
![]() |
|
imbaczek (reporter) 2008-09-05 16:36 |
any luck with using TRACE_SYNC? it doesn't show any desyncs in the frame the message shows up... |
Auswaschbar (reporter) 2008-09-11 17:29 |
trace @ 6154: New frame:6153 1852312857 Unit delta speed: 1343.68 91.7214 7335.11 0.158 8834 Stop moving called: 594.719 91.7188 1459.56 4304 Stop moving called: 142.18 91.7188 1560.39 5878 Start moving called: 192.769 91.7188 1616.85 8094 New frame:6154 2929676732 Unit delta speed: 1659.15 91.7188 4987.25 -1.19209e-07 777 Unit delta speed: 1350.24 91.7188 7335.27 0.00443071 8834 Unit delta speed: 192.769 91.7188 1616.85 0.24 8094 Weapon slow update: 7177 0 Stop moving called: 1965.15 91.7188 576.155 4668 New frame:6155 1189256394 |
Auswaschbar (reporter) 2008-09-11 17:30 |
trace @ 20504: New frame:20503 1832903111 Unit delta speed: 1091.31 91.7188 4169.63 0 2519 Unit delta speed: 216.003 91.7188 3119.22 8.9407e-08 6639 Cannon fire: 338.584 -0.839063 216 93.7202 1576 New explosive: 322.937 98.5553 1508.24 -13.9844 0.0123933 9.06723 Weapon fire: 338.868 97.0816 1504.56 216 93.7202 1576 Explosion: 319.032 3 4 Damage: 5630 2.925 Weapon fire: 320 159.719 1248 321.519 93.7188 1534.97 Unit delta speed: 2398.5 91.9034 5895.09 0.06 1758 Unit delta speed: 1576.7 91.7188 2439.31 0 1255 Weapon slow update: 3295 0 Weapon slow update: 2374 0 Weapon slow update: 8126 0 Weapon slow update: 7819 0 Weapon slow update: 5240 0 Weapon slow update: 8785 0 Weapon slow update: 4703 0 Weapon slow update: 5023 0 Explosion: 245.586 11.64 4 Damage: 5157 2.83725 New frame:20504 4168662495 Unit delta speed: 5512.06 91.7188 1320.04 -5.96046e-08 5698 Unit delta speed: 391.986 91.7188 5070.72 -2.98023e-08 3794 Unit delta speed: 1060.42 91.7188 4128.46 0.06 3177 Unit delta speed: 341.952 91.7188 1432.96 0.00117993 3234 Unit delta speed: 1918.65 91.7188 5568.31 0.06 3289 Unit delta speed: 2159.22 91.7768 5804.93 -0.0636368 7819 Explosion: 319.058 3 4 Damage: 5630 2.925 Weapon fire: 320 159.719 1248 323.884 93.7188 1531.63 Unit delta speed: 2147.2 91.7188 5547.28 1.19209e-07 2426 Unit delta speed: 2394.2 91.899 5894.92 0.0034529 1758 Weapon slow update: 6289 0 Weapon slow update: 8243 0 Weapon slow update: 7251 0 Weapon slow update: 692 0 Weapon slow update: 2071 0 Weapon slow update: 556 0 Explosion: 431.204 11.64 4 Damage: 8955 2.83725 Explosion: 243.035 11.64 4 Damage: 3920 2.83725 New frame:20505 3755125218 |
Auswaschbar (reporter) 2008-09-11 17:34 |
Don't think its very helpfull thought... |
imbaczek (reporter) 2008-09-11 18:36 |
i think we need to give a trace_sync enabled exe to all people involved and have them post the sync trace. playing those replays on my box results in no differences, which doesn't help at all. for the record: New frame:6153 1852312857 Unit delta speed: 1343.68 91.7214 7335.11 0.158 8834 Stop moving called: 594.719 91.7188 1459.56 4304 Stop moving called: 142.18 91.7188 1560.39 5878 Start moving called: 192.769 91.7188 1616.85 8094 New frame:6154 2929676732 Unit delta speed: 1659.15 91.7188 4987.25 -1.19209e-007 777 Unit delta speed: 1350.24 91.7188 7335.27 0.00443071 8834 Unit delta speed: 192.769 91.7188 1616.85 0.24 8094 Weapon slow update: 7177 0 Stop moving called: 1965.15 91.7188 576.155 4668 New frame:6155 1189256394 |
imbaczek (reporter) 2008-09-24 22:16 |
added a desync in after frame 17000 with r6453. |
imbaczek (reporter) 2008-09-24 22:17 |
...and a diff --- trace0.log 2008-09-24 22:04:36.875000000 +0200 +++ trace3.log 2008-09-24 22:06:34.000000000 +0200 @@ -604798,7 +604798,7 @@ New explosive: 5564.53 100.535 1056.06 15.6285 2.56766 5.18997 Weapon fire: 5557.03 97.7188 1063.31 5699.45 112.593 1099.37 ObstacleAvoidance avoidanceVec = 0 0 0 -Unit 5632 changed heading to -22623 from -22031 +Unit 5632 changed heading to -22257 from -22031 Unit delta speed: 5694.84 91.7188 880.259 0.06 5632 ObstacleAvoidance avoidanceVec = 0.410821 0 -0.0592971 Unit 5563 changed heading to 31600 from 32059 |
imbaczek (reporter) 2008-09-24 23:10 |
btw desync doesn't occur when watching the replay in host replay with the same people, but i think it's because they crash. |
imbaczek (reporter) 2008-09-27 22:10 Last edited: 2008-09-27 22:27 |
some more testing -Unit 9431 changed heading to 32749 from -32767 (wantedHeading: 32749) +Unit 9431 changed heading to 31625 from -32767 (wantedHeading: 25679) ObstacleAvoidance avoidanceVec = 0 0 0 added replay, traces start to differ in frame 5355, but a desync is detected only in frame 5361... very strange. ah the most important thing: Server: chk AD1640A4 instead of E5E323C4 frame 005356, (value=0000ffec 9.18074702e-41) backtrace 1 in "copysigned short" Server: chk 025BD314 instead of 328C16EC frame 005356, (value=00007fed 4.58911234e-41) backtrace 2 in "+=signed short" backtraces Server: === Backtrace 1 === Server: #0 ?? [K:\progz\spring/rts/Sim/MoveTypes/GroundMoveType.cpp:642] Server: #1 ?? [K:\progz\spring/rts/Sim/MoveTypes/GroundMoveType.cpp:319] Server: #2 ?? [K:\progz\spring/rts/Sim/Units/Unit.cpp:529] Server: 0000003 ?? [K:\progz\spring/rts/Sim/Units/UnitHandler.cpp:276] Server: 0000004 ?? [K:\progz\spring/rts/Game/Game.cpp:3135] Server: === Backtrace 2 === Server: #0 oggpackB_get_buffer [e:/projects/libogg-1.1/src/bitwise.c:447] Server: #1 ?? [K:\progz\spring/rts/Sim/MoveTypes/GroundMoveType.cpp:642] Server: #2 ?? [K:\progz\spring/rts/Sim/MoveTypes/GroundMoveType.cpp:319] Server: 0000003 ?? [K:\progz\spring/rts/Sim/Units/Unit.cpp:529] Server: 0000004 ?? [K:\progz\spring/rts/Sim/Units/UnitHandler.cpp:276] addr2line gets a little bit confused, but the first checksum refers to a copy ctor, so it has to be 641: ASSERT_SYNCED_PRIMITIVE((short)std::max((short) - turnRate, deltaHeading)); testing -(short)turnrate and (short)-turnrate for sync now. |
imbaczek (reporter) 2008-09-27 23:50 |
min=-1 max=-1ChangeHeading desired velocity after=-8245 ObstacleAvoidance avoidanceVec = 0 0 0 -ChangeHeading desired velocity 0.00181914 -0.999998 heading 32749 -Unit 9431 changed heading to 32749 from -32767 (wantedHeading: 32749) -deltaheading=-20 turnrate=1144 (short)-turnrate=-1144 -min=-20 max=-20ChangeHeading desired velocity after=32749 -ObstacleAvoidance avoidanceVec = 0 0 0 -ChangeHeading desired velocity 0.385175 -0.922844 heading 28617 -Unit 5309 changed heading to 29325 from 30469 (wantedHeading: 28617) -deltaheading=-1852 turnrate=1144 (short)-turnrate=-1144 -min=-1852 max=-1144ChangeHeading desired velocity after=29325 +ChangeHeading desired velocity 0.625623 -0.780126 heading 25679 +Unit 9431 changed heading to 31625 from -32767 (wantedHeading: 25679) +deltaheading=-7090 turnrate=1144 (short)-turnrate=-1144 +min=-7090 max=-1144ChangeHeading desired velocity after=31625 +ObstacleAvoidance avoidanceVec = 0 0 0 +ChangeHeading desired velocity 0.789625 -0.613589 heading 23316 +Unit 5309 changed heading to 29325 from 30469 (wantedHeading: 23316) +deltaheading=-7153 turnrate=1144 (short)-turnrate=-1144 +min=-7153 max=-1144ChangeHeading desired velocity after=29325 New frame:5356 3825650988 ObstacleAvoidance avoidanceVec = 0 0 0 ChangeHeading desired velocity -0.708024 0.706188 heading -8247 seems my asserts don't work as advertised... good that the replay gives repeatable results on the same box. further testing in progress. |
Kloot (developer) 2008-09-27 23:59 Last edited: 2008-09-28 00:02 |
"testing -(short)turnrate and (short)-turnrate for sync now." Those two are not equivalent for all inputs only when storing the result of the cast in a larger type, specifically for all odd integer multiples of 32768 (3, 5, 7...): int h = 32768; int j = -(short) h; int k = (short) -h; h: 32768, j: 32768, k: -32768 h: 98304, j: 32768, k: -32768 h: 163840, j: 32768, k: -32768 ... The casting behavior itself should be identical though. |
imbaczek (reporter) 2008-09-28 00:09 |
ObstacleAvoidance receives different vectors, tracing further back up... |
Kloot (developer) 2008-09-28 00:14 |
I removed the casting inconsistency in the meantime, good hunting. |
imbaczek (reporter) 2008-09-28 00:43 Last edited: 2008-09-28 00:45 |
got the lower and upper bound of the source of the error after adding even more logging: CUnit::Update before pos 3255.96 91.7188 1276 Unit 9431 CGroundMoveType::Update pos before update 3255.96 91.7188 1276 -ObstacleAvoidance desiredDir=0.00181914 0 -0.999998 +ObstacleAvoidance desiredDir=0.625623 0 -0.780126 ^^^^ this means that one of ObstacleAvoidance's argument components has sync issues; this can be either waypoint or owner->pos. since owner->pos looks ok, this must be the waypoint. looking deeper still, will nevertheless trace both. avoid L=0 R=0 processing 5309 at 3296.32 91.7188 1082.4 ^^^ don't be confused by this, this is the second jeffy in the raiding group. ObstacleAvoidance avoidanceVec = 0 0 0 -ChangeHeading desired velocity 0.00181914 -0.999998 heading 32749 -Unit 9431 changed heading to 32749 from -32767 (wantedHeading: 32749) -deltaheading=-20 turnrate=1144 (short)-turnrate=-1144 -min=-20 max=-20 -ChangeHeading desired velocity after=32749 +ChangeHeading desired velocity 0.625623 -0.780126 heading 25679 +Unit 9431 changed heading to 31625 from -32767 (wantedHeading: 25679) +deltaheading=-7090 turnrate=1144 (short)-turnrate=-1144 +min=-7090 max=-1144 +ChangeHeading desired velocity after=31625 CUnit::Update before pos 344 91.7188 7176 CUnit::Update before pos 3296.32 91.7188 1082.4 Unit 5309 CGroundMoveType::Update pos before update 3296.32 91.7188 1082.4 |
imbaczek (reporter) 2008-09-28 01:05 |
yup Unit 9431 CGroundMoveType::Update pos before update 3255.96 91.7188 1276 -waypoint 3256 91.7188 1256 -ObstacleAvoidance desiredDir=0.00181914 0 -0.999998 +waypoint 3272 91.7188 1256 +ObstacleAvoidance desiredDir=0.625623 0 -0.780126 3256 and 3272 look awfully off-by-one-ish... |
imbaczek (reporter) 2008-09-28 01:26 Last edited: 2008-09-28 01:28 |
entering uncharted territory, the pathfinder. instrumented CPathManager::NextWaypoint to output logs - cached path is this part } else { waypoint = multiPath->detailedPath.path.back(); multiPath->detailedPath.path.pop_back(); } Start moving called: 3296.32 91.7188 1082.4 5309 -Multipath cached path 3304 91.7188 1064 -Unit 5309 pos 3296.32 91.7188 1082.4 waypoint=3296.32 91.7188 1082.4 next waypoint=3304 91.7188 1064 -Multipath cached path 3304 91.7188 1048 -Unit 5309 pos 3296.32 91.7188 1082.4 waypoint=3304 91.7188 1064 next waypoint=3304 91.7188 1048 +Multipath cached path 3320 91.7188 1064 +Unit 5309 pos 3296.32 91.7188 1082.4 waypoint=3296.32 91.7188 1082.4 next waypoint=3320 91.7188 1064 +Multipath cached path 3336 91.7188 1048 +Unit 5309 pos 3296.32 91.7188 1082.4 waypoint=3320 91.7188 1064 next waypoint=3336 91.7188 1048 Start moving called: 3255.96 91.7188 1276 9431 -Multipath cached path 3256 91.7188 1256 -Unit 9431 pos 3255.96 91.7188 1276 waypoint=3255.96 91.7188 1276 next waypoint=3256 91.7188 1256 -Multipath cached path 3256 91.7188 1240 -Unit 9431 pos 3255.96 91.7188 1276 waypoint=3256 91.7188 1256 next waypoint=3256 91.7188 1240 +Multipath cached path 3272 91.7188 1256 +Unit 9431 pos 3255.96 91.7188 1276 waypoint=3255.96 91.7188 1276 next waypoint=3272 91.7188 1256 +Multipath cached path 3288 91.7188 1240 +Unit 9431 pos 3255.96 91.7188 1276 waypoint=3272 91.7188 1256 next waypoint=3288 91.7188 1240 New frame:5355 3825650988 CUnit::Update before pos 5960 91.7188 1160 CUnit::Update before pos 5698.75 91.7188 1192 @@ -164216,26 +164216,26 @@ CUnit::Update before pos 400 91.7188 7216 CUnit::Update before pos 3255.96 91.7188 1276 Unit 9431 CGroundMoveType::Update pos before update 3255.96 91.7188 1276 -waypoint 3256 91.7188 1256 -ObstacleAvoidance desiredDir=0.00181914 0 -0.999998 +waypoint 3272 91.7188 1256 +ObstacleAvoidance desiredDir=0.625623 0 -0.780126 note this is full 2 frames before desync is detected. IMHO needs attention (I've changed waypoint and nextWaypoint to syncedfloat3 locally.) btw this is getting scary, I'm historically afraid of the pathfinder ^_^ |
imbaczek (reporter) 2008-09-28 01:47 |
compared cached path estimations, files have different size, investigating |
imbaczek (reporter) 2008-09-28 02:16 |
after copying my path files from maps/paths to the ragnar's spring installation, traces stopped to differ. he's got a single core athlon xp 2600+, I've got a Core2 E8400. suggested remedies - hash the path files as synced values, so the desync is immediately on frame 1; also, fix the path estimator, obviously. I'm onto the former, not enough expertise for the latter. |
Kloot (developer) 2008-09-28 12:23 |
Back when I threaded the estimator I ran many delta-checks between an Athlon 4000+ and an X2 4600+, obviously must have missed something. I'll look into it. |
imbaczek (reporter) 2008-09-28 12:53 |
I'm not sure that's because of threading, haven't run 1-thread-forced tests with rag yet. That's the first suspect on my list, though. |
Kloot (developer) 2008-09-28 13:11 |
For reference, can you attach your generated (mt'ed) path data for CCR? |
imbaczek (reporter) 2008-09-28 13:26 |
here they are: http://imbaczek.wrzuta.pl/pliki/jcNh7xhTnC/ |
Kloot (developer) 2008-09-28 13:52 Last edited: 2008-09-28 14:29 |
Cheers. There's a rather large difference in filesize between your paths and mine: me: (generated by the Athlon 4000+, forced to a single thread) 518627 2008-09-28 12:33 Comet Catcher Redux.2849637338.pe.zip 28340 2008-09-28 12:33 Comet Catcher Redux.2849637362.pe2.zip you: 519042 2008-09-14 23:16 Comet Catcher Redux.2849637338.pe.zip 28390 2008-09-14 23:16 Comet Catcher Redux.2849637362.pe2.zip However, the path data for CCR writen by *76b1* looks like the 76svn paths: 518627 2008-09-28 12:34 Comet Catcher Redux.2849637338.pe.zip 28340 2008-09-28 12:34 Comet Catcher Redux.2849637362.pe2.zip But after unzipping and diffing, it turns out my 76b1 data is not identical to that generated by 76svn, so there are potentially two separate bugs at work here. edit 1: tested with BA 6.41 edit 2: the mismatch between 76b1 and 76svn above is due to PATHESTIMATOR_VERSION having been bumped from 34 to 35, but I did another test where the pe2's (strangely not the pe's) of -b1 and -svn weren't in sync so it's not very promising. |
Kloot (developer) 2008-09-28 16:46 |
I found the bug causing the 76b1 vs. 76svn diffs, but it was pretty severe (to the extent that NONE of the path data generated by any SVN install can be trusted). SVN 0006471 will forcibly regenerate the paths, if there is also a threading issue hiding somewhere it'll be easy to spot from now on. |
imbaczek (reporter) 2008-09-28 16:49 |
cool. we can announce big tests now. |
Auswaschbar (reporter) 2008-09-28 23:38 Last edited: 2008-09-29 00:01 |
I uploaded pathfiles generated with version 6474 (BA 6.41) on an 64bit linux, forced to 1 thread. If I run a 32bit exe in wine, the generated paths are the same (== md5sum is equivalent) (2 threads). However baczek seems to have different ones. I ran the same rev on my notebook (32bit linux) and the checksums are the same as my others. |
Auswaschbar (reporter) 2008-09-29 00:58 |
I tried with bibim and satirik: they both have the same checksumms while I get a different one (tried both 32bit wine and 64bit linux). They both have more than one thread running (just like me). |
bibim_ (reporter) 2008-09-29 01:08 Last edited: 2008-09-29 01:10 |
I just tried forcing the thread count to 1, and I get the same checksum as you have ! So it seems path estimator multithreading is broken on windows... |
imbaczek (reporter) 2008-09-29 10:16 Last edited: 2008-09-29 10:18 |
5c62f9770027b2e50c9848eda7a09f97 *Comet Catcher Redux.2849637340.pe.zip 78597d5833fe0a5062084010b17d3052 *Comet Catcher Redux.2849637364.pe2.zip with numthreads set to 1. with multithreading enabled: 1356c9f87c04bbc823173140145ceff9 *Comet Catcher Redux.2849637340.pe.zip 51faea14409afb445b561bc8381e2d72 *Comet Catcher Redux.2849637364.pe2.zip those are repeatable values. dual core cpu. |
bibim_ (reporter) 2008-09-29 12:05 |
Here are my different CCR checksums with r6476 on Windows: HardwareThreadCount=1: [ 0] Pathing data checksum: 37b08ad1 HardwareThreadCount=2: [ 0] Pathing data checksum: aa710f6d |
imbaczek (reporter) 2008-09-29 14:00 |
temporarily disabled threading in the path estimator, please remember to re-enable. |
Auswaschbar (reporter) 2008-10-14 15:09 |
fixed. |
![]() |
|||
Date Modified | Username | Field | Change |
---|---|---|---|
2008-09-05 00:12 | imbaczek | New Issue | |
2008-09-05 00:12 | imbaczek | File Added: baczek_20080904_234201_Comet Catcher Redux_0.76b1+.sdf | |
2008-09-05 00:12 | imbaczek | Description Updated | |
2008-09-05 00:16 | imbaczek | Description Updated | |
2008-09-05 00:18 | imbaczek | Description Updated | |
2008-09-05 00:28 | bibim_ | File Added: bibim_20080904_234214_Comet Catcher Redux_0.76b1+.sdf | |
2008-09-05 00:35 | imbaczek | File Added: 1033_satirik.sdf | |
2008-09-05 16:36 | imbaczek | Note Added: 0002562 | |
2008-09-11 17:29 | Auswaschbar | Note Added: 0002580 | |
2008-09-11 17:30 | Auswaschbar | Note Added: 0002581 | |
2008-09-11 17:34 | Auswaschbar | Note Added: 0002582 | |
2008-09-11 18:36 | imbaczek | Note Added: 0002583 | |
2008-09-15 00:13 | imbaczek | File Added: syncdebug-server-1033.log | |
2008-09-23 20:55 | Auswaschbar | File Added: syncdebug-server.log | |
2008-09-23 20:58 | satirik | File Added: trace1.log | |
2008-09-23 20:59 | satirik | File Added: syncdebug-client.log | |
2008-09-23 20:59 | satirik | File Added: infolog.txt | |
2008-09-24 22:13 | imbaczek | File Added: 2-trace0.zip | |
2008-09-24 22:15 | imbaczek | File Added: 2-trace3.zip | |
2008-09-24 22:15 | imbaczek | File Added: syncdebug-server-2.log | |
2008-09-24 22:16 | imbaczek | File Added: 20080924_215230_Comet Catcher Redux_0.76b1+.sdf | |
2008-09-24 22:16 | imbaczek | Note Added: 0002616 | |
2008-09-24 22:17 | imbaczek | Note Added: 0002617 | |
2008-09-24 23:10 | imbaczek | Note Added: 0002618 | |
2008-09-25 00:31 | imbaczek | Relationship added | related to 0001055 |
2008-09-27 22:10 | imbaczek | Note Added: 0002624 | |
2008-09-27 22:10 | imbaczek | File Added: 20080927_215550_Comet Catcher Redux_0.76b1+.sdf | |
2008-09-27 22:13 | imbaczek | Note Edited: 0002624 | |
2008-09-27 22:27 | imbaczek | Note Edited: 0002624 | |
2008-09-27 23:50 | imbaczek | Note Added: 0002625 | |
2008-09-27 23:59 | Kloot | Note Added: 0002626 | |
2008-09-28 00:01 | Kloot | Note Edited: 0002626 | |
2008-09-28 00:02 | Kloot | Note Edited: 0002626 | |
2008-09-28 00:09 | imbaczek | Note Added: 0002627 | |
2008-09-28 00:14 | Kloot | Note Added: 0002628 | |
2008-09-28 00:43 | imbaczek | Note Added: 0002629 | |
2008-09-28 00:45 | imbaczek | Note Edited: 0002629 | |
2008-09-28 01:05 | imbaczek | Note Added: 0002630 | |
2008-09-28 01:26 | imbaczek | Note Added: 0002631 | |
2008-09-28 01:28 | imbaczek | Note Edited: 0002631 | |
2008-09-28 01:47 | imbaczek | Note Added: 0002632 | |
2008-09-28 02:16 | imbaczek | Note Added: 0002633 | |
2008-09-28 12:23 | Kloot | Note Added: 0002634 | |
2008-09-28 12:53 | imbaczek | Note Added: 0002635 | |
2008-09-28 13:11 | Kloot | Note Added: 0002636 | |
2008-09-28 13:26 | imbaczek | Note Added: 0002637 | |
2008-09-28 13:52 | Kloot | Note Added: 0002638 | |
2008-09-28 13:55 | Kloot | Note Edited: 0002638 | |
2008-09-28 14:29 | Kloot | Note Edited: 0002638 | |
2008-09-28 14:29 | Kloot | Note Edited: 0002638 | |
2008-09-28 16:46 | Kloot | Note Added: 0002639 | |
2008-09-28 16:49 | imbaczek | Note Added: 0002640 | |
2008-09-28 23:34 | Auswaschbar | File Added: Comet Catcher Redux.2849637340.pe.zip | |
2008-09-28 23:36 | Auswaschbar | File Added: Comet Catcher Redux.2849637364.pe2.zip | |
2008-09-28 23:38 | Auswaschbar | Note Added: 0002644 | |
2008-09-28 23:41 | Auswaschbar | Note Edited: 0002644 | |
2008-09-29 00:01 | Auswaschbar | Note Edited: 0002644 | |
2008-09-29 00:58 | Auswaschbar | Note Added: 0002646 | |
2008-09-29 01:08 | bibim_ | Note Added: 0002647 | |
2008-09-29 01:10 | bibim_ | Note Edited: 0002647 | |
2008-09-29 10:16 | imbaczek | Note Added: 0002648 | |
2008-09-29 10:17 | imbaczek | Note Edited: 0002648 | |
2008-09-29 10:18 | imbaczek | Note Edited: 0002648 | |
2008-09-29 12:05 | bibim_ | Note Added: 0002649 | |
2008-09-29 14:00 | imbaczek | Note Added: 0002650 | |
2008-09-29 15:51 | Auswaschbar | Relationship added | related to 0001061 |
2008-10-14 15:09 | Auswaschbar | Status | new => resolved |
2008-10-14 15:09 | Auswaschbar | Resolution | open => fixed |
2008-10-14 15:09 | Auswaschbar | Assigned To | => Auswaschbar |
2008-10-14 15:09 | Auswaschbar | Note Added: 0002880 |