| View Issue Details [ Jump to Notes ] | [ Issue History ] [ Print ] | ||||||||
| ID | Project | Category | View Status | Date Submitted | Last Update | ||||
|---|---|---|---|---|---|---|---|---|---|
| 0004493 | Spring engine | General | public | 2014-08-07 00:51 | 2014-08-26 02:53 | ||||
| Reporter | silentwings | ||||||||
| Assigned To | jK | ||||||||
| Priority | normal | Severity | major | Reproducibility | always | ||||
| Status | resolved | Resolution | fixed | ||||||
| Product Version | 97.0.1+git | ||||||||
| Target Version | 98.0 | Fixed in Version | |||||||
| Summary | 0004493: large performance drop between 97.0.1-20 -> 97.0.1-23 | ||||||||
| Description | I made a benchmarking sdz, which replays 90 seconds of a recorded (intense) battle, and is portable between engine versions. Whilst doing this it takes fps/simspeed samples and then at the end it prints their mean and std dev. The system shows a large drop in sim speed between 97.0.1-20 and 97.0.1-23. These were the commits that were also responsible for the PFS-update related jitter (http://springrts.com/mantis/view.php?id=4461). In particular, for the means, it gives 97.0.1-20: FPS 19.9, SIM 0.72 97.0.1-23: FPS 20.7, SIM 0.55 The current dev -206 gives ~equal readings to -23, meaning the although the jitter was fixed (didn't test it yet) the perf loss was not. FWIW: Spring 97.0 gives ~equal readings to -23. For 96.0 I get FPS 19.1 and SIM 0.86. I haven't yet looked much in between 96.0 abd 97.0 to find out what was responsible for that. | ||||||||
| Steps To Reproduce | Download sdz from http://www71.zippyshare.com/v/76018093/file.html, start a game with at least 2 allyteams, a high unit limit, start the game and then type '\luarules loadunits'. | ||||||||
| Additional Information | The benchmark is using a system I wrote for recording & replaying games in a portable (between engine versions) format. It saves the unit positions at the start, and saves the orders given to them (into lua tables in files). The benchmarker simply places the units and then issues the orders given to them in realtime. It doesn't record absolutely everything. The benchmarker only takes fps/simspeed samples after the first 20s has passed, to give spring some time to adapt. It is extremely consistent (at least on my laptops) and only 2-3 runs per engine version are needed to get an accurate benchmark. If you think the benchmark is unrealistic, in a small or a big way, please tell me why! I've attached the output I got from doing a few runs with it, in a text file. | ||||||||
| Tags | No tags attached. | ||||||||
| Checked infolog.txt for Errors | |||||||||
| Attached Files |
| ||||||||
 Relationships |
|
 Relationships |
| Notes |
|
|
silentwings (reporter) 2014-08-07 02:04 |
"FWIW: Spring 97.0 gives ~equal readings to -23" should have read -20. |
|
silentwings (reporter) 2014-08-10 14:10 |
I forgot to mention that to run the benchmark the map must be DSD :D |
|
silentwings (reporter) 2014-08-25 13:39 |
I presume the recent changes to PF/PE mean that this ticket is now either out of date or fixed - close it if either jK/kloot agrees. |
| Notes |
| Issue History |
|||
| Date Modified | Username | Field | Change |
|---|---|---|---|
| 2014-08-07 00:51 | silentwings | New Issue | |
| 2014-08-07 00:51 | silentwings | File Added: 96-98 benchmarking.txt | |
| 2014-08-07 02:04 | silentwings | Note Added: 0013485 | |
| 2014-08-10 14:10 | silentwings | Note Added: 0013501 | |
| 2014-08-16 16:17 | abma | Target Version | => 98.0 |
| 2014-08-25 13:39 | silentwings | Note Added: 0013566 | |
| 2014-08-26 02:53 | jK | Status | new => resolved |
| 2014-08-26 02:53 | jK | Resolution | open => fixed |
| 2014-08-26 02:53 | jK | Assigned To | => jK |
| Issue History |