Some graphics testing results

[SpliFF]

Argh, keep in mind you did state you run AGP graphics which makes your system pretty low-end in gaming standards. From memory your card was one of the last to support AGP and I'm going to assume your CPU is an early P4 and your memory DDR (1) or even SDRAM, you haven't said. At a guess your benchmarks appear to apply to a system at least 5 - 10 years old. That's a long time in "gaming years" and is below the average I see in Springs Lobby (typically dual-core systems, which implies PCI-E and DDR2/3).

I see no reason why Spring shouldn't be able to run on your system but I also see no reason to make assumptions from your benchmarks and apply them to Spring development in general. The only way to turn the data you're generating into practical results is to polish these benchmark maps/scripts/units you're creating into a generalised Spring test mod.

Basically I'm thinking either a replay or a scripted map (preferable) that can be run under different system configurations and settings to create data that can be independently compared and verified.

As Kloot pointed out your assumptions, while entertaining, are of little value. Data is far more important to the scientific method.

[Argh]

Yeah, we need to test apples vs. apples here, I agree- this is frustrating.

Here ya go. This uses the old WB source, with one minor optimization that is in the new one. It does not feature normalmaps, it's not using the new Units with LUPS stuff going on, so it's not a really fair comparision, but it's a hell of a lot better than us all testing different things.

Wait 30 minutes from this post's timestamp, it's FTP'ing while I write this, and it's about 60MB.
http://www.wolfegames.com/TA_Section/Tester1.zip

I'll put out the current WIP with the normalmaps and a different testmap (water, heh) when I get home Sunday. Unfortunately, I won't get to do this test again, on this rig, until Tuesday.

Here's what I see with it, on a dual-core Athlon 64 6000, 2GB DDR2, Spring itself is on a RAID0... and a GeForce 9800GT.

FPS is about 3X faster overall, never goes below 68FPS for a full-map view.

Nobody in their right mind would be complaining about performance of this empty scene (I removed pathfinding actors to keep the waters un-muddied), it's faster than the monitor can handle. Of course, in terms of modern RTS games, it's kinda sparse, un-detailed, and there's nothing going on. So... that 90FPS I see looks impressive, until I think about a DoW 2 scene, where it's much more detailed.

CPU load is mainly showing "shadows / reflect" and when I shut off the shadows, then that load disappears, which is a very different result- the problem is, IDK whether that means I'm truly GPU limited, which I really doubt, or whether it's OpenGL stuff piling up in the queue on the machine at home- I know that if I put further load on the CPU, though, performance goes into the toilet, fast, which is why I keep poking at that side of things, even though I know it irritates y'all.

I still see the same FPS hit, in the same places, it just isn't anywhere near as crippling. It's really interesting how regional it is- that part remains the same, on both platforms. The roads / groundplates are still fairly expensive- in one area of the scene, I saw a 50% increase in FPS merely by getting rid of the road objects (just left-click and drag to grab, ctrl-d, they're Units), even though the terrain was flat in that part of the scene (a waste of time on my part, thought it would help performance but it doesn't).

Oh, and btw, the skybox appears to be screwed up ATM, that sphere shows the top of the skybox is... the bottom. Weird.

So, conclusions? IDK. This machine is obviously not bottlenecked anywhere, so long as I leave the CPU alone.

So, I stressed the CPU, or at least, tried to, by blowing up six skyscrapers. Nothing doing, it was fine.

Then I blew up 100 of my "robocars"- instant death on the Athlon XP. Nothing doing, FPS dropped to... 54. Hardly a crisis. At this point, I'm getting weirded out, frankly- something's changed, this box didn't use to handle that mess quite as well.

So, I guess it's just the box just plain runs that much faster, unless there's something weird going on with my hardware, or one of the updates to Spring really toasted performance on Athlon XPs- both machines are running XP SP3, etc. That's useful to know. We'll see if this remains the same with the current build of WB Tuesday- frankly, I was rather surprised by these results. This box has always run Spring pretty smoothly, but it's obviously gotten quite a lot better over here, while if anything my home PC is slower now.

IDK, I'll run this same test on the Athlon XP box. Maybe it's the normalmap Gadget. Hard to say presently, the results left me mainly feeling confused, this machine did not have such a completely overwhelming advantage before.

[hoijui]

wow.. sounds really good, that profiling thing!
maybe put it somewhere on the wiki under profiling.

[AF]

Microsoft compiler optimizations > mingw32 optimizations

A lot fo people saw a large performance hit when we moved to mingw32, simply because corporate redmond monies get poured into optimizing compilers more os than mingw and gcc

[Argh]

That may be, AF, but I was most puzzled by the huge discrepancy.

In 0.78, when P.U.R.E. was going forth, I thought we were already at mingw32, IIRC. At any rate... between 0.77 and 0.78, there was a change made, at some point, that made a fairly big dent in performance on the Athlon XP- I noted that in passing, IIRC, but I can't recall what I was talking about at the time.

Since 0.78, performance has remained fairly stable. ROAM made absolutely zero difference, which was very disappointing- before, changing terrain detail levels were a giant part of performance tweaking, and the cost savings are pretty much zero now. It's not that ROAM sucks, it's just that until you pass about 100, there's no real difference in cost that I can see.

All that said, it's quite clear to me that on something that's more modern, like this machine, there has been a substantial improvement in overall performance, or at least the loads feel like they're being distributed more effectively. IDK how much of that is the hard-coded limit on CEG event spawns, how much of it's other things, but I'm actually pretty impressed. I just wish that it transferred downhill, which doesn't appear to be the case. I guess I need to start saving up the $300 it'll take to upgrade my mobo, etc. to a median level again, and for now assume that if my machine chokes, it's probably irrelevant unless it's a blockbuster.

Lastly... I managed to get home and get the new WB build here. The performance results when I swapped it in for the old one were... sobering, and this on an old map, using the older content. FPS drops to 54 in some parts of the WB scene. I really have a LOT of problems imagining that this machine's having twice the problems drawing that low-poly stuff with normalmaps, tbh, unless there is something seriously wrong.

The shader's longer, sure- but not THAT much longer. And yes, we're sending more textures, but again... on this machine, that's a blip that makes no difference- this machine plays Bioshock / Fallout 3 with all the bells and whistles on, and you can't tell me that they aren't sending more textures across the bus. And the much longer shader operations in POPS fly without a hitch.

So there's something really seriously goofy, and while I'm relieved to see some coorelation, it's also going to be yet-another time suck for me. Perhaps it's drawing the units entirely using ARB for reflection passes, when it should be using the Lua rendering entirely, etc., resulting in a nearly doubled load, I just don't know yet :-/

For people who want to verify my results in the name of Science, you can get the WB archive and updated materials here:

http://www.wolfegames.com/TA_Section/Tester2.zip

[Tobi]

In 0.78, when P.U.R.E. was going forth, I thought we were already at mingw32, IIRC. At any rate... between 0.77 and 0.78, there was a change made, at some point, that made a fairly big dent in performance on the Athlon XP- I noted that in passing, IIRC, but I can't recall what I was talking about at the time.

Maybe this was because we started using SSE math then. Although I'm not entirely sure it was then, we did find that SSE on AMD processors was barely faster and often even slower than the equivalent x87 math.

[Argh]

That may be. In which case, I am partially to blame, as I thought that would be a good idea at the time

[Master-Athmos]

Wasn't there this "issue" that when you compile Spring you could set a variable that it's meant for AMD-XP CPUs which made it run reasonably faster (but only for those old XP CPUs while the usual setting is fine for the newer ones)?

[zerver]

Microsoft compiler optimizations > mingw32 optimizations

The default "Release" build in MSVC also removes all asserts, while the mingw build does not define NDEBUG so the asserts remain. This can make a difference.

[hoijui]

Code: Select all

set(CMAKE_CXX_FLAGS_RELWITHDEBINFO "${CXX_FLAGS_DEBUG_ADDITIONAL} -O2 -DNDEBUG")
set(CMAKE_CXX_FLAGS_RELEASE        "-O2 -DNDEBUG")
set(CMAKE_CXX_FLAGS_PROFILE        "-O2 -pg ${DEBUG_WARNINGS} -DNDEBUG")

been in CMake for a long time (maybe from the start), so this can not be it.
i guess Master-Athmos is talking about the -march= flag. the problem with changing that is that it could cause sync incompatibility.

i have an AthlonXP myself, and i did my performance tests on windows tests for SSE sqrt on that machine. about 2 months ago, i accidentially found a performance boost possibility there, and it got quite some faster.. so even if SSE was to blame, this is most likely not the problem anymore.

[zerver]

been in CMake for a long time (maybe from the start), so this can not be it.

Ok, maybe this was some old problem with Scons because I had the asserts trigger many times for release builds.