Spring eating my CFLAGS

[HAARP]

Simple question:

Code: Select all

WARNING: attempt to use environment CFLAGS has been ignored.
WARNING: attempt to use environment CXXFLAGS has been ignored.

Is this necessary? I'd like to use these flags for performance reasons

[Tobi]

I don't mind if you hack the warning out and make it use CFLAGS, but don't complain if it happens to desync then.

Leaving out or putting in particular flags WILL make you desync, which is why I disabled using CFLAGS altogether.

[HAARP]

That's what I wanted to hear, thanks.

Which are these particular flags that are known to desync?
I would've used these:
CFLAGS="-march=athlon-xp -O2 -pipe -fno-ident -fomit-frame-pointer -mfpmath=sse -ffast-math"
CXXFLAGS="${CFLAGS} -fvisibility-inlines-hidden"

[imbaczek]

probably would desync really quick... but you can try anyway

[Tobi]

That's what I wanted to hear, thanks.

Which are these particular flags that are known to desync?
I would've used these:
CFLAGS="-march=athlon-xp -O2 -pipe -fno-ident -fomit-frame-pointer -mfpmath=sse -ffast-math"
CXXFLAGS="${CFLAGS} -fvisibility-inlines-hidden"

The bold ones will desync you, need the ones from SConstruct (-mfpmath=x87 + some more). The others are safe, I think, though I don't recall what -fno-ident does atm.

[HAARP]

That's what I wanted to hear, thanks.

Which are these particular flags that are known to desync?
I would've used these:
CFLAGS="-march=athlon-xp -O2 -pipe -fno-ident -fomit-frame-pointer -mfpmath=sse -ffast-math"
CXXFLAGS="${CFLAGS} -fvisibility-inlines-hidden"

The bold ones will desync you, need the ones from SConstruct (-mfpmath=x87 + some more). The others are safe, I think, though I don't recall what -fno-ident does atm.

I am going to generally get rid of -mfpmath=sse, because I since heard this breaks stuff and does NOT improve performance on my march.
The only mfpmaths that are possible are: 387 | sse | 387,sse whereas the combination of both will break stuff badly.
-ffast-math improves performance by up to 20% but could also make apps behave strange at runtime. I used it so far without problems in other apps tho. I will try to see if it desyncs and report back.
-fno-ident basically strips useless information strings GCC tends to put into binaries, thus saving a few bytes. Shouldn't hurt

Thank you Tobi.

[Tobi]

It was -mfpmath=387 then

[HAARP]

Ok. First off, I made a small benchmark to see how many FPS I get under normal conditions. Made same units do some pathfinding, particles on the screen, etc. My FPS drop to 17-18. I will recreate these conditions once I have an optimized binary.

In detect.py I saw some functions to detect the build environment and adjust -march, however this doesn't seem to get used. I couldn't find any -march in the output (where it says

Code: Select all

g++ -o blah.o -c -fsingle-precision-constant -frounding-math -fsignaling-nans -mieee-fp....

) so I added it manually

I also saw in the build script that quite some work goes into selecting proper CFLAGS before compilation. Instead of just discarding them, I keep those flags and just add those I'd like to use, namely -march=athlon-xp and -fvisibility-inlines-hidden.

-ffast-math fails miserably even at compile-time with this:

Code: Select all

g++ -o build/rts/System/TdfParser.o -c -fsingle-precision-constant -frounding-math -fsignaling-nans -mieee-fp -O2 -pipe -fno-strict-aliasing -mfpmath=387 -pthread -ffast-math -fno-ident -march=athlon-xp -fvisibility-inlines-hidden -fvisibility=hidden -D_SZ_ONE_DIRECTORY -D_GNU_SOURCE=1 -D_REENTRANT -DSTREFLOP_X87 -DSYNCCHECK -DNO_AVI -DDIRECT_CONTROL_ALLOWED -Irts -Irts/System -Irts/lib/luabind -Irts/lib/lua/include -I/usr/include -I/usr/include/GL -I/usr/include/freetype2 -I/usr/include/SDL -I/usr/include/ogg -I/usr/include/vorbis -I/usr/include/python2.5 -I/usr/include/python2.4 rts/System/TdfParser.cpp
/usr/include/bits/mathinline.h: In function 'double asinh(double)':
/usr/include/bits/mathinline.h:608: error: call of overloaded 'log1pl(long double)' is ambiguous
/usr/include/bits/mathcalls.h:132: note: candidates are: long double log1pl(long double)
rts/lib/streflop/SMath.h:672: note:                 streflop::Extended streflop::log1pl(streflop::Extended)
/usr/include/bits/mathinline.h: In function 'float asinhf(float)':
/usr/include/bits/mathinline.h:608: error: call of overloaded 'log1pl(long double)' is ambiguous
/usr/include/bits/mathcalls.h:132: note: candidates are: long double log1pl(long double)
rts/lib/streflop/SMath.h:672: note:                 streflop::Extended streflop::log1pl(streflop::Extended)
/usr/include/bits/mathinline.h: In function 'long double asinhl(long double)':
/usr/include/bits/mathinline.h:608: error: call of overloaded 'log1pl(long double)' is ambiguous
/usr/include/bits/mathcalls.h:132: note: candidates are: long double log1pl(long double)
rts/lib/streflop/SMath.h:672: note:                 streflop::Extended streflop::log1pl(streflop::Extended)
/usr/include/bits/mathinline.h: In function 'double acosh(double)':
/usr/include/bits/mathinline.h:613: error: call of overloaded 'logl(long double)' is ambiguous
/usr/include/bits/mathcalls.h:110: note: candidates are: long double logl(long double)
rts/lib/streflop/SMath.h:627: note:                 streflop::Extended streflop::logl(streflop::Extended)
/usr/include/bits/mathinline.h: In function 'float acoshf(float)':
/usr/include/bits/mathinline.h:613: error: call of overloaded 'logl(long double)' is ambiguous
/usr/include/bits/mathcalls.h:110: note: candidates are: long double logl(long double)
rts/lib/streflop/SMath.h:627: note:                 streflop::Extended streflop::logl(streflop::Extended)
/usr/include/bits/mathinline.h: In function 'long double acoshl(long double)':
/usr/include/bits/mathinline.h:613: error: call of overloaded 'logl(long double)' is ambiguous
/usr/include/bits/mathcalls.h:110: note: candidates are: long double logl(long double)
rts/lib/streflop/SMath.h:627: note:                 streflop::Extended streflop::logl(streflop::Extended)
/usr/include/bits/mathinline.h: In function 'double atanh(double)':
/usr/include/bits/mathinline.h:616: error: call of overloaded 'log1pl(long double)' is ambiguous
/usr/include/bits/mathcalls.h:132: note: candidates are: long double log1pl(long double)
rts/lib/streflop/SMath.h:672: note:                 streflop::Extended streflop::log1pl(streflop::Extended)
/usr/include/bits/mathinline.h: In function 'float atanhf(float)':
/usr/include/bits/mathinline.h:616: error: call of overloaded 'log1pl(long double)' is ambiguous
/usr/include/bits/mathcalls.h:132: note: candidates are: long double log1pl(long double)
rts/lib/streflop/SMath.h:672: note:                 streflop::Extended streflop::log1pl(streflop::Extended)
/usr/include/bits/mathinline.h: In function 'long double atanhl(long double)':
/usr/include/bits/mathinline.h:616: error: call of overloaded 'log1pl(long double)' is ambiguous
/usr/include/bits/mathcalls.h:132: note: candidates are: long double log1pl(long double)
rts/lib/streflop/SMath.h:672: note:                 streflop::Extended streflop::log1pl(streflop::Extended)
/usr/include/bits/mathinline.h: In function 'double __acosh1p(double)':
/usr/include/bits/mathinline.h:741: error: call of overloaded 'log1pl(long double)' is ambiguous
/usr/include/bits/mathcalls.h:132: note: candidates are: long double log1pl(long double)
rts/lib/streflop/SMath.h:672: note:                 streflop::Extended streflop::log1pl(streflop::Extended)
/usr/include/bits/mathinline.h: In function 'float __acosh1pf(float)':
/usr/include/bits/mathinline.h:741: error: call of overloaded 'log1pl(long double)' is ambiguous
/usr/include/bits/mathcalls.h:132: note: candidates are: long double log1pl(long double)
rts/lib/streflop/SMath.h:672: note:                 streflop::Extended streflop::log1pl(streflop::Extended)
/usr/include/bits/mathinline.h: In function 'long double __acosh1pl(long double)':
/usr/include/bits/mathinline.h:741: error: call of overloaded 'log1pl(long double)' is ambiguous
/usr/include/bits/mathcalls.h:132: note: candidates are: long double log1pl(long double)
rts/lib/streflop/SMath.h:672: note:                 streflop::Extended streflop::log1pl(streflop::Extended)
scons: *** [build/rts/System/TdfParser.o] Error 1
scons: building terminated because of errors.

So that is out of business.

I'm now in the process of compiling. I will benchmark once I have binaries and then proceed to test if there are sync issues.

Tobi, do you have any idea if it's possible to get it to compile with -ffast-math? I don't really know enough C / C++ to fix this on my own. Also, why does the -march optimization not get used? Sync issues too?

edit: Just by optimizing to my -march, FPS got boosted from 17-18 to 19-21. I will try to see if there are sync issues now
edit2: In an online game, I got

Code: Select all

No response from 1N4148 for frame 21017 etc.

Interestingly, a chat message still got through and my ping was at a constant 150, so says the host. Can this be considered sync problems?

[Tobi]

That's just a sign of a little lag peak, nothing serious. When you get "Sync error blahblah" you know it didn't work

[HAARP]

Well, in the end I got booted (timeout) :/
edit: False alarm, it even does that when I use vanilla CFLAGS. Something's borked here. Maybe my or the host's connections.
edit2: Just spectated a game without issues. I will try to get a new build with -march optimization and test again

question: Linker flags (LDFLAGS="-Wl,-O1,--as-needed") do not get used at all. Any way to get them in there?
edit3: Linking is not done says gcc. Huh? Heh, whatever.

[HAARP]

Conclusion:

So far, -march=athlon-xp -fvisibility-inlines-hidden do not appear to pose any sync problems, but manage to boost my FPS by 2-3. I think it's worth giving a shot, considering you don't have to pay anything for it but CPU time for recompiling. Of course, this depends on your actual march

-ffast-math should be able to get an additional 2-5 FPS or so if I could get it to compile. It is known to break code at compile-aswell as runtime, but that's a risk I'm willing to take (test)
Tobi, any idea how I can make that work? (see paste above for error) /usr/include/bits/mathinline.h contains a few lines regarding fast-math, but me as a non-coder can't work anything out.
Alternatively, what would I need to change in the Sconstruct to compile offending files (like TdfParser.o here) without -ffast-math, but the rest with?
If i get that cflag to compile, I will do extensive tests to see if any sync issues come up.

-mfpmath=sse does, due to limitations in the glibc headers even make binaries slower and can lead to code breakage. Or so I heard. I did not test this, and neither am I going to.

[Linus]

Concerning the -mfpmath issue: I compiled 75b2 with sse (worked, no change in performance) and then with sse,387 although the gcc warned me about it and got 1 or 2 frames, but I'm not entirely sure about it, since it didn't really benchmark it at that time. Thing is, it compiles without errors and doesn't crash and it never desynced, so I assume it's safe. I don't know about the glibc; I compiled it with -march=athlon-xp as well, that might be the reason it's stable, but I doubt it.

Now 76b1 runs with sse,387 very well for me (no de-syncing in 10 games), so I didn't test it against other -mfpmath values => no benchmark, but the sse,387 often gave me a bit of performance boost and never any problems. sse will make your code larger, thus maybe slower, sse,387 might not. I say use at own risk, but don't worry to break things.

-ffast-math won't work for me either, but the default config I see in the intopts.py has some of the flags that -ffast-math sets just the other way round (frounding-math, -fsignaling-nans), that's why I guess somebody has put some thought in it. And what I read in the gcc manual suggests I'm really breaking IEEE arithmetic here, so I don't doubt it causes de-syncs.

By the way, on my athlon-xp -O3 optimization proved to be a bit faster than -O2 (I know that's not always the case, but with me here that's the way it worked).

[HAARP]

Ok, I just recompiled with -O3 and -mfpmath=sse,387
I managed to get another performance increase by an additional 2 FPS, tho I haven't tested for sync issues yet.
Thanks for the link Linus. I was hoping to find something like that, but never did :)

btw, is there a better way to test the performance than trying to recreate the same conditions each time and looking at the framerate?

edit: I'm getting sync errors. Not a good idea to use mfpmath.

[imbaczek]

not really. you could try to time replays at very high speed (such that requested speed exceeds what your cpu can do in realtime.)

[Tobi]

I used fast replay of replays combined with --quit commandline option sometime, and a modification to the source to output time profiler statistics at quit time to the console. This way I always had total time spent in e.g. simulation for a fixed game duration (relative to the other stuff, because --quit is wallclocktime based, not gametime).

[Linus]

Strange that sse,387 it didn't work for you, haarp ... but I found something in the gentoo wiki concerning Athlon processors, I quote:

As an interesting aside, AMD recommends using function inlining with their processors (Athlon and up).

Function inlining is switched on with -O3, but not -O2 optimization (you can do it manually with -finline-functions), so if you have an athlon(-xp), it's probably worth the additional compiling time (and the increased size).

Anyways, I'm pretty sure that there are no other (safe) optimizations other than the ones mentioned here.