Dev meeting minutes 2010-12-20

[knorke]

Knorke you don't have to manually install anything, you are using zkl, just join da game.

zkl can only hosts games that are on its list.
i want to test if/how my mod or other mods crash on master version.

[hoijui]

i approached it today, using the boost framework. I wanted to write a test for a simple utility function. i have that test running now, but i had to use a lot of very ugly tricks -> it is not committable. I also got some help from guys in #boost.

The main problem we are facing:
Lets assume we have 3 test classes, testing one class each. To run the tests, the test classes have to be compiled together with the stuff they should test. There are several ways to do that:

1. Compile each test class into an executable with just the stuff it needs
2. Compile all test and engine sources into one executable
3. Compile the engine (minus the main() function) into a static lib, and use that static lib to create spring.exe and together with test sources in tests.exe
4. same like the above, but using a shared/dynamic lib, instead of static. this requires all symbols being exported though, which is not possible with VS, and ugly in other cases

1. would mean, including half the engine with every test class, which means each test executable is ~10MB, not to speak of the time it takes to compile that, and the maintenance of the compile/CMake script for it.
2. As i think, a better solution, even though it still means that for testing, one has to practically compile the engine two times plus test sources.
3. Like 2. but without the double compile time for the engine sources. Probably the best solution.
4. because of the reliance on exported symbols, not practical.

right now, i am using 1 plus two sort of Null-impls, to drastically cut down dependencies. not acceptable though.
If there are better ideas, please tell me.

CTest
A sister project of CMake. It allows to integrate tests into the build process . but i still do not really understand how it would work out. Am also not sure how much sense it makes to use it to run boost tests.

[SpliFF]

Perhaps a better way of doing automated testing would be to prepare a mod for that purpose. The mod has a Lua script that runs through basic functionality like creating units with a range of FBI properties, model types, sound files, cegs, etc and giving them orders in a preset fashion. The script can then check that the orders were carried out (ie, pathfinder was successful through a maze). The script could also test a range of Lua commands like sendToUnsynced, GL calls, etc.

If the game crashes at any point it should be easy to look at the log and see what the last attempted action was.

This would also warn us when "mod breaking" changes have been introduced.

I don't mind coding something like this if the desire to use it is there.

[aegis]

I have a cluster of four quad core boxes we could use to run automated tests for crashes

[Licho]

Knorke you don't have to manually install anything, you are using zkl, just join da game.

zkl can only hosts games that are on its list.
i want to test if/how my mod or other mods crash on master version.

What why do you claim X is impossible and asking how to do X at the same time.

ZKL can host any mod. Click advanced.

And even if it was impossible in ZKL my post explicitly said how to make springie host different mods..

[knorke]

did i miss a post? i only saw

just join da game

why do you claim X is impossible and asking how to do X at the same time.
ZKL can host any mod. Click advanced.

Oh thats new then. Please tell me how I can use it to quickly test fuckingcrashmod.sdd, a mod only existing on my harddrive. I tried copying the filename and it only opens a "dummy" battle.

[hoijui]

Spliff, that is not a better way of testing, it is simply different, and even there it would not hurt if it could be integrated into some cross-platform test suite with graphical outcome display and stats and what not.
Unit tests are mainly a regression prevention mechanism. basically testing every way a function/class may be used on different systems (if possible), which can never be done with a test like you describe it. Unit tests will also usually be less complex then your scenario, faster to execute, and tell you quite exactly where a bug was introduced, and fixing it should be a simple thing.
as said, that does not make your idea less useful; they can coexist.

[Licho]

did i miss a post? i only saw

just join da game

why do you claim X is impossible and asking how to do X at the same time.
ZKL can host any mod. Click advanced.

Oh thats new then. Please tell me how I can use it to quickly test fuckingcrashmod.sdd, a mod only existing on my harddrive. I tried copying the filename and it only opens a "dummy" battle.

Oh for sdd you dont need lobby at all. Just put the sdd to mods and start spring of given version to test it..

[hoijui]

i wanted to do the move of some static libs to shared ones (md5, 7z, minizip), but.. they don't seem to be available as such (Gentoo here). Guess that is why we have them as static libs. I could still compile them as shared libs and link to them, but there would be no real benefit there.

[SirMaverick]

1. would mean, including half the engine with every test class, which means each test executable is ~10MB, not to speak of the time it takes to compile that, and the maintenance of the compile/CMake script for it.

Only because spring code isn't structured well (everything includes everything), or?

[hoijui]

that is true for some stuff (for example, file-system abstraction). but even with perfectly structured code, it would not work out in practise. in the long run. some dependencies just have to stay.. for example, if you would want to unit-test simulation related stuff, you will likely require all the simulation in most cases. maybe you could somehow exclude most of it with certain tricks, but that would in the end just mean more maintenance work for tests and more testing code; also not worth it.

[knorke]

Oh for sdd you dont need lobby at all. Just put the sdd to mods and start spring of given version to test it..

And how would I install/dl that "spring of given version"?
See my original post:

I installed a master version as "portable" but it still changed settings in my "play online"-install.
That is not motivating to try test versions.

No idea what ZKL has to with this?!

[Licho]

It installs such spring automatically if you join game which needs that engine version.

(Marked in title [engineXXXXXX]

It's meant for big MP testing of course. For some local tests you don't need any lobby at all.