Python S3O module and optimizer
Moderator: Moderators
Python S3O module and optimizer
I wrote a Python S3O model parsing/serialization module, and a simple model optimizer based on it!
https://github.com/Muon/s3o-tools
The optimizer removes redundant data and reindexes the mesh. It does miracles on poorly exported models with bad index tables (looking at you, Kernel Panic!). I've tested it on KP's model set: it reduced the size of the model data by about half - from 737409 bytes to 430561 bytes. There should also be a performance boost because of the indexing but I was too lazy to do any real measurement I'll probably add a tri-stripper later.
https://github.com/Muon/s3o-tools
The optimizer removes redundant data and reindexes the mesh. It does miracles on poorly exported models with bad index tables (looking at you, Kernel Panic!). I've tested it on KP's model set: it reduced the size of the model data by about half - from 737409 bytes to 430561 bytes. There should also be a performance boost because of the indexing but I was too lazy to do any real measurement I'll probably add a tri-stripper later.
Re: Python S3O module and optimizer
Damn nice stuff, way cleaner and better than the one I wrote. Just so you know, upspring doesn't handle triangle stripped pieces, while the engine theoretically does (I haven't tested it)
Upspring has a command called 'optimize', I think it does the same thing as your code - it removed redundant vertices. The performance gain from this is noticeable.
Upspring has a command called 'optimize', I think it does the same thing as your code - it removed redundant vertices. The performance gain from this is noticeable.
Re: Python S3O module and optimizer
One of the S44 devs just ran it on the mod's models, which were already Upspring-optimized and it cut them down to size nicely. I ran it myself too, and the reduction is significant (http://pastebin.com/GVyQAq3C). Hmm, Upspring doesn't sound nice. And that fact I couldn't compile it didn't help its case.
Re: Python S3O module and optimizer
* Can't guarantee that the S44 models have had Upspring's optimize run on them, I was commenting on the software's general lack of redeeming qualities.
FWIW Behe, your Python s3o parser is much nicer than mine.
Next up Muon, write an ASSIMP-loading model assembling tool to replace Upspring that is not Blender.
FWIW Behe, your Python s3o parser is much nicer than mine.
Next up Muon, write an ASSIMP-loading model assembling tool to replace Upspring that is not Blender.
Last edited by FLOZi on 23 Jan 2012, 23:49, edited 2 times in total.
Re: Python S3O module and optimizer
I think I know why you reduced even already optimized meshes:
You forgot the normals & texcoords! Vertices can have the same vertexcoords but different normals & texcoords, if so they need to be added separately to the vertexbuffer.
Re: Python S3O module and optimizer
I love upspring, its one of the best tools around - sure it has UI oddities, but what software doesnt? Besides that, its reliable and fast to work in.
Re: Python S3O module and optimizer
In what quantum reality is Upspring reliable?
Re: Python S3O module and optimizer
No, I do full vertex comparisons - position, normals, and texcoords. It's the most conservative optimizer on EarthjK wrote:
I think I know why you reduced even already optimized meshes:
You forgot the normals & texcoords! Vertices can have the same vertexcoords but different normals & texcoords, if so they need to be added separately to the vertexbuffer.
Also, updated: forgot to rename optimize_piece() to recursively_optimize_pieces() when I cleaned up the code so the optimizer only went 2 levels deep (root and immediate children).
Last edited by Muon on 24 Jan 2012, 14:27, edited 3 times in total.
Re: Python S3O module and optimizer
Ive put over 100 units ingame with upspring, didnt crash often, its use was deterministic. Sure I've run into quirks, but they were extremely minor.
Re: Python S3O module and optimizer
Okay! So, with the latest fix, the optimizer reduces Market Garden's objects3d from 57540187 bytes to 36098843 bytes - a whopping 20.4 MB reduction. And everything works!
Re: Python S3O module and optimizer
Maybe i was spoilt by the luxury of 3do Builder 2.02 (not 2.1, that was pants too)Beherith wrote:Ive put over 100 units ingame with upspring, didnt crash often, its use was deterministic. Sure I've run into quirks, but they were extremely minor.
Muon; Perhaps a message at the end of processing several files with the total change in Mb?
Last edited by FLOZi on 24 Jan 2012, 13:55, edited 1 time in total.
Re: Python S3O module and optimizer
It is DONE!
Also, fun fact: I just noticed my remapping algorithm also improves the spatial locality of the vertex data in exactly the same way as NvTriStrip's RemapIndices() function.
Also, fun fact: I just noticed my remapping algorithm also improves the spatial locality of the vertex data in exactly the same way as NvTriStrip's RemapIndices() function.
Re: Python S3O module and optimizer
Wonderful! Ill test how this works with BAR. Thanks for your effort!
Did you do a performance test? Try it with /cheat then /give 500 gertiger and compare FPS.
Did you do a performance test? Try it with /cheat then /give 500 gertiger and compare FPS.
Re: Python S3O module and optimizer
No, not a really real one - don't have time today. But /cheat and /give all was so much smoother after I ran my optimizer on everything. It didn't stutter at all at normal zoom! (As opposed to much stuttering pre-optimization.)
EDIT: Went and did it anyway. On 500 tanks it went from 15 to 16 FPS (Linux, Sandy Bridge graphics, lowest settings) Maybe it's more important with more graphical bling? Maybe it just removed a ton of dead vertex data
EDIT: Went and did it anyway. On 500 tanks it went from 15 to 16 FPS (Linux, Sandy Bridge graphics, lowest settings) Maybe it's more important with more graphical bling? Maybe it just removed a ton of dead vertex data
Re: Python S3O module and optimizer
Ill try it on my trusty 8800GT. Maybe NVTriStrip was made for nvidia cache architecture?
Re: Python S3O module and optimizer
Stuttering on /give all has other contributing factors like texture loading
Re: Python S3O module and optimizer
The remapping algorithm in there is not nVidia-specific. I didn't even know my remapping algorithm was the same as NVTriStrip's until today!Beherith wrote:Ill try it on my trusty 8800GT. Maybe NVTriStrip was made for nvidia cache architecture?
I measured the framerate after loading completely completed. (Went from 12 to about 30 FPS.)FLOZi wrote:Stuttering on /give all has other contributing factors like texture loading
Re: Python S3O module and optimizer
Regarding tri-stripping, you've probably already seen it, but:
http://users.telenet.be/tfautre/softdev ... istrip.htm
http://users.telenet.be/tfautre/softdev ... istrip.htm
Re: Python S3O module and optimizer
Just a side note on triangle stripping: 99% of spring's s3o's are using triangle only primitives. This is because they dont use quads if there is a single non quad present, and because upspring doesnt make or support triangle strips.
This can result in troubles, namely if one optimizes to triangle strips, then later has to edit the model - he must reconvert to an un-triangle-stripped version to edit it in upspring (for such things as changing radius, pieces, empty pieces, changing texture; so this can happen a lot in a mod under development)
This can result in troubles, namely if one optimizes to triangle strips, then later has to edit the model - he must reconvert to an un-triangle-stripped version to edit it in upspring (for such things as changing radius, pieces, empty pieces, changing texture; so this can happen a lot in a mod under development)
Re: Python S3O module and optimizer
Well, as I said, I don't do triangle stripping yet. The models will be 100% Upspring compatible after optimization. I looked at SS3OPiece::DrawForList() and seems that it's only able to restart triangle strips if the NV_primitive_restart extension is present, so I don't think that Spring would properly render strips without explicit degenerate triangles at all!