View topic - Python S3O module and optimizer



All times are UTC + 1 hour


Post new topic Reply to topic  [ 49 posts ]  Go to page 1, 2, 3  Next
Author Message
PostPosted: 23 Jan 2012, 22:01 

Joined: 23 Jan 2012, 21:42
I wrote a Python S3O model parsing/serialization module, and a simple model optimizer based on it!

https://github.com/Muon/s3o-tools

The optimizer removes redundant data and reindexes the mesh. It does miracles on poorly exported models with bad index tables (looking at you, Kernel Panic!). I've tested it on KP's model set: it reduced the size of the model data by about half - from 737409 bytes to 430561 bytes. There should also be a performance boost because of the indexing but I was too lazy to do any real measurement :-) I'll probably add a tri-stripper later.


Top
 Offline Profile  
 
PostPosted: 23 Jan 2012, 23:23 
Moderator
User avatar

Joined: 26 Oct 2007, 15:21
Damn nice stuff, way cleaner and better than the one I wrote. Just so you know, upspring doesn't handle triangle stripped pieces, while the engine theoretically does (I haven't tested it)

Upspring has a command called 'optimize', I think it does the same thing as your code - it removed redundant vertices. The performance gain from this is noticeable.


Top
 Offline Profile  
 
PostPosted: 23 Jan 2012, 23:38 

Joined: 23 Jan 2012, 21:42
One of the S44 devs just ran it on the mod's models, which were already Upspring-optimized and it cut them down to size nicely. I ran it myself too, and the reduction is significant (http://pastebin.com/GVyQAq3C). Hmm, Upspring doesn't sound nice. And that fact I couldn't compile it didn't help its case.


Top
 Offline Profile  
 
PostPosted: 23 Jan 2012, 23:44 
Moderator
User avatar

Joined: 29 Apr 2005, 00:14
Location: #moddev - join it!
* Can't guarantee that the S44 models have had Upspring's optimize run on them, I was commenting on the software's general lack of redeeming qualities. :P

FWIW Behe, your Python s3o parser is much nicer than mine. :wink:


Next up Muon, write an ASSIMP-loading model assembling tool to replace Upspring that is not Blender.


Last edited by FLOZi on 23 Jan 2012, 23:49, edited 2 times in total.

Top
 Offline Profile  
 
PostPosted: 23 Jan 2012, 23:46 
Spring Developer
User avatar

Joined: 28 Jun 2007, 06:30
:oops:
I think I know why you reduced even already optimized meshes:
You forgot the normals & texcoords! Vertices can have the same vertexcoords but different normals & texcoords, if so they need to be added separately to the vertexbuffer.


Top
 Online Profile  
 
PostPosted: 24 Jan 2012, 10:20 
Moderator
User avatar

Joined: 26 Oct 2007, 15:21
I love upspring, its one of the best tools around - sure it has UI oddities, but what software doesnt? Besides that, its reliable and fast to work in.


Top
 Offline Profile  
 
PostPosted: 24 Jan 2012, 11:37 
Moderator
User avatar

Joined: 29 Apr 2005, 00:14
Location: #moddev - join it!
In what quantum reality is Upspring reliable? :shock:


Top
 Offline Profile  
 
PostPosted: 24 Jan 2012, 11:44 

Joined: 23 Jan 2012, 21:42
jK wrote:
:oops:
I think I know why you reduced even already optimized meshes:
You forgot the normals & texcoords! Vertices can have the same vertexcoords but different normals & texcoords, if so they need to be added separately to the vertexbuffer.

No, I do full vertex comparisons - position, normals, and texcoords. It's the most conservative optimizer on Earth :P

Also, updated: forgot to rename optimize_piece() to recursively_optimize_pieces() when I cleaned up the code so the optimizer only went 2 levels deep (root and immediate children).


Last edited by Muon on 24 Jan 2012, 14:27, edited 3 times in total.

Top
 Offline Profile  
 
PostPosted: 24 Jan 2012, 11:51 
Moderator
User avatar

Joined: 26 Oct 2007, 15:21
Ive put over 100 units ingame with upspring, didnt crash often, its use was deterministic. Sure I've run into quirks, but they were extremely minor.


Top
 Offline Profile  
 
PostPosted: 24 Jan 2012, 12:49 

Joined: 23 Jan 2012, 21:42
Okay! So, with the latest fix, the optimizer reduces Market Garden's objects3d from 57540187 bytes to 36098843 bytes - a whopping 20.4 MB reduction. And everything works!


Top
 Offline Profile  
 
PostPosted: 24 Jan 2012, 13:19 
Moderator
User avatar

Joined: 29 Apr 2005, 00:14
Location: #moddev - join it!
Beherith wrote:
Ive put over 100 units ingame with upspring, didnt crash often, its use was deterministic. Sure I've run into quirks, but they were extremely minor.


Maybe i was spoilt by the luxury of 3do Builder 2.02 (not 2.1, that was pants too)

Muon; Perhaps a message at the end of processing several files with the total change in Mb?


Last edited by FLOZi on 24 Jan 2012, 13:55, edited 1 time in total.

Top
 Offline Profile  
 
PostPosted: 24 Jan 2012, 13:53 

Joined: 23 Jan 2012, 21:42
It is DONE!

Also, fun fact: I just noticed my remapping algorithm also improves the spatial locality of the vertex data in exactly the same way as NvTriStrip's RemapIndices() function.


Top
 Offline Profile  
 
PostPosted: 24 Jan 2012, 14:24 
Moderator
User avatar

Joined: 26 Oct 2007, 15:21
Wonderful! Ill test how this works with BAR. Thanks for your effort!

Did you do a performance test? Try it with /cheat then /give 500 gertiger and compare FPS.


Top
 Offline Profile  
 
PostPosted: 24 Jan 2012, 14:29 

Joined: 23 Jan 2012, 21:42
No, not a really real one - don't have time today. But /cheat and /give all was so much smoother after I ran my optimizer on everything. It didn't stutter at all at normal zoom! (As opposed to much stuttering pre-optimization.)

EDIT: Went and did it anyway. On 500 tanks it went from 15 to 16 FPS (Linux, Sandy Bridge graphics, lowest settings) :| Maybe it's more important with more graphical bling? Maybe it just removed a ton of dead vertex data :?


Top
 Offline Profile  
 
PostPosted: 24 Jan 2012, 15:01 
Moderator
User avatar

Joined: 26 Oct 2007, 15:21
Ill try it on my trusty 8800GT. Maybe NVTriStrip was made for nvidia cache architecture?


Top
 Offline Profile  
 
PostPosted: 24 Jan 2012, 15:07 
Moderator
User avatar

Joined: 29 Apr 2005, 00:14
Location: #moddev - join it!
Stuttering on /give all has other contributing factors like texture loading


Top
 Offline Profile  
 
PostPosted: 24 Jan 2012, 15:25 

Joined: 23 Jan 2012, 21:42
Beherith wrote:
Ill try it on my trusty 8800GT. Maybe NVTriStrip was made for nvidia cache architecture?

The remapping algorithm in there is not nVidia-specific. I didn't even know my remapping algorithm was the same as NVTriStrip's until today!

FLOZi wrote:
Stuttering on /give all has other contributing factors like texture loading

I measured the framerate after loading completely completed. (Went from 12 to about 30 FPS.)


Top
 Offline Profile  
 
PostPosted: 24 Jan 2012, 15:51 
Moderator
User avatar

Joined: 29 Apr 2005, 00:14
Location: #moddev - join it!
Regarding tri-stripping, you've probably already seen it, but:

http://users.telenet.be/tfautre/softdev ... istrip.htm


Top
 Offline Profile  
 
PostPosted: 24 Jan 2012, 16:07 
Moderator
User avatar

Joined: 26 Oct 2007, 15:21
Just a side note on triangle stripping: 99% of spring's s3o's are using triangle only primitives. This is because they dont use quads if there is a single non quad present, and because upspring doesnt make or support triangle strips.

This can result in troubles, namely if one optimizes to triangle strips, then later has to edit the model - he must reconvert to an un-triangle-stripped version to edit it in upspring (for such things as changing radius, pieces, empty pieces, changing texture; so this can happen a lot in a mod under development)


Top
 Offline Profile  
 
PostPosted: 24 Jan 2012, 16:45 

Joined: 23 Jan 2012, 21:42
Well, as I said, I don't do triangle stripping yet. The models will be 100% Upspring compatible after optimization. I looked at SS3OPiece::DrawForList() and seems that it's only able to restart triangle strips if the NV_primitive_restart extension is present, so I don't think that Spring would properly render strips without explicit degenerate triangles at all!


Top
 Offline Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 49 posts ]  Go to page 1, 2, 3  Next

All times are UTC + 1 hour


Who is online

Users browsing this forum: No registered users and 2 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB® Forum Software © phpBB Group

Site layout created by Roflcopter et al.