Planned rendering engine rewrite

[SpliFF]

UPDATE: PDF Outline of proposed class structure (30KB) showing synced/unsynced boundary. Unsynced classes will be moved to a seperate rendering thread. Model loader will get a thread per import, allowing assets to load in parallel (4 CPUs = 3-4 models loading simultaneously).

IMPORTANT NOTE: For the lazy. Before you decide to skim this post and give uneducated feedback let me say this once. This is NOT a proposal to remove S3O / 3DO support from Spring. It's a proposal to change the way these formats are handled by the model loading and rendering code and/or facilitating updating these assets to more modern formats in a simple or even transparent manner (even if a mod is no longer maintained). Now read on:

I'm going to create a git branch to refactor and cleanup the model and rendering system and the older modeltype/texturetype specific pipelines which are currently making my attempts to integrate new model formats much more difficult than it should be.

Ultimately what I want is a way of loading s3o / 3do via the assimp system (giving me an aiScene for these types). To do this will require assimp plugins for these model types and/or a method of batch converting these formats to something the assimp system supports (like 3ds/obj + lua companion). There is already a lot of code to assist me there, particularly in UpSpring and the engine itself.

Despite anything said to the contrary the tests i've done exporting to 3DS via Upspring worked perfectly - with the exception of losing the hierarchy and possibly some metadata like radius, texture type and pivot point - however - since I can now write the metadata to a lua companion file these limitations can be completely overcome so the conversion is seamless and nothing is lost.

A batch converter that works on whole directories and/or archives should be a reasonably simple affair however loading these formats directly is also possible. The tradeoffs are basically:

1.) If we write our own assimp plugins, and the assimp community don't want to support s3o/3do in the official package, then we need to update these loaders ourselves each time the assimp ABI changes. It also means the assets get left in a legacy format that no 3D software handles directly (ie, without a plugin). This could impact future development of those assets.

2.) If we write a batch converter for these assets then we want to encourage mod owners to use it, otherwise it means running the converter client-side the first time a legacy mod is loaded. Writing converted models back to the mod archive wouldn't be hard but BD warns me that repacking a mod client-side, even if every client does it the same way, could cause desyncs. Writing a batch converter is MUCH easier than an assimp plugin though since it only involves a small rewrite of UpSpring to take commandline flags and read/write lua.


Either way, *IF* I can pull this off successfully it means several very important things:

1.) The amount of code required for model loading and rendering will drop to less than 1/3rd of what it is now - several hundred lines of 3DO / S3O specific code can be removed entirely.

2.) The rendering pipeline will increase in speed, many conditional if/else and switch statements can be removed from key parts of the rendering system. Conditional statements break the CPU's prediction cache. Optimisations that couldn't be done because they'd break the 3DO pipeline can be added.

3.) The amount of data stored in model and piece structures will be reduced, since a lot of struct/class members are specific to one format or simply duplicating data that's already stored in the assimp scene. Also:

4.) Several temporary and proxy objects may become unnecessary or simpler, as the data they currently store will be available via assimp scenes and read/write VBOs. This has ramifications beyond just optimising since it may make it easier for pieces and piece projectiles to be detached from their parent unit and used for rubble and COB explosions without worrying about what happens when the parent unit is freed.

5.) Texture sharing and rendering will become more efficient, since only 1 texture map / queue will be needed instead of 3 separate ones.

6.) Animation, per-model lighting, model cameras and bone support will become closer to reality, since an assimp scene exposes this data to the engine (but 3Dmodel proxies and model pieces do not).

7.) Model postprocessing becomes possible. The assimp library is capable (within limits) of optimising and repairing broken meshes and removing unnessesary and duplicate data. In practise this means the same s3o / 3do asset could render faster than it did before - even before considering the extra speed of the pipeline itself.

8.) Models can become more general purpose, instead of being so intrinsically tied to unit drawing code. This makes model loading for Lua UI, projectiles and other uses a more seamless affair.

9.) Updating the model / rendering code in the future becomes much easier. You'll generally only need to modify and test 1 codepath and one set of structures instead of 3 seperate paths.

10.) A *LOT* of GL state changes can be removed. The engine will no longer need to setup and cleanup different contexts for rendering 3DO, S3O and assimp.

11.) One of assimp's postprocessing flags will triangulate meshes containing quads and polygon faces. This removes the requirement for rendering and COB / Lua interfaces (like Shatter()) to have seperate pipelines for handling non-triangular faces.

So it's a pretty good plan and just needs time to make it feasable. It'll delay the assimp loader by a month or more but save me days wasted fixing bugs and adding features to code I'm planning to rewrite anyway. Nobody really NEEDS this loader right now so the delay isn't critical. I'd rather do this right, than do it quickly.

[Forboding Angel]

Well, for my money, hell yeah! I only have one request which is simple enough, please don't break bos/cob scripting. I'm not sure how much of a part (if any) that the renderer plays in the scripting side of things, but I would be most perturbed if I could not script in Bos anymore.

[MidKnight]

Sounds awesome! Go for it!

[SpliFF]

I have no intention of breaking or deprecating BOS. Even if the data structures change I'll update BOS / Lua Unit Scripts to handle the change. The main factor is that BOS wants access to piece names, transformations, faces and vertices (for Shatter etc). Where it gets them from is the only consideration.

[Forboding Angel]

Gotcha, makes sense. Well for what it's worth, you get a hell yeah and rock the fuck on from me. I'd really love to see this get implemented.

[Argh]

So... basically we'd put our current assets through a batch-processor, convert them to assimp-readable formats, as a final stage of assembly?

If we write our own assimp plugins, and the assimp community don't want to support s3o/3do in the official package, then we need to update these loaders ourselves each time the assimp ABI changes. It also means the assets get left in a legacy format that no 3D software handles directly (ie, without a plugin). This could impact future development of those assets.

That is potentially very scary.

With UpSpring, we don't have that problem, because transport back and forth from source to destination and back is possible, and you can continue to edit the source even after export. It'd be a major step backwards (imo, at least) if transport of an asset is a one-way ride. That would put us in the same position as a lot of game engines, where once you build the asset, you can't perform further edits unless you work on the source model.

Lose the source? Source got altered for another use, and you forgot to save a copy? Somebody GPL's the model after export, but refuses to share the model's source, and you need to make changes? You're screwed.

At the very least, we need a converter that will convert the assets back and forth that's bulletproof. I presume from what you're saying that running the process in reverse should be possible... just be aware that this is a potential showstopper right there, in terms of practical issues. I am for this change overall, but that issue is pretty much the number one reason why this is something that needs to be rigorously tested with artists so that the kinks can be worked out.

Otherwise, I have to echo Forb's "hellz yeah", this sounds incredibly awesome in terms of the implications

[SpliFF]

You misunderstand (as usual).

Once an asset (model) is converted from s3o/3do to obj+lua there is never going to be a valid reason to convert it back. Your new model (an OBJ or 3DS file) can be read by almost any modelling tool INCLUDING UpSpring. If you come up with a wacky reason you want to go back then use UpSpring (see below).

The only new thing here is the Lua metadata file which basically contains the S3O header data and hierarchies that 3DS/OBJ can't store.

If you want to edit a converted asset you do it in any 3D tool that understands OBJ or 3DS (pretty much ALL of them actually). If your hierarchies don't change then you don't have to change your Lua file either.

If you lose your new assets and only have an S3O/3DO file then simply run it back through the converter (like you do now with UpSpring).

In fact, the converter itself will probably BE UpSpring. It already does 90% of what is required. It just needs a batch mode and some Lua support for metadata (read and write). It's open source.

Don't create a fuss over an issue that isn't there.

[Argh]

So... you're talking about converting to 3DS, then using Lua to define the hierarchies and rotation points?

UpSpring would basically serve as intermediary, providing that data via an export step, and the final result would be a 3DS and a Lua file that provided the information the engine would need?

OK, sounds great... but if we import a raw 3DS into UpSpring, it will arrive sans rotational offsets, etc., so export under your proposal is still effectively a one-way trip, unless you have the source S3O or you're willing to reconstruct all of that in UpSpring again. That's not a desirable state of affairs, even if the raw mesh remains easily imported into whatever we're using that handles 3DS. Obviously, this can be cured by writing some code into UpSpring that allows us to re-import these modified 3DS files (should be easy), but it's important that it exists (and works perfectly).

And I'm really not trying to "cause a fuss", my apologies if you take it that way. This issue's rather important, is all, in terms of practical workflow.

I think you have this conception of art being a largely one-time process. It absolutely does not work that way, and we very frequently have to add things like emitter points long after we've produced the original asset, to support new gamecode or just because we've forgotten some small thing along the way. Doing it by altering Lua text files would be incredibly slow and cumbersome, and would not be welcomed by non-coders.

So please be understanding about my concerns about anything that might remotely make import / export / editing any harder than it already is, it's born from my experience working with this engine.

[Masse]

If I understood correctly this is epic! No need to convert models to S3O means it's much easier to modify models later :)

[SpliFF]

So... you're talking about converting to 3DS, then using Lua to define the hierarchies and rotation points?

No. I'm talking about using Lua to store that data! And if you're going through UpSpring you get a free GUI for the Lua as well.

UpSpring would basically serve as intermediary, providing that data via an export step, and the final result would be a 3DS and a Lua file that provided the information the engine would need?

Almost. The UpSpring process becomes a once-only affair and ONLY if your asset is still an S3O/3DO. Once it's converted you only need to repeat the process if your nodes change or you want to edit the radius visually. If you're only editing/painting/rotating or adding meshes to existing nodes your Lua will always be valid.

OK, sounds great... but if we import a raw 3DS into UpSpring, it will arrive sans rotational offsets, etc.,

No. I don't think you actually understand the difference between a "mesh" and a "node". A node stores your offsets and piece name. A node can have 0 or more meshes. A node can SHARE meshes with other nodes. Meshes are irrelevant to piece hierarchy. Most useful formats can store nodes and most editors should be able to create them.

so export under your proposal is still effectively a one-way trip, unless you have the source S3O or you're willing to reconstruct all of that in UpSpring again.

... and some Lua support for metadata (read and write).

How is the ability to read back what you just wrote and keep the metadata "a one-way trip".

I think you have this conception of art being a largely one-time process. It absolutely does not work that way,

...now. Anyway, you think wrong.

and we very frequently have to add things like emitter points long after we've produced the original asset, to support new gamecode or just because we've forgotten some small thing along the way.

Use your editor and/or UpSpring/Lua to add a node, update your CEG or BOS. SAME as now except UpSpring is OPTIONAL. If you use a format that stores node names and hierarchy then the Lua is optional too (Spring will provide reasonable defaults for radius).

Doing it by altering Lua text files would be incredibly slow and cumbersome,

Use UpSpring, re-export.

and would not be welcomed by non-coders.

Apart from the fact you can still use UpSpring what the fuck do you think BOS is? What the fuck is a unit script or FBI? How the hell do you expect to create a unit, let alone a working emitter, without touching any "code"? You want a GUI, use UpSpring, you know Lua, use notepad. Everybody is happy.

So please be understanding about my concerns about anything that might remotely make import / export / editing any harder than it already is, it's born from my experience working with this engine.

It's born from your experience working with incomplete formats, an engine that doesn't support any standard formats, and a process that REQUIRES an intermediate (and somewhat buggy) tool EVERY time you make a change. Every complaint you're making applies to the current process even more than the proposed one. How is doing the SAME thing you're doing now, with the option of skipping certain steps, "worse"?

Assimp allows formats Spring has never seen before. MD3, Collada etc can make even the Lua file optional. It's only purpose is to supplement formats that can't store metadata internally or to use when you have an asset you can't edit but you want to use it anyway (and perhaps scale / rotate / remap it).

That metadata is the whole reason S3O exists, to overcome the limitatations of 3DO, OBJ and possibly 3DS. But since NO 3D modelling software supports S3O / 3DO directly it's a hassle to use. Assimp supports more than 20 formats, and at least 5 of them are superior to S3O in almost every way.

Your concerns are baseless since you fail to read and/or understand my posts and the underlying formats. Your experiences are based entirely around the use of a tool you admit is often crude and difficult to use. I offer a method whereby you can use this tool LESS and you voice a series of unfounded concerns. I hope now those concerns are now put to rest. If not please make more of an effort to read my original and subsequent posts before voicing them again. I HATE repeating myself.

If I understood correctly this is epic! No need to convert models to S3O means it's much easier to modify models later :)

See, he got it.

[Das Bruce]

Am I the only one who would prefer to edit text files than use UpSpring?

[Forboding Angel]

Here is a valid concern. Without a gui, how would we properly define collision sphere's and unit heights? From what I can tell, upspring currently chokes miserably when trying to load a 3ds. The model comes in as a single piece and is rotated incorrectly.

I'm not saying these are reasons not to proceed, all I'm asking is how will we accurately define those things without the help of upspring (or would we be using upspring in tandem)?

Course, I can reasonably deduce that if you were going to all this trouble 3ds loading in upspring would also be fixed, making what I said largely irrelevant, correct?

[Otherside]

sounds like epic win

[hoijui]

not that i understand much, but it sounds nice!

i personally like 8, 9 and 10 the most.
as to me, all the graphical stuff of the engine code is what i am most scared of, it would be great to see some of it go. would also probably make engine code easier to understand for other (new) devs.
will this also untie rendering stuff a bit more from the rest of the engine source? (eg, relevant for cutting out parts from a headless build)

good evening and good luck!

[Umbra]

I'm not sure I understand the scope of things when you say "rendering engine."
Do you mean all the code that handles anything that draws stuff on the screen such as opengl, and anything required by such code?
Or is it just stuff such as the model formats?

[SeanHeron]

I'd fully support this (but seeing I don't have all too much artwork in s3o, I guess I would...).
I know from Bob that the step - convert from obj/3ds into s3o is a big pain in the neck to him. So to be able to miss that would be great (yes obviously there will still be a step where metadata needs to be added..). Rescaling and all kinds of stuff would be far easier than it is now.

So brilliant work SpliFF, and hope you can go ahead with this as you'd sketched it out!

@Argh - just in case it wasn't clear - (my understanding is) the only "one way" process would be converting the currently used s3o's into (what most of them probably started out as) obj or 3ds. Then that is the "native" Spring format (as SpliFF says: + the metadata .lua file). Andof course you can mess the hell around with that, no need for conversions if you make changes (or additionaly or seperately, you mess with the metadata).
Of course there'd be no necessity to convert s3os - if you have stuff knocking about at obj/3ds, you could instead take those and add the (new format) metadata (again). I think SpliFF was saying that won't be necessary with the proper batch conversion tools though (as of course it would be a good bit of work otherwise).

[FLOZi]

I believe Argh's major concern is probably what Forbs is, and indeed mine.

[jK]

I hate it to be the moaner ...

So first a rewrite of the model is necessary, I say that already since ~1year. But I don't think your motives are the right ones, so your final product won't be much better than the current one.

Be straightforward and say the truth that you just want to drop the s3o/3do loading in the engine cause you want to use assimps structs&classes directly: THIS IS FAIL. assimp is everything else than a `stable` library, its ABI can change with each revision! Also there is none advantage in doing so, it won't be faster, it won't be cleaner.
The modelparser code is already modularized and can be extended easily, it's the render code that needs to be modularized.

0.2.) Isn't this contrary to your main reason of adding assimp support? I thought it was the aim to remove any spring-specific tools to create a model.

1.)several hundred lines of 3DO / S3O specific code can be removed entirely.

2.)Optimisations that couldn't be done because they'd break the 3DO pipeline can be added.

10.) A *LOT* of GL state changes can be removed. The engine will no longer need to setup and cleanup different contexts for rendering 3DO, S3O and assimp.

3do and s3o share already 99.9% of their GL states, so this just untrue.

2.) 99.99999999% of all conditional statements are linked to IsInView, ShowHealthbar, IsLua...
So you can't optimize anything there, yeah you would even need to add more if-flags, cuz you have to dynamically switch textures bindings and other GL states for an assimp based renderer.

5.) What is a texture map/queue and why does the current one needs 3 of them?

8.) Adding a Lua interface for the current modelparser is easy, and doesn't need any changes in it.

11.) This is one reason why you can't ever transform 3do in a diff format (easily), 3do sometimes define faces with 1 or 2 vertices. You can't triangulate those! and the engine needs those to calculate the facing dir of a piece.


As already said I plan a model rendering rewrite already for a very long time myself and got a lot of ideas how it should be realized, so I might be a bit harsh. Still I think your motives are wrong. Not to forget that you didn't talked anything how your renderer should look like, you just talked about how much fail the current one is into your opinion.

[SeanHeron]

I can't answer jK's points (I can only say it sounds as if it might be a good idea for the two of you to get together on the lobby and chat things through some).

Edit: SpliFF goes on to say most of what I say in the following - just sharper. So making mine small to be in the way less. (ie - just read the post after mine).
But on Flozi's:
Ok - I can see that concern. I don't think it's fair to expect SpliFF to fix Upspring though. And from my perspective, even given the problem you're outlining (no GUI method of setting collision sphere / height, with instantly viewable outcome), I'd still suggest going forward.

Best of course would be if someone can be found to fix the import problem with Upspring. The second best solution's (I'd think) might be either that the metadata is reloadable/editable on the fly within Spring (I don't expect to see that happening - though might allow for some cool stuff from a mod/game perspective as well ).

Or, if you're saying what you're really worried about is: Having a tool that you can change the values in, and see the changes immediately (so that you can set them up proper real fast). Then perhaps this might be a workaround:
- give a damn that Upspring breaks everything;
- load up your model (I'm figuring it still displays it correctly);
- play around with collisionsphere, note down the values you've come up with in notepad/your lua-metafile (just a coordinate and radius if I'm not mistaken?);
- and then don't save the model.
Not so sure if the same would be valid for height, but I expect so.

I may of course be totally off the mark, and what you were referring to as your concern is actually that Upspring doesn't load 3ds in general - in which case it'd be unclear to me how that problem changes from now... (ie if you didn't import from 3ds so far - then why would you in the future. And if you do now, then whats different?). Apologies if I'm very befuddled here.

[SpliFF]

Here is a valid concern. Without a gui, how would we properly define collision sphere's and unit heights?

Both of these things can be guessed by the importer,however, if you have "pointy bits" or an oddly shaped object and decide the default values are wrong then there are many options:

1.) If your model was already 3DO / S3O then UpSpring will export the Lua for you. Therefore existing models require no extra work.

2.) Convert your model from another format to OBJ/3DS, load in the new UpSpring, set your offsets, rotations, radius, hierarchy, piece names, etc then export ONLY the Lua metafile (this will be an option). You now have your Lua file and it should be compatible with your original model regardless of its format (unless your conversion to OBJ/3DS really screwed up the scale). This is essential equivalent to option 1 with the extra step of making the model compatible with UpSpring.

3.) Guess. Load the model and turn on unit rendering debug in Spring. This will visually show your hitboxes and radius and you can adjust, reload, repeat until you're happy. This could take awhile but perhaps the new system could be designed to let you reload a model while you're still inside Spring and have all new units use the new version.

4.) Wait until somebody ports the assimp library into UpSpring or similar tool, then do everything like above without creating an intermediate OBJ/3DS (one of the few formats UpSpring supports).

5.) Write a blender / 3D Studio script to create the Lua metadata from named objects, essentially making UpSpring obsolete (except as a S3O/3DO converter).

6.) Like above but implement the hitbox / radius / piece editor as a spring mod or subsystem. That way you'll always be seeing EXACTLY what Spring sees and you can do it in near realtime.

From what I can tell, upspring currently chokes miserably when trying to load a 3ds. The model comes in as a single piece and is rotated incorrectly.

Both of these can be fixed. I imagine it just wasn't a high enough priority or the 3DS file is missing the node hierarchy. I think it's probably the former. If the hierarchy is missing you create it then export. The next time you import UpSpring will see the companion Lua and rebuild the hierarchy from that. The rotation issue should be an easy fix.

Course, I can reasonably deduce that if you were going to all this trouble 3ds loading in upspring would also be fixed, making what I said largely irrelevant, correct?

Correct. It would also be a good time to check it compiles and runs on Mac / Linux without issues so artists aren't forced to use Windows. Last I heard it runs ok on Wine.