Planned rendering engine rewrite

[bobthedinosaur]

Accurate batch conversion! rrr for S3o's I mean. I could care less about nostalgia of crappy formats, as long as efficiency and some easy mod tools for converting and making new models. I'm all for it!

[jK]

For better or worse the time has come for S3O and 3DO to die.

S3o can die if you give me a tool that can convert all my s3o files correctly to whatever the new format is. I am A-OK with that as long as there is something to do the conversion. Otherwise I will have to rebuild 100s of models and that is asking a lot.

also will there be an upspring like too or are you going to further complicate the workflow?

don't forget all the maps which would need to be republished ...

[Argh]

...not to mention old games without maintainers that would suddenly not work.

You guys all sure that fixing things that aren't all that broke instead of just moving on to MD5 for future content is a good idea?

[jK]

...not to mention old games without maintainers that would suddenly not work.

You guys all sure that fixing things that aren't all that broke instead of just moving on to MD5 for future content is a good idea?

esp. when it is such trivial

[Pxtl]

Wait... back up... this project is going to break S3O/3DO support? As in, when you're talking about cutting out s3o code, you're not talking about refactoring that into the Assimp system, but gutting it altogether?

Yeah, unless you provide a 1-step batch-operable converter that is 100% accurate (not a gui app like upspring, and without bugs on output), that's kind of a dead end.

[Argh]

Wait... back up... this project is going to break S3O/3DO support? As in, when you're talking about cutting out s3o code, you're not talking about refactoring that into the Assimp system, but gutting it altogether

Um... yeah.



Let's say that all of this speeds up the fixed-function stuff by 100%. Net effect on framerate? Damn near zero. Spring is not significantly CPU-bound, in terms of drawing Units, except for Lua OpenGL stuff that's poorly written.

IDK, maybe this will make shaders run twice as fast. Or cut the geometry load in half. Or remove the need for depth-testing. Oops, I forgot, I don't know anything and I should not talk to my betters about rendering, because I obviously coded P.O.P.S. by reading the code off of a cardboard box at Code R' Us.

[zwzsg]

Old games without maintainer are probably already half-broken.



But breaking every map that use custom features is kinda .

IMPORTANT NOTE: For the lazy. Before you decide to skim this post and give uneducated feedback let me say this once. This is NOT a proposal to remove S3O / 3DO support from Spring.

Pfew! Thanks for the disclaimer, you would have me worried otherwise.

[SpliFF]

Uh... it would be bad to kill s3o, virtually all our free assets are in s3o and porting them en masse is not trivial. I suppose somebody will say similar of 3do, but I really don't care about all those models.

So, either s3o needs to be supported, or a quick flawless batch converter needs to be included. I understand that it may not appear, from a coding standpoint, really that important but from a content development standpoint s3o is crucial.

I agree. Totally. 100% for both. I'm not removing anything without providing a solution that converts a whole mod, not just a single asset, ACCURATELY, in 10 minutes or less and the resulting workflow consistently requires LESS effort than now.

Otherwise, what would be the point? We'd be pretty much in exactly the same situation as now. The only reason this debate is going on so long is that you're all so jaded from 10 years of fighting with your own models and tools.

So to end this nonsense once and for all:

Can I go from S3O to 3DS+Lua? Yes, automatically.
Can I go from 3DO to 3DS+Lua? Yes, automatically.
Can I go from 3DS or OBJ to 3DS+Lua? Yes, the same way you do now.
Can I go from 3DS+Lua back to S3O? Yes, automatically.
Can I go from 3DS+Lua back to 3DO? Maybe.
Can I do a whole mod at once? Yes, automatically.
Can I go from [Another Format] to 3DS+Lua? Yes, if you can save to 3DS first. You don't need to stay in 3DS, it's just to feed UpSpring.
Can I still use UpSpring? Yes if your assets are 3DS, OBJ or 3DS+Lua
Do I even need UpSpring? No. Even more so if you already generated your Lua or you prefer to do it by hand.

Will you fix UpSprings bugs? Yes, if they are relevant to this process.
Will UpSpring still be maintained? As long as it's still useful, yes.

Hope that helps clear things up. Dropping 3DO/S3O != Dropping your assets.

...not to mention old games without maintainers that would suddenly not work.

They don't work now. Maintaining a mod requires, you know, maintenance. The engine is always a moving target.

You guys all sure that fixing things that aren't all that broke instead of just moving on to MD5 for future content is a good idea?

Assimp supports MD5. The MD5 import is currently broken, it's worked in the past so this is either a temporary thing or an issue with how I built the library.

But breaking every map that use custom features is kinda

They use they same loader, so they won't break if the map or map folder is converted. I'll even offer to batch convert the entire jobjol archive and post the converted assets online.

NOT a proposal to remove S3O / 3DO support from Spring.

Pfew! Thanks for the disclaimer, you would have me worried otherwise.

I should have clarified that. I meant without providing an automatic upgrade path for mod authors AND end-users. In short, if you have S3O/3DO assets you don't have to throw them away or spend hours redoing them. Spring can even detect old archives and convert them on-the-fly however I don't know what this will do to MP sync so I'm asking mod authors to do it instead.

This whole debate could easily become irrelevant anyway, if somebody wants to port the current S3O/3DO loading, rendering and texturing code to the new system. The issue is that the relevant code is not all neatly tucked away in its own classes; It's splattered across the rendering, unit, COB and Lua code like so much roadkill.

[Argh]

They don't work now.

The ones that don't rely on old, deprecated Lua still work just fine. Like, IDK, old NanoBlobs versions, etc. all work just fine. Final Frontier works all right, E&E etc. They just aren't as pretty as new games.

I mean, I'm all right with "moving on" in general. That's fine. But you're not "moving on", you're proposing changes that would break years of backwards compatibility.

Aren't there any reasonable alternatives to that?

[MidKnight]

I think the best course of action you could take right now is to...

And do the debating afterward.

[SpliFF]

I mean, I'm all right with "moving on" in general. That's fine. But you're not "moving on", you're proposing changes that would break years of backwards compatibility.

I'm proposing changes that break things and providing a means to fix those things at the same time. I'm proposing a solution that is automatic and should take less than 5 minutes from start to finish

Aren't there any reasonable alternatives to that?

Yes, converting the assets when Spring starts. Most people playing those mods are probably doing it SP. In addition I only have BrainDamages word for it that repacking archives desyncs them. Even if true it's entirely possible that issue can be overcome by adding a streflop'ed packer or better sync checking.

[Argh]

Just throw them in a folder like we do with map paths, etc. No need to repack the archives at all, they'd just need to get re-built if the sources' hash differed.

Seems like the logical thing to do. And then if you have 5 things with models named "tree5", it won't be an issue, either, because their hashes would be different, which would be nice (right now that results in a namespace problem, and stuff wins / loses).

[SpliFF]

...not to mention old games without maintainers that would suddenly not work.

Unless the user or someone else converts them.

You guys all sure that fixing things that aren't all that broke instead of just moving on to MD5 for future content is a good idea?

You know, you have this thing for MD5 but it's really not a great format to be focusing on. If I HAD to pick a single format (which I don't) it would be Collada.

Collada is a newer format, designed by the Kronos Group (the Open GL spec writers) and is more general purpose, easier to use, less proprietry, more likely to be supported by tools going forwards and has more features (like embedding textures and physics support). It also works right now, whereas MD5 does not, which could possibly be a testament to the fact it's better documented and easier to parse.

[FLOZi]

Dropping 3DO/S3O != Dropping your assets.

Debatable at best.

[Argh]

Actually, I don't care whether it's MD3, MD5 or Collada. I've just been assuming that support for it on Blender is still pretty weak, since it's still new and in development, as opposed to the id stuff that's been mature for years, is all. Since those of us not willing to pirate Max / Maya will have to use Blender, it's important that support for the format be solid. I guess that's something worth investigating.

[SpliFF]

Just throw them in a folder like we do with map paths, etc. No need to repack the archives at all, they'd just need to get re-built if the sources' hash differed.

FINALLY! Now we're talking about SOLUTIONS instead of worrying about problems. If that works as you say it means even the end-user can get automatic conversion of older formats in the event they don't have a converted mod. Everyone wins.

[SpliFF]

Dropping 3DO/S3O != Dropping your assets.

Debatable at best.

There is no debate. If the assets can't be converted then the engine doesn't get changed. You're not just going to wake up and find S:44 broken and unplayable.

[Argh]

Well, I try.

That did seem like the best way out- no futzing about with developing a new tool for us cave-people with lots of S3Os we aren't planning to lose even if I can use Collada or whatever tomorrow (we could just keep using UpSpring and pretend that you weren't doing anything), and no need for a GUI, or any other stuff... just perfect, no-nonsense, absolutely correct conversion

[SpliFF]

...as opposed to the id stuff that's been mature for years, is all....

Mature as in supported by 3rd-party plugins of mixed quality and features. Collada is far more likely to become a core standards format like PNG, XML or HTML. It's still early days, I was really just pointing out that MD5 is not the only format to support advanced features.

2.) If we write a batch converter for these assets then we want to encourage mod owners to use it, otherwise it means running the converter client-side the first time a legacy mod is loaded. Writing converted models back to the mod archive wouldn't be hard but BD warns me that repacking a mod client-side, even if every client does it the same way, could cause desyncs.

So basically this whole thread has been a giant WANK. You want Spring to load S3O/3DO... it can. It just needs to postprocess them the first time they're used and write them to a folder (like path cache). Then assimp can treat them as native objects and we go back to only needing 1 rendering class, which is what I wanted all along. No mods get broken, no mod authors have to do any work. It's win win.

[Argh]

Mature as in supported by 3rd-party plugins of mixed quality and features.

Well, sure... but if it's "available" but there's no content workflow that doesn't require buying / pirating high-end software, I don't really think that's in keeping with Spring's overall mission. The only stuff I've really heard about Collada is Max and Maya, but I don't keep up on these things- perhaps Blender's also very happy with it at this point. I'll go read what Collada says atm.

OK, according to Blender, as of 2.42, COLLADA support was added, and it's officially listed in their export file formats. I guess that means that it's down to the inevitable fine print, but that sounds good.

So basically this whole thread has been a giant WANK.

Not really. We've gone through the classic process of identifying the business case, is all. I think that that is an elegant way to solve the over-arching problem, and if it's developed that way, I'm satisfied that it's where things need to go. There will be inevitable problems with making *sure* that it can sync, after being converted to IEEE-spec numbers and back again, I suspect, so any celebration is probably very much in the future.

So now you "just" need to convince jK that it's not a Massive Waste of Time, I guess.

I understand his POV on this, and I'm still not sure why you can't just have a "new format" switch tied to unitDef that allows for all-new rendering behaviors, instead of trying to fix all the old spaghetti code, which probably won't speed things up a lot in the end, by the time you hook COB into it all over again. But I'll leave that argument for the experts.