Drawing Order

[Argh]

Meh. Look, I'll explain everything, maybe people will start to get it. The only real questions are:

A. Do we want working Unit translucency in this engine?

B. If yes, do we do it with the current Unit rendering code, which means that my proposed fix is required, or do we do it the way the commercial engines do, which requires a new geometry specfication and rendering pipeline?



I'll explain why those are our choices, after I explain why it works at all. I think that after I'm done, everybody will get it.

Extra reading, for people who don't really get stencil test concepts yet:

http://psysal.livejournal.com/67933.html

For a less-folksy (but I personally found it more clear) explanation:
http://www.opengl.org/documentation/spe ... ode97.html


First, some previous work. I've been playing with the stencil buffer and Unit geometry for quite a long time now:

1. The IonLaser's explosion creates a translucent Unit that's drawn with gl.Unit. It uses the following OpenGL settings:
Spring.UnitRendering.SetUnitLuaDraw(unitID,true)

glDepthTest(GL_LEQUAL)
glAlphaTest(GL_GREATER,0)
glBlending(GL_SRC_ALPHA,GL_ONE)
glTexture("unittextures/AAFX_ExplodeSphere.tif")
glColor(1.0,1.0,1.0,0.5)
glUnit(unitID,true)

This is how the Ion Laser's explosion works. It was "top secret" tech when I made it, lol, because obviously nobody had thought of using Unit geometry to create an ultra-fancy, 3D explosion effect. If I did it again today, I'd probably still do it like that.

However, that's not really relevant. What's relevant is that this Unit geometry is drawn during DrawWorld, so we have to set up culling appropriately. Note that GL_GREATER. That will show up again here in a minute, it's very important.


2. Ok, let's look at the current version of Blackbox, my code for drawing around the Map geometry now that I don't use fog. Here, we have a different problem- we do NOT want it to win any depth tests, otherwise it overwrites stuff.

glResetMatrices()
glColor(1,1,1,1)
glBlending(false)
glDepthTest(GL_GREATER,0)
glDepthMask(true)

glTexture(texture2)
glPushMatrix()
glCallList(box)
glPopMatrix()

Hmm. There's that GL.GREATER again. No alpha-test is required here. With GL.GREATER, it loses the depthtest to anything drawn later, which unfortunately includes water, since we can't turn off water rendering past the map edge.


3. Finally, let's look at the UnitRendering settings for the modified version of Kloot's code:

glPushAttrib(GL_ALL_ATTRIB_BITS)
glAlphaTest(GL.GREATER,0.01)
glBlending(GL.SRC_ALPHA, GL.ONE_MINUS_SRC_ALPHA)
glDepthMask(true)
glDepthTest(GL.LEQUAL)
glCulling(GL.BACK)
glLighting(true)
glLight(0, true)
glLight(1, false)
glLight(0, GL.POSITION, sdx, sdy, sdz, 0)
glPushMatrix()
glColor(tcr, tcg, tcb, tca)
glTexture(4, normalMapNames[unitDefID][1])

DepthTest must be GL.LEQUAL, to ensure that Units don't draw through mountains or lower Pieces through upper ones. It's unavoidable- it must work like that, for solid geometry.

AlphaTest is what we're concerned about here anyhow. When set to GL.GREATER,0.01, anything below that 0.01 is going to lose the masking test with other geometry. That's to handle transparency properly.

To handle translucency, it's necessary to use GL.GREATER, because that allows translucent objects to win the alphatest and get drawn, as opposed to being culled.

However, since OpenGL doesn't "know" that they're translucent, it writes a solid stencil to the buffer. Geometry drawn later that isn't depth-sorted will always lose that alpha-test. So... to make things work, we must depth-sort the objects. The question is simply how, not whether. You can avoid depth-sorting internal to a particular piece of geometry (as I'm about to explain) but you can't avoid the stencil buffer.

As for how I'm depth-sorting the geometry internally to a Unit... I took a page from Freelancer and the other professional engines. They assign materials to each part of an object, and do a sort based on material type before drawing the pieces. I've looked at several other model formats (MD5, the FO3 format, DoW) and this is clearly the way most professional engines do things. They have materials assigned per piece. With S3O we've been locked to one material and one rendering method per Unit, and I've said several times why that's a problem, but maybe people will listen, now that I know how to explain the technical ramifications.

I don't think that most professional engines bother depth-sorting the pieces. They're just doing what I did. They just draw the translucent stuff last and hope for the best, and 99% of the time, nobody can tell.

But, in the end, they have to depth-sort the objects in their scenes, just like I'm proposing.

This is almost certainly why there's very little translucency in Fallout 3's engine- my guess is that, due to their system of building worlds, where pretty much everything other than exterior map base meshes are objects (that includes rocks down to the last tin can) they ran into major problems with depth-sorting on their huge draw distances in their large scenes, and eventually gave up, for performance's sake.

But I digress- I said I'd explain how I do it with S3O, so here it is.

It's very simple. S3Os are drawn in a very strict order, from the top-most piece in a hierarchy to the lowest. It's completely consistent.

I discovered this by accident, and quickly realized what I had- crude sorting right there in the geometry format, without having access to per-piece materials. And therefore, I don't need an internal sort of the Unit triangles, so long as I'm not doing anything really complex (there are situations where this won't work, of course, but they'll take artists pushing designs a lot farther than I need to to break stuff).

However... when I tested things, I quickly saw that the Units themselves aren't being depth sorted. This is what's causing the problem. So I proposed this fix.

It won't fix translucent stuff vs. water- my translucent objects will always win vs. water, culling it. That's something else that could be addressed by deferring rendering of Pieces based on their material assignment... but that requires a new version of S3O, one that includes a material tag... and appropriate support in the engine.

So, there ya go. That's the whole story, and a fuller explanation of why this is necessary, from a technical perspective. There simply aren't any other ways to do it given how the engine works right now, folks.

To do it the right way requires a very different approach to Unit rendering. The graphics geniuses in the professional world probably use materials not only to defer stuff, but also to use the most efficient shaders possible per material. I suspect that when you add that up, it's a fairly significant advantage in terms of final performance.

We can have that, but not with the current format and rendering pipeline, where it's assumed that all Pieces are created equal, have a tex1 / tex2, use the same OpenGL methods, etc.

[KaiserJ]

maybe this isn't a great suggestion...

but instead of using a true translucent unit part, couldn't you instead fake it by applying a texture to that part that had transparent areas?

like if the x was a blue pixel, and the _ was a transparent pixel...

x_x_x
_x_x_
x_x_x

from a distance, it would look translucent, the same way archie comics skin appears to be made up of measles from close up; it's worked for me making models for other applications before....

would it stop the culling problem and not impact performance too much?

[Argh]

You can't use transparency to do a stippled effect. Mipmaps mean that at a distance it's a solid.

I appreciate the thought, though. As I've said, I can deploy what you've seen screenshots of in the current engine, though. The only reason I even opened this thread is because I want to do more than minor stuff like real windows on buildings, that bubble-farm, etc., and certain things are impossible atm. Fixing the drawing order would fix some of it... having a genuine per-Piece material system would fix a lot more of it.

[momfreeek]

a bit of pure logic (pls at least verify the statements carefully):

Code: Select all

STATEMENT 1:
there are issues with a translucent model being drawn before solid models that appear behind them.

STATEMENT 2:
there are *no* issues with a translucent model being drawn after solid models that appear in front of them. (surely if this were false you would have noticed it?)

CONCLUSION:
you will avoid any issues by drawing the translucent model last

If the 2 statements are true (and assuming the 2 obvious cases of drawing them in proper distance order have no issues), then the conclusion follows absolutely.

DISCLAIMER:
this only takes into account 1 translucent model.. sorting the translucent models may be necessary (drawing the rest of the models first.. but randomly)

[Kloot]

1. you aren't using the stencil buffer AT ALL, nor is it related to this topic
2. you can Lua'ize a per-piece matsys if you're so convinced that's the Right Way (it isn't)

[Argh]

1. you aren't using the stencil buffer AT ALL, nor is it related to this topic

Er... when a given object is drawn, that's a stencil mask, isn't it? Or am I getting a term wrong here? I need to go re-find the article I read about depth-masking, etc., I was sure it used the word "stencil" to describe stuff that's already in the framebuffer vs. stuff that's being rendered now...

2. you can Lua'ize a per-piece matsys if you're so convinced that's the Right Way (it isn't)

Well, what is, in your opinion?

I'm perfectly willing to believe that I am Doing It Wrong, but I'd like to hear how we're supposed to avoid the logical problems I've described, in terms of drawing priority.

And a Lua'ized mat-system would be unbelievably slow, given that I'd have to do the piece transformations all over again to make it work, IIRC.

[Argh]

Also... this appears to agree with my conclusions about there being no truly perfect solution:

http://www.opengl.org/wiki/Alpha_Blending

I'm not asking for perfection. I'm just asking for it to work well enough that major borks are fairly uncommon. What I originally asked for should remove a lot of borks... and a translucent tag and second sort should take the error rate down even further, probably well within acceptable ranges.

At any rate, if somebody has a better idea and can demonstrate it's actually working, I'll be happy to use it. I figured everybody would be happy with working translucency, not that I'd be having a huge and fairly boring quasi-argument with people.

[souledge]

Congratulations on finding a well known problem in computer graphics, good luck "fixing" it for spring.

[Argh]

You should probably view the screenshots of how well it's already working and the scope of the requested change on the first page before making further comments.

[souledge]

You should probably view the screenshots of how well it's already working and the scope of the requested change on the first page before making further comments.

Well you seem to have backtracked quite a bit from the intention of your original post to something that will improve rendering under certain conditions.

You seem to have a very different definition of working. If you say you've got it working, post a screenshot of some nice big colourful units where every texture is translucent.

[Argh]

Wish granted, the entire object, translucency 128:

Before:


After
So... eh... I guess you didn't get that whole thing about drawing order within the geometry. Are we all on the same page now?

[hoijui]

i only get little of the whole discussion.
what i understood:
Argh: units drawn in the right order needs to be done.
others: ordering is too expensive
Argh: no, we just use one point per unit (center)
others: to inaccurate
Argh: better then no ordering

[Argh]

One last step:

Argh: if too inaccurate, have a UnitDef tag on translucent stuff, do two smaller sorts to make it accurate enough.

But yeah, other than the long-winded blah-blah, that is pretty much it.

[jK]

no, it is:

Argh:
Hey, I wrote some uber-lua. While doing so I found a bug. Fix it! You only have to do "foobar".

the truth is:
1st, he just added

Code: Select all

gl_FragColor.a = texture2d(tex0, texcoord).a;

in kloot's lua glsl-shader and perhaps deactivated the alphatest with:

Code: Select all

gl.AlphaTest(false);

. So he didn't anything great or similar.
2nd, it is possible to support transculant model drawing, but it is everything else than "very basic", nor is it a 5min job! The current model rendering code is a hell, it was just written for 1 model format (3do), then someone pushed s3o with brute force into it, then ppl added new rendering passes (cloak, inbuilt, reflective, shadow, ..), then trepan build on top of all his lua UnitRendering API (it is the only clean thing in the whole code!), and ppl still push more stuff into it. Fact is that it isn't written for all those stuff and it already explodes: all those dependencies on advshading, inconsistence rendering between the diff passes, ati bugs which are hard to debug, slow rendering in whole, a hell of work to add something new w/o breaking something old, ...
So any useful changes in the model rendering passes are bound to a total rewrite of the WHOLE model renderer, what is a 3-4month job!

[Argh]

What part of "you're wrong, and my screenshots show that you're wrong" do we need to go through again?

I'd be the first to say that yes, we need a new model format with a new rendering system.

But in terms of the specific problem as presented, you're completely wrong, again.

Proof. Reverse the transparency of the unit geometry- first Piece is translucent, last Piece is opaque:

As you can clearly see, jK's statements are incorrect. The translucent objects, being drawn first, are winning the alpha test.

The order in which things are being drawn determines the winner, it's not that complicated to understand.

[AF]

I like how you lot complain so much and get so riled over argh, yet you yourselves kick up such a great fuss about it that your just as bad.

Argh can indeed be faulted, but he's not the only one who revels in the situation. Why else do you guys attempt to disprove argh if he is so blatantly wrong? Let Arghs work speak for itself. If he is wrong then he is wrong, and you dont need to pedantically correct him.

And argh, when at an impasse, it is unwise to perpetuate it, if only because your wasting your time doing so. That time reversing stuff and taking the screen shot and posting to try and disprove Jk was an utter waste of both your time and our time.

Grow up people

[Peet]

Argh can indeed be faulted, but he's not the only one who revels in the situation. Why else do you guys attempt to disprove argh if he is so blatantly wrong? Let Arghs work speak for itself. If he is wrong then he is wrong, and you dont need to pedantically correct him.

The problem with that is that other people who don't know anything about the habitual quality of argh's information will be greatly misinformed.

[jK]

What part of "you're wrong, and my screenshots show that you're wrong" do we need to go through again?

just READ ... please once ...

I'd be the first to say that yes, we need a new model format with a new rendering system.

I didn't said anything of a new model format! We don't need a new proprietary model format ...

But in terms of the specific problem as presented, you're completely wrong, again.

Just show 1 line where I was wrong.

Proof. Reverse the transparency of the unit geometry- first object is transparent, last object is opaque:

http://www.wolfegames.com/TA_Section/gl ... ency10.jpgAs you can clearly see, jK's statements are incorrect. The translucent objects, being drawn first, are winning the alpha test.

WTF - negative meaning -

[jK]

Argh can indeed be faulted, but he's not the only one who revels in the situation. Why else do you guys attempt to disprove argh if he is so blatantly wrong? Let Arghs work speak for itself. If he is wrong then he is wrong, and you dont need to pedantically correct him.

The problem with that is that other people who don't know anything about the habitual quality of argh's information will be greatly misinformed.

correct, else we (the devs) get blamed for not listening to the great Argh.

[Argh]

I just showed you that, depending on the order in which translucent objects are called vs. opaque ones, you get different results when culling occurs.

So... basically, you're wrong, and I'm right. It's simple science, man. I keep showing you proof, you keep throwing words at me. Pictures don't lie.