SM3 (split from Mesh Deformation, Skeletal Animation, ...)

[lurker]

Your post is a little unclear, but I guess you're saying you want to use a really low resolution blend map and thinking about fitting it in an existing texture? I understand wanting to save textures, but yes that's rather limited use, and you seem to be forgetting that each blend map texture has four channels in it, not one.

[Argh]

Yeah, I'm saying, since we need three channels for a normalmap, a grayscale for depth, but we have that alpha channel in the diffuse texture still floating free... why not use it for the blendmap?

IOW... two RGBAs, and all 8 channels are actually being used for something vital at that point. Would save a third texture transfer, and the main cost is that we wouldn't have a channel free for specular maps.

If we're confused about input vs. output states- the output state would need three textures, if the whole "assemble-a-map" concept is still in play, because we would need at least two 8-bit channels that we would not have available otherwise- the final values of specularity and reflectivity.

I was kinda assuming it's not, that we're going to try and get SM3 running and that will be that for now.

OK, really going to bed, I should have been asleep hours ago.

[lurker]

Stop saying that. It's not saving a transfer. You're doing many at a time, it saves a FOURTH of a transfer. And it only works under certain conditions.

I find it interesting that you've expressed a need for normal maps at the same resolution as the color textures but you haven't even noted in passing the fact that storing one in RGB, compressed, will damage it pretty well.

[Argh]

Of course it damages it. That's just the cost of doing business.

Pretty much all commercial games use DDS compression for normalmaps. They have errors. It's not a big deal.

I just don't see any point in arguing about this- low-resolution normalmaps look like ass, it's that simple. "Best Practices" in the Industry is actually to use 2X sized normalmaps, if possible, for even finer detail and less-obvious errors, but 1:1 ratios will look fine for maps if the tile scale's arbitrary- you can just shrink it a bit more, and voila, all but the most egregious errors (artists' fault, basically) will just disappear.

[lurker]

Do most games still store the normalmap as RGB? If they're doing 2x and getting as many pixels in as possible it can make sense to eat the extra damage that causes compared to other storage methods. Since RGB takes *half* the space of RGBA. And that's another reason not to use a dedicated alpha channel for blend maps if you can avoid it, very space-expensive.

[Argh]

No, most commercial games actually read the pixels from the bitmaps and store them in a different order, or the artists format the DDS a different way than the standard nVidia settings- Carmack came up with an optimization, go read about it, I certainly can't quote it atm.

And I thought that the A was necessary to have a working depthmap shader, as opposed to just a bumpmap, but somebody with more experience would have to address that.

Can we live without depthmaps? Yeah, but we'll regret it in the long term, imo, and we'll be right back to having this same discussion in another year, when it's obvious that the job wasn't finished and our new, spiffy, "advanced" format starts looking dated again.

Let's just say that having to drop depthmaps from Kloot's shader, to get shadowmaps and the rest of the normalmap shader working on my hardware... was painful, in terms of the lost quality of the final images. I just didn't have a choice- it simply wouldn't compile on pre-8800 hardware otherwise.

[lurker]

Has anyone suggested removing depth maps from sm3? And why did you have to do that to get a shader to compile?

Alpha channel needed on normal maps to make depth maps work? That doesn't make any sense to me.

Okay, if you're saying that about normal maps now you must have answered a different question earlier than the one I asked. So I'll give the appropriate response. Because of that method of normal map storage (that I was already aware of and talking about, you don't need to tell me to go read it) to get good-looking results out of it, you no longer have a spare channel in the texture.

[Argh]

So, you're basically saying, 3 RGB? If 2 RGBA == 4 RGB, OK, makes sense to me. I thought that each channel had an equal cost, guess I learned something new there. But what about the cost of doing the texture fetch? Oh, duh... you're going to put this in PBO, it isn't dynamic...

why did you have to do that to get a shader to compile?

Because the extra data that had to be sent over from the vertex shader for a depthmap to work wouldn't fit under the limit of 32 floats allowed on a 7800. Meh, already explained all that elsewhere, not going into it again.

[lurker]

Oh. Annoying. Well I'll look for where you wrote that.

2 RGBA take the same storage space as 4 RGB, yes.

If you want to keep things high quality, then your layout would look something like this:
RGB color + A height
RGBA for the normal map
RGB for specularity, reflection, and glow or something

[lurker]

Crash course on DXT compression:

RGB is compressed like this:
Take your 4x4 pixel block.
Find two colors that represent this block. Store them in 565 format.
For each pixel in the block, store a number from 0-3.
0 is color A, 1-2 are in the middle, 3 is color B.
This comes out to 64 bits for the block, or 1/8 of 8888

Alpha is compressed like this for DXT3:
Store your alpha values at 4 bits per pixel. Done.
This comes out to 64 bits for the block, or 1/8 of 8888

Alpha is compressed like this for DXT5:
Pick two alpha values, then use 3 bits per pixel, [details omitted].
This comes out to 64 bits for the block, or 1/8 of 8888

[Argh]

Right. I did know that ratio- the 1/4 1/8 stuff. That's one of several reasons I switched to DDS. Just wasn't thinking about it in terms of storage on the GPU for this, too tired.

As for the "annoying", yeah. I keep thinking there must be a way to either compress two or more results and unpack them on the fragment shader without losing too much floating-point accuracy, or something, but I eventually gave up after trying a few things, I know jack squat about compression.

[lurker]

Sweet dreams.

Oh, and another small note about storing a blend map in an alpha channel on any of these textures is that you'll end up having to access twice the number of pixels from your shader, one multiplied by the blend map scale, one not. Can this have an impact?

[jK]

time to cleanup this discussion

first some facts about normalmaps:

• compression of normalmaps is a problem old as normalmappping itself
• you only need 2 channels for a normalmap - the 3 coord can be reconstructed via one sqrt (you would have no matter what to do 1 sqrt cuz the values in a compressed texture are everything else than normalized, so this doesn't cost any performance!)
• saving those channels in a DXT compressed texture can give quite good results when using the alpha channel
• in the case of post 8xxx(nv) / hd4xxx(ati) gpus there are even better ways (3DC -> EXT_texture_compression_latc / EXT_texture_compression_rgtc)
• so saving the normalmaps in lossless compression formats (png?) in the map format and then dynamically handling the vram uploading (DXT vs. 3DC vs. uncompressed), is the best way

And btw not each diffuse layer needs a normalmap.
Also the heightmap is quite redundant in sm3, cuz it's just used for ugly bumpmapping. It can still be used for relief-mapping, something that can heavily increase the visual quality but costs also a lot performance, so it should be limited to 1 layer and the user should be able to disable it.

Saving the blendmap in the tilemap texture isn't smart, because you would need 2 texture accesses to read it from there (tiling vs. fullmap texcoords), so saving all blendmaps in separate RGBA textures is very self-evident.

Then there is also a pretty neat way to reduce texture bindings + lookups: 3d-textures.
Mostly you just blend 2 texture layers per pixel, something that can be done via 3d-textures (yeah, you can even put 4 and more texture layers in one 3d-texture!). It wouldn't just save texture bindings + lookups + better texture caching, you would also just need 1 blend channel for the whole 3dtexture.
The only problem can be the texture size limit for those, so I remember nv just supports 384^3 (pre8x) - 512^3 (post8x), but it is imaginable that this limit just depends on the total texels of the texture, so perhaps 1024*1024*4 textures are still possible (this would need some testing).

And btw using the full 16 texure bindings is pretty madness! Something like that kills any available vram bandwidths!

`Addendum`
here a list of texture bindings needed for any terrain renderer:

• terrain normalmap
• shadowmap
• infomap (LOS,metalview,...)
• horizon cubemap for diffuse lighting
• horizon cubemap for specular lighting (can be perhaps merged with diffuse lighting)

[Beherith]

So, does bump mapping work in latest? If yes, then to what level does it work?

Edit:
Ive been testing around with it, and it seems complete, save for one thing; Can we define a detail texture per shader as well? Or at least some sort of support for overlay mode blending.

[Tobi]

Then there is also a pretty neat way to reduce texture bindings + lookups: 3d-textures.

Hmm that seems like a useful idea

`Addendum`
here a list of texture bindings needed for any terrain renderer:

• terrain normalmap
• shadowmap
• infomap (LOS,metalview,...)
• horizon cubemap for diffuse lighting
• horizon cubemap for specular lighting (can be perhaps merged with diffuse lighting)

AFAIK SM3 calculates specular lighting inside the shader without requiring a cubemap as 'lookup table' (as Spring unit renderer does), and also it doesn't do cube environment mapping (I assume you mean that with horizon cubemap for diffuse lighting, as I can't place it otherwise.)

So that saves two textures again

Though I guess it could be cool to make metal map with true reflections

[jK]

AFAIK SM3 calculates specular lighting inside the shader without requiring a cubemap as 'lookup table' (as Spring unit renderer does), and also it doesn't do cube environment mapping (I assume you mean that with horizon cubemap for diffuse lighting, as I can't place it otherwise.)

So that saves two textures again

Though I guess it could be cool to make metal map with true reflections

SM3 doesn't use normalmapping yet, with normalmapping the quality of the used lighting formula becomes more important.
And with diffuse lighting cubemap I don't mean the reflection cubemap, I mean something like a Irradiance Environment Map, it can be constructed by blurring the reflection map, but you can also bake hemisphere lighting etc. into it.

PS: reflections on the terrain are pretty hard to realize, because you don't have a linear reflection plane.

[Tobi]

SM3 doesn't use normalmapping yet, with normalmapping the quality of the used lighting formula becomes more important.

Why is that?

Does it become a different (more complex) formula then some pow and clamp, or are the shader functions just not accurate enough?

(Intuitively it seems quite a waste to me to use a whole extra texture + bandwidth just to make something like specular lighting (slightly?) more accurate. Or are the calculations just so much slower then a texture lookup?)

EDIT: okay read a bit of wikipedia, suppose there are enough utterly complex lighting models to make the calculation slower then a texture lookup

And with diffuse lighting cubemap I don't mean the reflection cubemap, I mean something like a Irradiance Environment Map, it can be constructed by blurring the reflection map, but you can also bake hemisphere lighting etc. into it.

PS: reflections on the terrain are pretty hard to realize, because you don't have a linear reflection plane.

Ah right.

With 'true reflections' I actually meant only as true as unit reflections currently are; i.e. using the cube reflection map that's already generated in the unit drawer. (Sorry for the confusion.)

[Argh]

All I care about is the cubemap, I don't even care too much about Unit / other draw-function reflections, although that would be the finishing touch for "omg it's awesome".

Oh, and this may be important: "transpose" is not a valid operation on most ATi. When building the final shader, it needs to transpose manually:

Vertex:

Code: Select all

               //Manual transform of the matrix, for GLSL 1.X compatibility
               mat3 tbnMatrix = mat3(svOS, tvOS, gl_Normal);
               tbnInvMatrix[0].x = tbnMatrix[0].x;
               tbnInvMatrix[0].y = tbnMatrix[1].x;
               tbnInvMatrix[0].z = tbnMatrix[2].x;

               tbnInvMatrix[1].x = tbnMatrix[0].y;
               tbnInvMatrix[1].y = tbnMatrix[1].y;
               tbnInvMatrix[1].z = tbnMatrix[2].y;

               tbnInvMatrix[2].x = tbnMatrix[0].z;
               tbnInvMatrix[2].y = tbnMatrix[1].z;
               tbnInvMatrix[2].z = tbnMatrix[2].z;

Fragment:

Code: Select all

                  //Manual transform of the matrix, for GLSL 1.X compatibility
                  mat3 tbnMatrix;

                  tbnMatrix[0].x = tbnInvMatrix[0].x;
                  tbnMatrix[0].y = tbnInvMatrix[1].x;
                  tbnMatrix[0].z = tbnInvMatrix[2].x;

                  tbnMatrix[1].x = tbnInvMatrix[0].y;
                  tbnMatrix[1].y = tbnInvMatrix[1].y;
                  tbnMatrix[1].z = tbnInvMatrix[2].y;

                  tbnMatrix[2].x = tbnInvMatrix[0].z;
                  tbnMatrix[2].y = tbnInvMatrix[1].z;
                  tbnMatrix[2].z = tbnInvMatrix[2].z;

I figured that that part of my discussion about getting the normalmap shader working on ATi probably wasn't well-known, it's a gotcha problem, so I thought I'd share before any wheels get spun. BTW, have any ATi users tested SM3 yet?

[jK]

@Argh
You don't need the inverse/transpose of the TBN matrix for Normal Mapping.

[jcnossen]

PS: reflections on the terrain are pretty hard to realize, because you don't have a linear reflection plane.

You could bind the heightmap as a texture and then raycast your way through it within the shader