Stupid Lua Question- GLSL Shader Integration

[Argh]

Holy smokes. Turns out that if you use a transparency texture, you can hose the rendering of an object. All sorts of horrible z-fighting ensues.

And UnitNoDraw is basically like "hide" I wish it would draw via a shader when that was true, but no.

I can give it a minimal texture, iow, but I can't really make it disappear entirely and figure out what's causing the clipping. Meh, I've been up all night working on this, time to sleep for a few hours, then do RL stuff.

@Chosker:

1. I'm not talking about just covering it with a cubemap, that would look really horrible.

I'm talking about using another texture, or the G value of a texture, to determine the amount of reflection that's possible, like we do now, which uses a cubemap Spring generates. Basically, I think that it would be wiser to duplicate the current functionality of S3O, adding new capabilities to it only when it all works correctly, rather than put everybody in a position where they'd need to redo months of work on textures. For example... use the G value to generate the right value of reflectivity... then if a switch is on, use a specular map to change the final color of the fragment. Or something. Basically, my goal on that is to not make everybody, including me, feel like they have to start over or re-do their work- I'd prefer if everybody just made their normalmaps and called it a day, pretty much.

It's effectively a specmap, the only problem has been that we have no control over the resulting specular color value, which sucks because it's hard to get a lot of materials right without it. I brought that up maybe two years ago, nothing happened, maybe it can be solved this way.

2. This is a simple dot3 bumpmap. It's not a relief shader, nor do I have much interest in tackling one right now- the objects in P.U.R.E. were designed with dot3 in mind, and it's fast, which is relevant.

If I can get reflection / specular / glow maps working, then it should look pretty spectacular, I suspect. While I think that a relief shader is a cool idea, I am pretty dubious about its utility with the unit-counts of a RTS... especially as Lua rendering modes in Spring do not currently support DDS correctly, so mipmaps are an issue. Dunno, I guess that's just going to need performance testing under real-world conditions... if it all ever works.

3. I'll release the implementation sooner if I get some help.

I've solved the brute-force side of this problem- the basic issue of lighting angles, etc., was mainly just a matter of trying enough things until I found the solution, not some amazing genius on my part. Kloot's comment told me what was wrong, so I kept hammering away at it until I found the right matrix input to solve the lighting problem (that's a big hint, btw, if you're trying to duplicate this).

Meh, I'm going to be gone for the rest of the weekend. If people would like to step up and maybe find me some sourcecode, that would be useful. I *think* I understand enough about how what I did works that I can "hook up" other algorithms into the lighting algorithm, but I haven't backtracked and tested that assumption yet.

[lurker]

Holy smokes. Turns out that if you use a transparency texture, you can hose the rendering of an object. All sorts of horrible z-fighting ensues.

Yeah, that's pretty well-known.

[jK]

Holy smokes. Turns out that if you use a transparency texture, you can hose the rendering of an object. All sorts of horrible z-fighting ensues.

Yeah, that's pretty well-known.

That's why it isn't implementated in the engine ...

and btw you don't have any normalmapping ...
You didn't created any binormal space matrix at all, you just set up a random matrix which gives a decent result, but if the unit moves or if you have multiple pieces (which have their own object space matrix) you will directly see that your shader does some random stuff but normalmapping.

[Argh]

and btw you don't have any normalmapping ...

That's a light at 10000,100,10000. Just thought I'd demonstrate this in a way that was a little more... obvious. And yes, I'm aware that the second one is lit opposite of how it should be- it's an issue with heading and the resulting matrix, just too lazy to fix it yet.

Light at -10000,100,-10000:


Look, people. It's far from perfect, and has three major flaws:

1. The clipping issue.

2. There's an issue with heading.

3. There's something odd going on with stuff that's nearly perpendicular to the lightsource- it tends to get lit the opposite of how it should, just like the heading issue.

I suspect that problems 2 and 3 can be solved by initializing the value of the light vector at the start, then never re-calculating it again. It will run faster, too.

All that said... it's a working dot3 normalmap, and all of these problems are solvable, imo.

Now... are you guys going to pitch in, or should I just expect further bullshit and snarky replies to this?

[Chosker]

1. I'm not talking about just covering it with a cubemap, that would look really horrible.
I'm talking about using another texture, or the G value of a texture, to determine the amount of reflection that's possible, like we do now, which uses a cubemap Spring generates. Basically, I think that it would be wiser to duplicate the current functionality of S3O, adding new capabilities to it only when it all works correctly, rather than put everybody in a position where they'd need to redo months of work on textures. For example... use the G value to generate the right value of reflectivity... then if a switch is on, use a specular map to change the final color of the fragment. Or something. Basically, my goal on that is to not make everybody, including me, feel like they have to start over or re-do their work- I'd prefer if everybody just made their normalmaps and called it a day, pretty much.

It's effectively a specmap, the only problem has been that we have no control over the resulting specular color value, which sucks because it's hard to get a lot of materials right without it. I brought that up maybe two years ago, nothing happened, maybe it can be solved this way.

ok good, I don't want people to redo stuff either, and I don't really see a reason why it should be that way.
I don't really see why you would want to control the resulting specular color value though. most of the time a "lightcolor" value suits just fine. otherwise it would be used for *very* subtle effects (you'd never really want a full-red specular), hence why I think it's not needed

2. This is a simple dot3 bumpmap. It's not a relief shader, nor do I have much interest in tackling one right now- the objects in P.U.R.E. were designed with dot3 in mind, and it's fast, which is relevant.

good. I never mentioned it being relief bump :) however there's a big difference when a dot3 normalmap is generated from a hi-poly model than when it's only generated from an nvidia-tool-converted heightmap
I'm posting this image as an example:

it's a photoshop-generated normalmap vs. a 3d-generated normalmap with the photoshop-generated one overlayed on top. if you saw it in 3d you'd see the difference is huge in terms of amount of detail, mostly because it describes curves alot better. then again it's kinda hard to do if it's a tank with little curves

If I can get reflection / specular / glow maps working, then it should look pretty spectacular, I suspect. While I think that a relief shader is a cool idea, I am pretty dubious about its utility with the unit-counts of a RTS... especially as Lua rendering modes in Spring do not currently support DDS correctly, so mipmaps are an issue. Dunno, I guess that's just going to need performance testing under real-world conditions... if it all ever works.

yeah it would look spectacular. I probably wouldn't use a glowmap but if you find a good use for them then it's good

[Argh]

I figured out some stuff this weekend. The reason why stuff's wrong is because the camera angle isn't being taken into consideration. The issue is that section of the algorithm dealing with reflectVec. I'd observed in earlier testing that reversing that reversed the apparent angle of the lighting on the normals.

I'd also observed that when the object's side-on to the light, the apparent level of specularity was wrong- i.e., one side would match the other, which is obviously wrong.

Should be easy to pass the vector to the second portion of the shader, but I'm wondering if it needs to be corrected in terms of the position of the vertex or not. Anybody have any thoughts on that?

Also... I've read a little bit more about this stuff, seems there's more than one way to find the correct TBN matrix. The way I figured out will probably look wrong when we're talking about anything that needs to be smoothly shaded, I dunno yet.

And lastly... my current algorithm requires that a sun light be created and positioned. I figure if I can put the sun far enough away, it will be practically the same as a simple directional light, which I don't see any support for in the Spring OpenGL documentation (correct me if I'm wrong- I looked at the gl.Light pname variables, and did not see a DIRECTIONAL available. At any rate, I would like to know if there is a way to use the vector for the sun to position that light in way that's "correct enough" that it looks almost identical to the spring map lighting (I figure if I put the light a few million units out, it'll only come up wrong if the sun vector is practically straight down...

[Tobi]

Argh, most the comments aren't bullshit. People are just pointing out that the math behind your current shader is just plain wrong, so it can never give true normalmapping/bumpmapping: at best an approximation that only works in certain camera/model/part orientations.

Take a closer look at your TBN matrix for a start, and realize what it should actually do: transform the light vector from world space (if your light is in world space?) to tangent space.

This should be an invertible transformation AFAIK so TBN should always have an inverse. Yours hasn't for any geometry which has N.z = 0.

Also the vectors from which the matrix is constructed should be orthonormal basis for the tangent space. Yours aren't unless the geometry lies in the XY plane... (Check dotproducts T * B, T * N and B * N, they should all be 0. Since N is nonconstant and T and B are constant in your case they can not all be 0: they aren't when geometry lies in anything but the XY plane..)

Here are some links a quick google on "TBN" turned up that seem to be OK'ish wrt the math:
http://www.blacksmith-studios.dk/projec ... ing_cg.php
http://www.blacksmith-studios.dk/projec ... vation.php

Keep up the good work tho, but do realize the math isn't easy

[Argh]

the math behind your current shader is just plain wrong

They don't have all of the math, Tobi. There's a reason why I quit posting source after I said I got the lighting working

I keep telling you guys... the shading works- the amount of light per fragment is correct, all parts, all angles. It works no matter where I put the lightsource.

Light is at -10000,-100000,-10000:


You can turn a model, and it interacts correctly with the lightsource, the pieces change lighting based on lighting, etc., etc.. That part is working.

What's not working yet, but will hopefully be working when I can puzzle it out... is deriving the proper value for reflectVec, which is the final piece of the puzzle. That's where it's borked (well, aside from the clipping issues).

The equations that currently govern reflectVec's value (and the result for specularity, which is what needs to be altered properly) are:

normals = normalize((normals - 0.5) * 2.0);
vec3 reflectVec = reflect(-lightVec, normals);
float spec = max(dot(reflectVec, viewVec), 0.0);

That last formula is where things are going wrong, and I think it's because of viewVec. viewVec is defined in the fragment shader as:

viewVec = normalize(gl_Vertex.xyz);

I am pretty sure this is where it's going wrong. Sometimes the result is correct, sometimes it's the inverse. Inverting the value of lightVec inverts the final values.

[jK]

this just shows me that aren't able to parse the modelfile youself to get a list of all vertices and texcoords, but those are must-haves to get a TBN-matrix. And no, it isn't enough to read them in a shader, they need to be computed per face, so as long as you don't use a geo shader you have to do all that math on the cpu.

[Argh]

1. Why is asking for such a basic graphical function call a big deal? Practically any game engine worth using supports arbitrary loading of some mesh format. Spring doesn't- er, it does, for people who've written loaders themselves. Not exactly ideal.

2. Now that I've read a lot more about the TBN matrix...

A. Spring is doing it somewhere. In fact, it's in jcnossen's SM3 implementation. Why isn't it available?

B. Most applications have a special call that passes this back during rendering, so that people can make shaders more easily for arbitrary meshes. Or just assume that you want it rendered that way. For an example... this is how it's done in Blender's game engine.

Hell, DirectX has it built in to its shader specification- one line of code, you're done.

It really pisses me off that GLSL expects you to do this on the CPU, and yet we can't do that without writing something to read triangles we already have in memory somewhere. Waste of time and computational cycles.

In short... this is a much bigger mess than it should be.

At any rate, I have it almost to the point of "unless you're looking really close, it'll pass". I'm still working out a way to deal with the reversed sides of mirrored objects and other small problems. gl_Vertex is apparently returning values in worldspace; I'm still figuring out how to tackle relative-rotation stuff, and then it'll work all right, I think.



I've read a bit more... all we actually need is a way to feed the triangles into a TBN matrix calculator (preferably the one already built into Spring) then put that call in the gl.BeginEnd loop. The only problem is that you can't use gl.Unit like that. It's a black box, from this end.

[Argh]

Here's how Blender is handling this issue, I think.

The only thing missing, from my very cursory read over this, is code to handle loading the mesh based on its current state. A S3O is just static meshes being rotated and transformed- even doing it twice (i.e., letting the current "buried" code do its thing too), is not that big of a deal. Not ideal, yeah... but the cost savings by doing the actual rendering job on the GPU should make up for it.

I'll try reading up on how we arrive at final mesh positions in Spring- hopefully it's a final state, and not just a collection of pieces, but it's probably the latter.

[AF]

Argh its not that GLSL is missing a vital component and thus wasting cycles, its that this work is done anyway and directx etc also waste these cycles as you call, its just GLSL gives you the choice of how you want it to work rather than it being locked away in an API, that way you dont end up taking the performance hit if you don't want to.

[Argh]

The problem, AF, is not on that end. It's that gl.Unit provides us with vertices that have been transformed, but does not allow us access to the resulting verts.

What I'm seeing here is that I get triangles, which I can light correctly, because that doesn't require the TBN matrix, but I don't have any way to do the TBN matrix, period- there is no way to load up the S3O as a raw mesh, so I'm missing data.

UnitRaw was, I assume, meant to do that, but it doesn't work- using it results in a horrible jumble of polygons that clearly have normals reversed, etc., at least over here. Or maybe it was meant for something else- it was clear that Trepan was trying to get this done before he stopped working on the whole Materials area.

I'm not entirely certain that we're going to see high enough speeds (or a broad enough level of compatibility) on MD5 rendering for it to be practical for a big project for some time. Otherwise I'd "jump ship" and bet on Blender for development of art assets for P.U.R.E. But I think that's very premature, given that we haven't seen a demo yet to look at performance, etc.... and that until then I'm going to need to continue to try and squeeze as much out of S3O as possible.

We need a way to request the current state of a given S3O within a glBeginEnd(), or a way to load up the meshes that make up a S3O and create display lists from them, which then we can perform the piece transforms and TBN matrix CPU-side. Wasteful to do the transforms twice, but what the hell, the cost savings on using a low-resolution fallback texture for people who have older hardware is probably going to make up for it many times over.

As discussed here, we have most of the data we need to perform lighting calculations, and can, as jK has demonstrated with his S3O / GLSL rendering project, do a lot of shaders, so long as it doesn't require a TBN matix.

Now that I know how to do basic shader operations and build a lightsource that works, for example, I'm very tempted to do "real-time lighting" where instead of dropping the RGB value, I drop the A value based on light values, giving a fast fake lighting effect. Maybe I'll try that as my next project with this.

So, all of this bitching and moaning aside, I've learned a lot of useful things from my crash-course in GLSL. Besides the fact that, yes, Kloot is right, as usual. I apologize for being so grumpy, people, but this was something that I had to learn the hard way, and I get pretty twitchy when I waste a week on something. That said, this hasn't been a waste, exactly- I have a "bumpmap", where the per-triangle lighting works all right, and the minor issues with the normalmap won't be that visible at a distance. If I could just, say, be able to make the base Unit invisible unless the GLSL shader failed... then I could go ahead and proceed with the rest of this, and fix it later on if we can ever do the TBN matrix.

[Argh]

Ok, I've converted it to a simple real-time lighting shader, for explosions and other effects. Might as well get something useful out of this.

Still working on some elements, but I'll make some code available when it's working satisfactorily.