S3O Shadows

[Argh]

OK, I'm trying to fix S3O shadows, and I'm stumped, thought I'd see if anybody has a clue.

Tried various things to fix the output to the shadow buffer, hit a wall.
Here's the vertex program:

Code: Select all

!!ARBvp1.0
# vertex program for generating the shadow buffer

ATTRIB pos = vertex.position;
PARAM mat[4] = { state.matrix.projection };
PARAM mat2[4] = { state.matrix.modelview };
OUTPUT opos = result.position;
TEMP temp,temp2;

DP4 temp.x, pos, mat2[0];
DP4 temp.y, pos, mat2[1];

ABS temp2,temp;
ADD temp2,temp2,program.env[17];
RSQ temp2.x, temp2.x;
RSQ temp2.y, temp2.y;
ADD temp2,temp2,program.env[18];
MAD temp, temp, temp2,program.env[16];

DP4 temp.z, pos, mat2[2];
DP4 temp.w, pos, mat2[3];

DP4 opos.x, temp, mat[0];
DP4 opos.y, temp, mat[1];
DP4 opos.z, temp, mat[2];
DP4 opos.w, temp, mat[3];

#need texture coord for alpha testing (to shadow masked texels correctly)
MOV result.texcoord[0], vertex.texcoord[0];

END

It's identical to the vertex program for groundshadow.vp, except for that that last line, which appears to be entirely superfluous (i.e., you can drop it and nothing changes). Thought I'd emphasize that, so you know where to start making fun of my ignorance (it seems to work like it always does, though, with that line removed).

I went and checked vs. the tree shadows (which is the obvious model here, although there is an important difference):

The only difference is the end:

Code: Select all

MUL result.color.xyz, vertex.normal.z, program.env[11];
MAD result.color.w, program.env[12].w, pos.y, program.env[11].w;
MOV result.texcoord, vertex.texcoord;

So, I went and checked the source, to see what glProgramEnvParameter4fARBs are being set.

For UnitDrawer, we have something very odd:

Code: Select all

glProgramEnvParameter4fARB(GL_VERTEX_PROGRAM_ARB,10, mapInfo->light.sunDir.x,mapInfo->light.sunDir.y,mapInfo->light.sunDir.z,0);
                glProgramEnvParameter4fARB(GL_VERTEX_PROGRAM_ARB,12, unitAmbientColor.x,unitAmbientColor.y,unitAmbientColor.z,1);
                glProgramEnvParameter4fARB(GL_VERTEX_PROGRAM_ARB,11, unitSunColor.x,unitSunColor.y,unitSunColor.z,0);
                glProgramEnvParameter4fARB(GL_VERTEX_PROGRAM_ARB,13, camera->pos.x, camera->pos.y, camera->pos.z, 0);

I think that's where the problem lies. There doesn't appear to be a variable getting the state of the alpha channel in tex2. Therefore we can't do a meaningful MUL operation in the vertex shader, or some other comparison step that would allow us to chop out the texel values returned because the object is not visible there.

Am I making sense? I'm not quite sure how to address this, but my best guess is that we need to set something like:

GL_VERTEX_PROGRAM_ARB,14, unitTransparentColor.x,unitTransparentColor.y,unitTransparent.z,2

... except that I don't know if that's the right variable name. But doing a MUL step vs. whatever that is called should allow the vertex shader to operate properly- it'll either be 1 or 0.

Heck, if it allowed for non-1 non-0, we'd have true alpha trans and it would effect shadow fragments, which would be neat.

At any rate, I may be entirely wrong about all of this. What I know for sure is that I'm stuck

[jK]

erm what? Alpha transparent shadows aren't doable in a 1 channel rendertarget ...

and no, all your assumptions don't make sense at all. AlphaTest is broken for totaly different reasons ...

PS: I will solve it with introducing GLSL shaders, but don't push me!

[Argh]

PS: I will solve it with introducing GLSL shaders, but don't push me!

Meh, consider yourself not pushed, I'll drop this for now. I suppose there's nowhere else I can go with this, that would be remotely useful?

I'd be grateful if it works sometime in the near future, that's all. It really ruins the presentation of my new content in World Builder.

I know you're having "fun" trying to get other things working right though.

[lurker]

and no, all your assumptions don't make sense at all. AlphaTest is broken for totaly different reasons ...

One sentence summary? Related to the variable cutoff point or something really strange?