ROAM terrain rendering, help us make spring much faster! - Page 4

ROAM terrain rendering, help us make spring much faster!

Discuss the source code and development of Spring Engine in general from a technical point of view. Patches go here too.

Moderator: Moderators

Post Reply
User avatar
Argh
Posts: 10920
Joined: 21 Feb 2005, 03:38

Re: ROAM terrain rendering, help us make spring much faster!

Post by Argh »

Sorry, I didn't really report on the rest.

1. It's faster, at least on my hardware- that's a hell of an achievement, gj!

2. Other than shadows, no serious bugs thus far.
User avatar
Beherith
Posts: 5145
Joined: 26 Oct 2007, 16:21

Re: ROAM terrain rendering, help us make spring much faster!

Post by Beherith »

Yes, as of yet, only terrain self shadows seem to work, ill work on it more later.
Auswaschbar
Spring Developer
Posts: 1254
Joined: 24 Jun 2007, 08:34

Re: ROAM terrain rendering, help us make spring much faster!

Post by Auswaschbar »

Tested with map FolsomDamn, the edges of the dam are broken (sawteeth-like).
User avatar
Dragon45
Posts: 2883
Joined: 16 Aug 2004, 04:36

Re: ROAM terrain rendering, help us make spring much faster!

Post by Dragon45 »

cliffstretch is annoooyyyinngggg btw

just throwin that out there, my #1 complaint with spring terrain evar.
xyz
Posts: 152
Joined: 29 Nov 2008, 16:06

Re: ROAM terrain rendering, help us make spring much faster!

Post by xyz »

Beherith wrote:Left a huge bug in shadow code, causing me to render shadows 5 times per frame. Now shadows work, and are super fast, and no flicker either.
Git repo updated, I can supply a test exe if anyone wants to try without compiling.
Hi, nice work, but for me the shadows still flicker when I move the camera around, or change the view radius.
zerver
Spring Developer
Posts: 1358
Joined: 16 Dec 2006, 20:59

Re: ROAM terrain rendering, help us make spring much faster!

Post by zerver »

Auswaschbar wrote:Tested with map FolsomDamn, the edges of the dam are broken (sawteeth-like).
DAMn it :mrgreen:
User avatar
Beherith
Posts: 5145
Joined: 26 Oct 2007, 16:21

Re: ROAM terrain rendering, help us make spring much faster!

Post by Beherith »

xyz wrote:Hi, nice work, but for me the shadows still flicker when I move the camera around, or change the view radius.
Found the cause: Shadows are rendered before terrain!
Can anyone tell me why this is so?
User avatar
Argh
Posts: 10920
Joined: 21 Feb 2005, 03:38

Re: ROAM terrain rendering, help us make spring much faster!

Post by Argh »

Uh... because... uh...

No, can't think of a single good reason for that. Ideally, it'd render terrain and all Unit display lists in a single pass, to create the shadowmap within the frustum, instead of how it currently operates, where it renders the map's shadowmap, then renders each Unit to the shadowmap separately.
Kloot
Spring Developer
Posts: 1867
Joined: 08 Oct 2006, 16:58

Re: ROAM terrain rendering, help us make spring much faster!

Post by Kloot »

No, can't think of a single good reason for that.
Then there can't possibly be one, right? *ROLLEYES*
Can anyone tell me why this is so?
Efficiency. It lets the shadow pass be combined with the reflection pass, which saves a number of expensive (slow) context switches in between.
User avatar
Argh
Posts: 10920
Joined: 21 Feb 2005, 03:38

Re: ROAM terrain rendering, help us make spring much faster!

Post by Argh »

Reflections should include the shadows, though, if we're talking about Spring's current, render-everything approach to building a reflection cube.

Moreover, I totally fail to see how rendering every single Unit display list, one at a time, running them through ARB shader passes, is more efficient than putting all of them into one big display list and running a single ARB over the resulting geometry.

Not that I'm really complaining overmuch- shadowmap performance on this engine really isn't that bad. Reflections, on the other hand, are terrible, and need serious optimization. It's not ideal that practically everything is rendered to the cubemap, and that game developers have no control over this, when it has such huge performance implications. It's really sad that the only control we have is control over the cubemap's viewport size, which hardly makes an impact, rather than being able to have direct control over whether things get rendered to the cubemap at all.

But you did remind me, Kloot, that I was going to try rewriting the "non-reflective" ARB to just KIL, and see what happens to the speed. Probably not much- the problem isn't the shader, it's the design engine-side.

Anyhow, excuse the derail, my apologies.

[EDIT]One other idea occurs to me, though. Render the Unit geometry's stencil to a FBO, then use that to do all the Unit shadowmaps to the map (and other Units) and do self-shadowing as part of the Unit's rendering process, once again, treating the final positions of the Pieces as a single display list, so that floating and other problems related to z-order are perhaps mitigated (if you're in any doubt that z-order is partially causing the observed problems, I can make screenshots of that, too). Then we'd finally have transparent shadows working again, because that's essentially how the trees work... they just use a simple billboard as the stencil, but whatever.[/EDIT]

[EDIT2]Lastly, just for reference... we are sending every single Piece through this, with all of the setup required:

Code: Select all

!!ARBvp1.0
# vertex program for generating the shadow buffer

ATTRIB pos = vertex.position;
PARAM mat[4] = { state.matrix.projection };
PARAM mat2[4] = { state.matrix.modelview };
OUTPUT opos = result.position;
TEMP temp,temp2;

DP4 temp.x, pos, mat2[0];
DP4 temp.y, pos, mat2[1];

ABS temp2,temp;
ADD temp2,temp2,program.env[17];
RSQ temp2.x, temp2.x;
RSQ temp2.y, temp2.y;
ADD temp2,temp2,program.env[18];
MAD temp, temp, temp2,program.env[16];

DP4 temp.z, pos, mat2[2];
DP4 temp.w, pos, mat2[3];

DP4 opos.x, temp, mat[0];
DP4 opos.y, temp, mat[1];
DP4 opos.z, temp, mat[2];
DP4 opos.w, temp, mat[3];

#need texture coord for alpha testing (to shadow masked texels correctly)
MOV result.texcoord[0], vertex.texcoord[0];

END
We should not be doing that. Write the Piece geometry to one big display list, then run that over the entirety when it's finally assembled.

Do I need to also mention that this isn't even the only ARB shader involved in every shadow volume construction pass? That we're also doing the same thing for every weapon projectile that has geometry assigned, etc.? Think about how many computational cycles we can save, if we consolidate down to one piece of geometry, all items included, then run one ARB...[/EDIT2]
User avatar
Beherith
Posts: 5145
Joined: 26 Oct 2007, 16:21

Re: ROAM terrain rendering, help us make spring much faster!

Post by Beherith »

Argh, if what you say is so significant, why not code it yourself? At the very least, do what I did, and fork spring, fool with the code, and hope that your solution is better than current.
As I dive deeper and deeper into the code of spring, I find that is is quite logically constructed, and not all that hard to understand.
User avatar
Argh
Posts: 10920
Joined: 21 Feb 2005, 03:38

Re: ROAM terrain rendering, help us make spring much faster!

Post by Argh »

Largely because I'm still building a video game, and that requires my time be spent on other things, like P.O.P.S., where I can get around the engine limitations in a shorter timeframe without arguing with people.
User avatar
AF
AI Developer
Posts: 20687
Joined: 14 Sep 2004, 11:32

Re: ROAM terrain rendering, help us make spring much faster!

Post by AF »

I you just did it then you wouldnt have to argue with people as you'd have hard evidence to back you up. People would thus have no choice but to either accept your improvements or code their own
User avatar
clericvash
Posts: 1394
Joined: 05 Oct 2004, 01:05

Re: ROAM terrain rendering, help us make spring much faster!

Post by clericvash »

AF wrote:I you just did it then you wouldnt have to argue with people as you'd have hard evidence to back you up. People would thus have no choice but to either accept your improvements or code their own
Exactly, and rather than do a work-around which can break easily when someone alters springs code, again as AF said, just look into it yourself.
User avatar
Argh
Posts: 10920
Joined: 21 Feb 2005, 03:38

Re: ROAM terrain rendering, help us make spring much faster!

Post by Argh »

Sorry, it's not that simple, guys. I have multiple time constraints here. P.O.P.S. was the blue-sky project this time, Unit shaders and normalmaps were the practical content project, because most of the tech was working when I released the last version of P.U.R.E..

I am bitchy about the shadowmaps because I want to release the floating-world style maps and work more on that project, so that mappers can be free to develop maps with whatever geometry they want. Amongst other things, I think I solved the performance issues- P.O.P.S. has an LOS-in-the-shader solution that should work, giving us true LOS over the map, etc. That, and the shadowmaps drive me nuts, from an aesthetic POV- the distortions over roads, etc., are horrible.

Lastly, it's not that I haven't seriously considered it lately. But 90% of the things I would want to do, if I decided to work on the engine, are OpenGL 2.x+. I see very little point in working on the engine, until we arrive at a strategy for either transitioning the engine to 2.x+, or agreeing on (and seeing active interest in) building a new, optional architecture specifically to address rendering and basically quit trying to improve things that can only be pushed so far, with 1.x spec.
User avatar
Argh
Posts: 10920
Joined: 21 Feb 2005, 03:38

Re: ROAM terrain rendering, help us make spring much faster!

Post by Argh »

Ok, I still don't have time to deal with the entire mess which is shadowmaps, but I have figured out a few things.

1. We're not drawing the scene properly from the light's POV, and the camera should not be used the way it is. What we're doing is very irregular, in fact.

Read this article, it explains nice and concisely what we're supposed to be doing, and includes full source.

And if we would rather have the final shadowmap in GLSL...

My guess, and it's just a guess, is that SJ did his funky stuff with the frustum (and a whole lotta fudge-factors, which have to be seen to be believed) because he couldn't figure out how to use clipping planes or other techniques for getting rid of geometry, so he just centered things on the camera, expanded a bit past the frustum, and generated a result... which is massively distorted. But it "worked", and nobody questioned it a lot, because so few people could even run shadows at all then.


2. I've tested, re-tested, and tested again, and I am now quite sure that the shadowmap distortion is because the projection is generated from the wrong position, using the wrong math, and the results are incorrect, because they weren't created correctly- the depth is wrong, and the position of objects relative to the lightsource isn't being done correctly.

It's not a problem in Unit shaders. It's not a problem with Unit rendering, period.

First, obvious fudge factors:

Code: Select all

void CShadowHandler::GetShadowMapSizeFactors(float& p17, float& p18)
{
if (shadowMapSize == 2048) {
p17 = 0.01f;
p18 = -0.1f;
} else {
p17 = 0.0025f;
p18 = -0.05f;
}
}

Code: Select all

 //if someone could figure out how the frustum and nonlinear shadow transform really works (and not use the SJan trial and error method)
//so that we can skip this sort of fudge factors it would be good
float borderSize=270;
float maxSize=gu->viewRange*0.75f;
if(shadowMapSize==1024){
borderSize*=1.5f;
maxSize*=1.2f;
} 
Look at how it's using constants here, where it should be adjusted dynamically.

Code: Select all

float3 sundir=mapInfo->light.sunDir;
cross1=(sundir.cross(UpVector)).ANormalize();
cross2=cross1.cross(sundir);
centerPos=camera->pos;
// centerPos.x=((int)((centerPos.x+4)/8))*8;
// centerPos.y=((int)((centerPos.y+4)/8))*8;
// centerPos.z=((int)((centerPos.z+4)/8))*8;
// centerPos.y=(ground->GetHeight(centerPos.x,centerPos.z));
 
CalcMinMaxView();
 
//it should be possible to tweak a bit more shadow map resolution from this
float maxLength=12000;
float maxLengthX=(x2-x1)*1.5f;
float maxLengthY=(y2-y1)*1.5f;
xmid=1-(sqrt(fabs(x2))/(sqrt(fabs(x2))+sqrt(fabs(x1))));
ymid=1-(sqrt(fabs(y2))/(sqrt(fabs(y2))+sqrt(fabs(y1))));
//logOutput.Print("%.0f %.0f %.2f %.0f",y1,y2,ymid,maxLengthY);
 
shadowMatrix[0]=cross1.x/maxLengthX;
shadowMatrix[4]=cross1.y/maxLengthX;
shadowMatrix[8]=cross1.z/maxLengthX;
shadowMatrix[12]=-(cross1.dot(centerPos))/maxLengthX;
shadowMatrix[1]=cross2.x/maxLengthY;
shadowMatrix[5]=cross2.y/maxLengthY;
shadowMatrix[9]=cross2.z/maxLengthY;
shadowMatrix[13]=-(cross2.dot(centerPos))/maxLengthY;
shadowMatrix[2]=-sundir.x/maxLength;
shadowMatrix[6]=-sundir.y/maxLength;
shadowMatrix[10]=-sundir.z/maxLength;
shadowMatrix[14]=(centerPos.x*sundir.x+centerPos.z*sundir.z)/maxLength+0.5f;
glLoadMatrixf(shadowMatrix.m);
glProgramEnvParameter4fARB(GL_VERTEX_PROGRAM_ARB,16, xmid,ymid,0,0); //these registers should not be overwritten 
If you look at where all this is derived, you'll see why it's labeled, "fudge factor". maxLength is some arbitrary constant. Note that nothing like this is in Paul's Project, which I tested, and it works great.

Code: Select all

  for(fli=left.begin();fli!=left.end();fli++){
for(fli2=left.begin();fli2!=left.end();fli2++){
if(fli==fli2)
continue;
float colz=0;
if(fli->dir-fli2->dir==0)
continue;
colz=-(fli->base-fli2->base)/(fli->dir-fli2->dir);
if(fli2->left*(fli->dir-fli2->dir)>0){
if(colz>fli->minz && colz<gs->mapy*SQUARE_SIZE+20000)
fli->minz=colz;
} else {
if(colz<fli->maxz && colz>-20000)
fli->maxz=colz;
}
}
}
Here, I was able to confirm why, with a 4096 shadowmap, it's still really aliased at any distance. Look at how much map it's re-drawing, which is especially sad, considering that if the POV hasn't moved much between updates, it's not even necessary- a 4096 square would only require updates (of the map shadow and Features, not Units, obviously) every render pass if user is moving fairly rapidly or is a long, long way up, in fact. I'd have to say... cut the size down to 1.5 max shadowmap viewport resolution for building the shadow, and move that frame with the camera (which SJ's code does... but not the right way).

The very few situations where you're going to see a problem (really huge mountains, low light sources)... mappers can just deal with that limitation, it's worth it for crisp shadows, let alone what it's going to take to do some blurring.
User avatar
Caydr
Omnidouche
Posts: 7179
Joined: 16 Oct 2004, 19:40

Re: ROAM terrain rendering, help us make spring much faster!

Post by Caydr »

Argh, you test, test again, post, re-test, search through code to find something that's inefficient, post, and you don't have enough time to do anything but work on your game.

My method of developing games (trolling forums and playing Jedi Knight) appears to be at least as efficient, if not more enjoyable.
IMSabbel
Posts: 747
Joined: 30 Jul 2005, 13:29

Re: ROAM terrain rendering, help us make spring much faster!

Post by IMSabbel »

One heretic thought:One might even be thinking that an engine is part of a game, and thus "making a game" might include "working on that engine".

But otoh, somebody has a serious opinion of entitlement (see the other thread, where he whines around that AI coders dont spend enough of their free time to make AIs for his commercial mod^H^H^H game)
User avatar
smoth
Posts: 22309
Joined: 13 Jan 2005, 00:46

Re: ROAM terrain rendering, help us make spring much faster!

Post by smoth »

BUMP!
User avatar
Beherith
Posts: 5145
Joined: 26 Oct 2007, 16:21

Re: ROAM terrain rendering, help us make spring much faster!

Post by Beherith »

Atm I cant run spring, and also im stuck with the shadow bug in ROAM:
The terrain self shadows work fine with unit shadows off. Turning unit shadows on totally borks all shadows. Anything but water 0 also borks shadows. I seriously think im not doing anything differently in rendering than the old shadow renderer (i only touched terrain shadows), yet i still cant find the bug.
Post Reply

Return to “Engine”