Perfect Texture Format (DDS vs TGA vs PNG)

[lurker]

No, that's just compression ratios for stuff that's not being used atm.

What stuff?

Uncompressed it's not the same.

What isn't?

But apparently there's a significant gain because of access times.

What? You're not making any sense.

Meh... I need to go look at Spring again. Are we actually using the equivalent of dwFourCC in DirectX? I know Spring's ripping DDS, but I'm not sure about how it's shoveling that into the GPU.

dwFourCC is a header member saying what format the contents are in; how is that remotely relevant?

[Argh]

Um, I'm asking a serious question here.

DDS just makes all of the mips (using whatever filters you want to apply, per stage) and stores them in the file format, compressed using whatever specification you choose.

They just get ripped out one at a time when Spring loads them. It's unclear to me whether they're still compressed properly when they're transferred to the GPU:

http://spring.clan-sy.com/websvn/filede ... rev=0&sc=0

I mean, how does that work? I see mips being loaded into memory, but I don't really see references to it being loaded onto the GPU. Is that handled automatically by OpenGL? Or am I missing something? I can see that, depending on the format, Spring's reading the DDS and decompressing it, and storing it... but I was under the impression that in DirectX, you just tell DirectX to send the whole DDS to the GPU, which handles that pretty much on its own, or to DirectX or the drivers(sorry, I haven't the foggiest how that works, I've never looked at DirectX in my life).

Also, while I'm on this topic, is the DDS-ripper being used on the maps, for the big bitmaps that store the tiles? I've been wondering that for awhile, since it only supports 5 mipmaps, and my experiment with tiles that are exactly the same size as the tiles and repeat everywhere (Unpleasantville's "grass" surface btw) didn't result in anything like the performance I was hoping for (although that map compressed very nicely).

[lurker]

Um, I'm asking a serious question here.

But you're using pronouns without context, making your question unintelligible to me.

Look how that file uses glCompressedTexImage1DARB; this is directly loading the compressed data.

But let me look at this from the perspective of not knowing what spring does. If it loads DXT textures directly, they take 1/4 the space. If it uncompresses them, they take 1/1 the space. where did you get 1/2?

Edit: The bit I couldn't understand; were you saying that the compression doesn't matter, only the load speed increases from having mips? Then why did you say 1/2 the video ram.

[Argh]

glCompressedTexImage1DARB; this is directly loading the compressed data.

Yes, but then what? It appears from this:

http://books.google.com/books?id=VDqw6M ... t#PPA91,M1

That just for ripping it to 1:1. Not exactly what I'm talking about. DDS isn't meant be ripped out and stored in 1:1- that just makes it a crappy JPG and the only advantage is that the mips are pre-built- we could get far better fidelity with manually-created mips made with PNG.

IOW, if that's all it's actually doing, then we should allow for manually-specified mip levels, and I'll just use 4 for typical game models. Smaller overall VRAM requirements, except under very unusual circumstances.

Edit: The bit I couldn't understand; were you saying that the compression doesn't matter, only the load speed increases from having mips? Then why did you say 1/2 the video ram.

I read that in a Bethesda article, where they were talking about the difference between the compression ratios and the actual size of the texture in VRAM, if the object needing the texture's actually being drawn, IIRC. It's not quite as simple as it initially appears- sometimes you have the main texture uncompressed atm, sometimes it's mip 8, which is tiny. Meh, I don't feel like arguing that point, anyhow- we're both saying, "DDS compresses better", and that's totally beside the point to the OP.

[lurker]

This avoids compression overhead at the time that a texture image is loaded and allows a texture image to be stored using less space.

Where did you get the idea it ripped it out and uncompressed? I see nothing about that in your link.

I'd be quite interested in seeing that article if you can find a link to it.

Still don't know why you were going on about directx headers.

[rattle]

um, I'm telling you, DXT3 and DXT5 don't come out the same way.

test_dds.zip

(61.97 KiB) Downloaded 21 times

I'm pretty sure they do, especially when the artifacts are identical.

[SpliFF]

Argh, your argument about losing color profiles is still incorrect. PNG supports color profiles. At any rate your image software will set the color profile correctly on export. Besides, if you're designing screen graphics in CMYK color space you're doing it wrong. It sounds like an argument for no real purpose because for all practical reasons PNG is lossless. Arguments about performance differences need to be backed up by figures or I simply don't believe them. PNG has no verified drawbacks over TGA or TIF for use in Spring, end of story until you prove otherwise.

I know Spring's ripping DDS, but I'm not sure about how it's shoveling that into the GPU.

with glCompressedTexImage2DARB() (run once for each mip level).

By default all s3o with non-dds texture have mipmaps generated for them:

Code: Select all

CBitmap bm;
	if (!bm.Load(string("unittextures/"+tex1)))
		throw content_error("Could not load S3O texture from file unittextures/" + tex1);
	tex.tex1 = bm.CreateTexture(true);

The true flag passed to CreateTexture enables the mipmapping. Then in the CreateTexture function:

Code: Select all

	if(mipmaps)
	{
		glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MIN_FILTER,GL_LINEAR_MIPMAP_LINEAR);

		glBuildMipmaps(GL_TEXTURE_2D,GL_RGBA8 ,xsize, ysize,GL_RGBA, GL_UNSIGNED_BYTE, mem);
	}

then in glBuildMipmaps:

Code: Select all

void glBuildMipmaps(const GLenum target,GLint internalFormat,const GLsizei width,const GLsizei height,const GLenum format,const GLenum type,const void *data)
{
	if (gu->compressTextures) {
		switch ( internalFormat ) {
			case 4:
			case GL_RGBA8 :
			case GL_RGBA :  internalFormat = GL_COMPRESSED_RGBA_ARB; break;

			case 3:
			case GL_RGB8 :
			case GL_RGB :   internalFormat = GL_COMPRESSED_RGB_ARB; break;

			case GL_LUMINANCE: internalFormat = GL_COMPRESSED_LUMINANCE_ARB; break;
		}
	}

	// create mipmapped texture

/*	if (!gu->atiHacks && glGenerateMipmapEXT_NONGML) { // broken on ATIs and NVs (wait for their OpenGL3.0 drivers :/)
		// newest method
		glTexImage2D(target, 0, internalFormat, width, height, 0, format, type, data);
		glGenerateMipmapEXT(target);
	}else*/
	if (GLEW_VERSION_1_4) {
		// This required GL-1.4
		// instead of using glu, we rely on glTexImage2D to create the Mipmaps.
		glTexParameteri(target, GL_GENERATE_MIPMAP, GL_TRUE);
		glTexImage2D(target, 0, internalFormat, width, height, 0, format, type, data);
	} else
		gluBuild2DMipmaps(target, internalFormat, width, height, format, type, data);
}

In short, as long as gu->compressTextures is true and the user enabled texture compression:



Then ALL textures are compressed (S3TC) and use mip-mapping. If there really are quality differences then yeah, it's the compressor at fault. If your seeing higher than expected vram usage then it's your spring settings.

The actual texture conversion time is going to be dependent on the number of textures and the quality of your drivers. I suspect some of it is done in hardware. Would like to see some actual benchmarks.

Other than that I imagine there's no real drawback in terms of game speed to using PNG,TIF,etc since it all eventually ends up in the same S3TC format (i think).

In my opinion creating mipmaps manually is a wate of time. Others may disagree but I don't see this feature as making DDS vastly superior to PNG. Again this largely depends of the quality of the mip-map generation and compression internal to Spring and OpenGL.

[Argh]

In short, as long as gu->compressTextures is true and the user enabled texture compression

Which is off by default, btw. If it does what you say it does, then it should be on.

However, I just tested it, and it doesn't seem to change anything. My mipmaps still look like when I don't test them. My guess is that it just affects the map renderer.

[lurker]

Yes, but that only applies to textures that aren't already compressed.

Edit for your edit: Doubtful on that theory, and don't the map textures already have mipmaps? Maybe you should use a texture with extreme color variance to compare.

[SpliFF]

Correct, it also only applies to textures that are power of two.

[Argh]

What textures that "aren't compressed"?

I mean, in P.U.R.E., everything that isn't UI is compressed- the map's using DXT1, with only 5 mips, the skins are all DDS. And practically everything is a power-of-two.

The thing that makes me doubt all of this is that switching from DDS 8:8:8:8 resulted in slightly lower quality at half the file size, after some careful filter adjustments on my part, but doesn't seem to have improved real-world performance one bit. The download's smaller, but that's all.

Compared to modern game engines I've seen recently, the performance isn't anything like equivalent, and they're throwing around a lot more geometry (after tessellation) and larger overall loads in terms of texture sizes than I am. At most, in a big fight in a cityscape in P.U.R.E., you're seeing maybe 40 different skins, with sizes ranging from 512^2*2 to 128^2*2. Not that huge of a load, by modern standards. Then maybe call it 2048^2 for map textures, 1024*2 for the CEG texture atlases... it's not that giant.

So, where's the giant advantage coming from? More stuff being done with shaders? It sure isn't something magical about the textures, or amazing optimization of the scenes- Fallout 3 uses DDS, and it's showing far more detailed scenes than Spring even with stuff turned down. And yeah, they use LOD, but I'm pretty sure that on stock settings, you're looking at far higher typical texture loads than Spring, unless you're staring at the ground.

[SpliFF]

What I think he means is that DDS textures (compressed or not) don't go through the process outlined above. The are handled by a seperate nv_dds module which is based on nvidias released code. Since the internal format of DDS is essentially a set of S3TC textures the glCompressedTexImage2DARB function can pass each mip level more or less directly to VRAM. If you're using R8G8B8A8 then there's a good chance it goes straight through to the vram in that format (ie, mostly uncompressed). It will look better but take up more vram.

I've read you lose performance going over 512x512. I haven't confirmed that. Using 1024^2 textures for CEG may be the culprit but I wouldn't rule out other causes.

Most of the modern games you're refering to use combinations of GPU programming (shaders) and ARB/Vendor extensions to sqeeze everything they can out of the hardware. Spring isn't as sophisticated in that respect but it isn't far off.

[jK]

Found an answer to whether DeviL uncompresses DDS:
http://www.ogre3d.org/forums/viewtopic.php?p=100965

They say it still uncompress it into sysmem, but send it compressed to GPU (duno if he fixed it already).

Here's an interesting one, from the Google/Nvidia texture tools page:
http://code.google.com/p/nvidia-texture-tools/

That's proprietary stuff (not meant to be run on runtime), nvidia should (or is?) use it in their drivers for the OpenGL runtime compression.

jK: mipmaps are generated for almost all loaded textures :)

tru (but all model textures are and most lua textures etc.)
So PNG/TGA is still the preferred format for GFX Sprites.

Oh, right. The alpha interpolation is 3 bits because alpha takes less space. So that should just about never look bad compared to a flat 4 bits; comments, jK?

My experience is that DXT5 is fine for (extreme) low frequency in the alpha channel, but if you have hardedges in your alpha channel you will get artifacts on the border of those. That's where DXT3 is just better. (this applies to teamcolor, so I always use DXT3 for tex1)

What I think he means is that DDS textures (compressed or not) don't go through the process outlined above.

yup, DDS don't go through it, that's why DDS8:8:8:8 are totally dumb (except you want specific control on the mipmaps).

[Argh]

That's fine and all- I don't mind trading small amounts of quality for lower filesize on delivery, which is what I'm doing now, and less VRAM footprint. Seems like a fair tradeoff, so long as it doesn't screw up the art too bad, and the settings I finally arrived at seem to cure that problem.

DDS8:8:8:8 are totally dumb except you want specific control on the mipmaps

Which you certainly do, if you're going to use an uncompressed format at all! Letting the drivers / OpenGL do it fails massively, and can and does result in really blurry, nasty results.

[SpliFF]

Found an answer to whether DeviL uncompresses DDS:
http://www.ogre3d.org/forums/viewtopic.php?p=100965

They say it still uncompress it into sysmem, but send it compressed to GPU (duno if he fixed it already).

Yeah, I posted before reading the whole thread. I attempted to track down the actual behaviour but the DevIL documentation is rather vague about it. It would probably be best to keep the current behaviour of treating DDS as a special case. That still leaves a lot of Spring code which essentially reproduces functionality already part of DevIL. There is a lot of room for simplification there.

Here's an interesting one, from the Google/Nvidia texture tools page:
http://code.google.com/p/nvidia-texture-tools/

That's proprietary stuff (not meant to be run on runtime), nvidia should (or is?) use it in their drivers for the OpenGL runtime compression.

No, it's for end-users (ie, developers).
EDIT: Sorry, my bad. I'm confusing the texture tools with the SDK.

Below is the nv_dds header in the Spring source:

Code: Select all

//This file contains source code provided by NVIDIA Corporation
//Modified DDS reader class from NVIDIA SDK
///////////////////////////////////////////////////////////////////////////////
//
// Description:
// 
// Loads DDS images (DXTC1, DXTC3, DXTC5, RGB (888, 888X), and RGBA (8888) are
// supported) for use in OpenGL. Image is flipped when its loaded as DX images
// are stored with different coordinate system. If file has mipmaps and/or 
// cubemaps then these are loaded as well. Volume textures can be loaded as 
// well but they must be uncompressed.

I believe it's more or less taken from Nvidias released code. Unsure under what license it is released. Most sites I found list it as 'Freeware'.

[lurker]

As far as not seeing improvement, this is just on your computer or also testing on computers with little vram?

[Argh]

As far as not seeing improvement, this is just on your computer or also testing on computers with little vram?

Dunno about machines in the really low range for the game (128MB or lower), I don't have any machines with hardware that old (and until RC5.2's out, most of them seem to be in the category that crashes due to GLSL issues, so we'll just have to see).

I do know that as of RC5.2, the load's going to be as low as it's ever going to get, short of a splatting map format that actually works.

I mean, it's not like the final version of P.U.R.E. / World Builder will have less stuff in them than they do now

[SpliFF]

That's fine and all- I don't mind trading small amounts of quality for lower filesize on delivery, which is what I'm doing now, and less VRAM footprint. Seems like a fair tradeoff, so long as it doesn't screw up the art too bad, and the settings I finally arrived at seem to cure that problem.

What settings? Also you do realise there is no actual tradeoff? With 4:1 compression you can increase the size of your texture to offset the (minor) loss of color fidelity.

DDS8:8:8:8 are totally dumb except you want specific control on the mipmaps

Which you certainly do, if you're going to use an uncompressed format at all! Letting the drivers / OpenGL do it fails massively, and can and does result in really blurry, nasty results.

I like to see that claim validated with screenshots before I accept it out of hand.

Here is a screenshot from WP:


"Comparative screenshots of special edition of Unreal Tournament game with high resolution S3TC compressed textures on S3 Savage 4 videocard"

That screenshot doesn't indicate how the compression was performed but it does indicate that using compression gives a net gain in image quality for the same amount of VRAM.

[Argh]

What settings? Also you do realise there is no actual tradeoff? With 4:1 compression you can increase the size of your texture to offset the (minor) loss of color fidelity.

LOL! If it was only that simple

Fillrate becomes a problem, if you do that. It's like making your car go faster by making it weigh half as much, then putting in an engine that's half as powerful. Moreover, that's a very rapid diminishing-returns curve, because fillrate's going to impact every version of that texture anywhere, whereas VRAM issues are static.

I like to see that claim validated with screenshots before I accept it out of hand.

Go compare P.U.R.E. RC3 texture quality vs. RC5. Basically, the tradeoff with going off of DDS 8:8:8:8 was lower download sizes and (perhaps) better performance on marginal computers. The quality of DXT3 is not the same, and using 8:8:8:8 meant that each mipmap was extremely clean. Which meant some muttering and cursing, before I finally settled on settings I think are somewhat satisfactory.

It's very easy to do a comparison test yourself, as well try PNG vs. any DDS, it's obvious which one looks better, unless you have a really uber videocard with AA / Aniso turned way up. To really see the difference, turn both down to zero, it's obvious that 8:8:8:8 beats other lossless formats. But it's huge, and players hate downloading huge... and some players have fairly low-end gear... so quality has to be compromised a bit.

[SpliFF]

No, fillrate becomes a potential problem. Do you have any evidence to prove Spring is fillrate limited? From the link you gave:

"there is a simple trick to detect if graphics are fillrate-limited. To do this, run the game in windowed mode. Also, if not done already, put an FPS (frames per second) counter somewhere in the application window. Now, render a view typical for the game or one desired to benchmark; keep the FPS rate in mind. Now, gradually resize the window down to a size of a few pixels while observing the frame rate. If the FPS skyrockets, the game is fill rate-limited.

I can't test because I'm at work (and supposed to be working). It would be more useful if you would stop putting forward theories and concentrate on actual verifiable (to us, not just yourself) results. Logically if fillrate was such an issue then nobody would implement S3TC. Ipso-facto, you're probably incorrect again. I'm sick of taking your word for things only to discover the information is inaccurate or disputed or like in this case, probably irrelevant. By all means prove me wrong but I've yet to see any good evidence that DXT5 is worse than R8G8B8A8 in quality, speed or anything for the same amount of VRAM usage.

EDIT: PNG is not lossless in Spring - we were talking about it's nature as a file format. Talking about lossless formats in Spring is meaningless because they are being compressed and resized internally.