changing CSMFGroundTextures

[enetheru]

Hi,

I've been working on removing the constant SMALL_TILE_SIZE and have the map loader load tiles of different sizes, but I've run into a problem, spring crashes on pbo.UnmapBuffer() at CSFGroundTextures.cpp:497

https://github.com/enetheru/spring/blob ... xtures.cpp

You can see the changes I've made in that link to both CSMFLoadTiles and CSMFExtractSquareTextures and parent class.

I've double, triple, quadruple checked my shit but I cant see where I'm going wrong. the old code was a bit weird in the way it calculated its byte offsets and data sizes.

anyway plz hlp.

[enetheru]

nvm, found my bug, was treating an array as a type char instead of as a type int.

[jK]

Why do you want to increase the diffuse tex resolution?
SSMF already shows that you need a highres detail tex with splatting and not a high detail diffuse tex.

[jK]

nvm, found my bug, was treating an array as a type char instead of as a type int.

That\s not the only bug ...

[enetheru]

nvm, found my bug, was treating an array as a type char instead of as a type int.

That\s not the only bug ...

indeed

Why do you want to increase the diffuse tex resolution?
SSMF already shows that you need a highres detail tex with splatting and not a high detail diffuse tex.

I don't, in fact I wouldn't mind reducing the diffuse texture size. but that's not the purpose of this.

I'm working towards more generic tile based code. possibly in the future tile rotation.

And in general I've been finding that I would like to work in the game's native tile resoultion, 1024x1024. if I re-render a part of my diffuse texture in blender, and want to replace that portion in the smt. easy. instead of either generating the whole thing again, or writing some code to update that portion.. i can fix it for the general case.

I also want to start working with tileset like images ala japanese 8 bit RPG. I can make my smt 128x128 or 256x256 tiles, push the source Images that i have directly to it and then build my tilemap from them.. no need to force a 32x32 size on map creators when its useful and easy to have it another way.

It allows easier freedom of choice/expression/style and doesn't effect existing stuff.

[enetheru]

nvm, found my bug, was treating an array as a type char instead of as a type int.

That\s not the only bug ...

The way the original code copies the data to the pbo doesn't make any sense to me.. I've been trying to figure out how the memory is laid out and am just getting frustrated.

Any help in this regard would be mots welcome, ie a point in the right direction for documentation of any sort would be totally amazing

[enetheru]

hehe, sigh.. figured it out. seems interesting..i guess there are pretty good reasons. out of curiosity i would be interested in finding out what they are..

[jK]

And in general I've been finding that I would like to work in the game's native tile resoultion, 1024x1024. if I re-render a part of my diffuse texture in blender, and want to replace that portion in the smt. easy. instead of either generating the whole thing again, or writing some code to update that portion.. i can fix it for the general case.

The way the original code copies the data to the pbo doesn't make any sense to me.. I've been trying to figure out how the memory is laid out and am just getting frustrated.

As you may have noticed yourself already, that's where your logic error is. You never can do what your claimed at top. There are no 512x512 or even 1024x1024 DDS files anywhere in the smt. The smt saves DDS _blocks_ and not the full image.
Spring uses a _very_ advanced technology for the ground texture that was even patented years after Spring was already using it!
The idea is that many areas of a map texture are equal or similar so they can use the same DDS block.
So the smt is an atlas of all used DDS blocks on the map. And for each maptile (512x512 tex, 64x64 heightmap) it saves an index list into that atlas where it should fetch each DDS block from. That's why the memcpy copies such small ranges only, it copies _one_ DDS block.

This way you can have 16x16 maps with <40MB in size, while just their ground texture alone would be 16x16 * (512*512*4byte tex data) * (1:4 dxt compression ratio) * (1.333 for the mipmaps) = 256MB * (1:4) * 1.333 = 85MB.

And that's the reason why you always need a tool as mapconv when you want to make changes to a groundtexture.

[enetheru]

well I've got my own smt creator, I can already make the tiles any size I please :)

When people use the existing tools (mapconv) they don't really create nice tilable terrain, instead they compile the smt's such that each of the 32x32 tiles are unique, the index table has all unique values and hence no savings. mainly because its impractical to do it any other way. sad but true.

In reality the only thing dds specifies is a block size of 4x4.
32x32 is completely arbitrary, why not 64x64, 128x128, etc etc. ( probably a good reason )

1024x1024 is what the engine ends up using for its PBO its in the code:

Code: Select all

CSMFGroundTextures::LoadSquareTexture(int x, int y, int level)
...
	const int numSqBytes = (mipSqSize * mipSqSize) / 2;
...
	pbo.New(numSqBytes);
	ExtractSquareTiles(x, y, level, (GLint*) pbo.MapBuffer());

It's just that the data is laid out in the 32x32 block fashion

grepping through the code it seems that there are quite a few systems tightly coupled such that changing the nature of the terrain texture is pretty difficult.. so instead of doing that.. I working through the other end of the pipline, being able to load maps with arbitrary tile sizes.

It's still going to be essentially the same job.

with the current tools and engine
If i have an asset library where all the texture tiles i have are 256x256
eg. http://lunar.lostgarden.com/labels/free ... 0046043177

I can't easily use an existing tile based map creation program
eg. http://www.mapeditor.org/

Because generating images the size of whats needed to use mapconv eliminates most options.

If I could make an smt with the tiles directly(I can). then my tilemap will reference them directly and they'll show up in the game..

This isn't about filesizes, or gpu memory or whatever.. its about enabling creation.

[jK]

When people use the existing tools (mapconv) they don't really create nice tilable terrain, instead they compile the smt's such that each of the 32x32 tiles are unique, the index table has all unique values and hence no savings. mainly because its impractical to do it any other way. sad but true.

And still mapconv is able to `compress` the map a lot (there are less >50MB 16x16 maps).

... 32x32 ... 4x4 ...
32x32 is completely arbitrary, why not 64x64, 128x128,
...
1024x1024 is what the engine ends up using for its PBO its in the code:
...
If i have an asset library where all the texture tiles i have are 256x256
...

Please don't mix NxN w/o telling what you mean with it.
I.e. there are 32x32 dds blocks in a 512x512pixels maptile square, but there is nothing with 32x32pixels nor 32x32bytes.

1024x1024 is what the engine ends up using for its PBO its in the code:

where does 1024x1024 come from?

32x32 is completely arbitrary, why not 64x64, 128x128, etc etc. ( probably a good reason )

Cause vram is limited and 512x512 was already overkill when the code was written (~64MB vram was standard). And as already said diffuse tex resolution is not a problem, yes, spring even use a very hires one. For details you need texsplatting + detailtex as SSMF does.


PS: I am not against such a change, it's just that it is a bad way to add detail to terrain rendering. All references you bring are 2D which is a totally different thing, and are rendered in a totally different way.

[enetheru]

Please don't mix NxN w/o telling what you mean with it.
I.e. there are 32x32 dds blocks in a 512x512pixels maptile square, but there is nothing with 32x32pixels nor 32x32bytes.

sorry

1024x1024 is what the engine ends up using for its PBO its in the code:
where does 1024x1024 come from?

Code: Select all

	const int mipSqSize = smfMap->bigTexSize >> level;

debugging shows that value is 1024.

[enetheru]

Here i am trying to sleep and i figure out how the data is laid out and how i can make it all work.

I'll have to get on to it after work tomorrow

[enetheru]

PS: I am not against such a change, it's just that it is a bad way to add detail to terrain rendering. All references you bring are 2D which is a totally different thing, and are rendered in a totally different way.

Its just a stylistic choice.

since sleep is evading me..

The idea is to pre-render a set of diffuse tiles, with smaller associated normal, splat distribution etc tiles.

Using these I can build maps in blender quickly, and export a tilemap as a text file. then I can use my tileconv program to build the smt, and stitch together the associated images from the tilemap.

Prefab maps with all SSMF details included.

given gajo has the source files, i'm sure that he could also use it in the scenario editor he's making.

[jK]

I really hate all smf file related code, it's really in a very bad state of maintainability (no comments, not KISS code, ...).

So: after thinking further, I remind that texture aren't per 1x1 map `tile` (it's afaik the name the code uses). Instead textures are accumulated from 2x2 maptiles, that's why maps can't have uneven numbers (no 1x1, 6x7, 5x5 maps etc.). So the actual texture size is 1024x1024px.

I really hate the code -_-

[PicassoCT]

But its working thus far.. so it cant be that bad..

[enetheru]

But its working thus far.. so it cant be that bad..

It was primarily the naming and inconsistency that confused me. I finally built up the mental model clean enough to have a go at it, after many failed, and not so failed attempts.

If you think about the 4x4 dxt1 blocks as being 64bit pixels things start to make sense a little easier.

Now I've got that working, my own tile creation code is broken I believe. that or my tilemap code.. bleh onwards!..

[smoth]

If you ever put out a better format I would remake MY maps and a conversion tool could be written for old maps.

It would not be the first time the community updated old maps

[enetheru]

If you ever put out a better format I would remake MY maps and a conversion tool could be written for old maps.

It would not be the first time the community updated old maps

no need to be hastly.. the "format" is actually pretty good much like ID's MegaTexture but without all the units etc texture data being included.

The only thing I would like to do is to split the smf apart..there's no need to have height/minimap/typemap/tilemap/grassmap/metalmap joined together