A crazy idea about making a new map format.

[Argh]

Basically, it's like this. Over the last two days, I've been offline, but got to hang out with DRB, and we looked at a lot of the latest-greatest RTS games out there.

Even the most advanced engine I saw (LOTR 2) had terrain heightmaps that were quite clearly one-half the resolution of Spring's, and had at least double the resolution in their final bitmaps. Yup, super-fancy engine, shaders of all sorts... and half of Spring's heightmap resolution. In an engine like that, terrain with super-detailed mountains isn't really practical... but tbh, I was impressed with how well that engine performed on DRB's not-uber hardware.

I looked at quite a lot of newer game engines (6), and saw that a lot of these engines were doing splatting like SM3, or used baking approaches and something like the texture loading system used in Spring. I didn't really see anything radically new on that end of things.

However, the striking thing was where the CPU / GPU resources were being used. The engines weren't doing dynamic LOD (a couple of them I saw appeared to use a sector-based approach, where mesh sectors > some distance away were culled), nor were they allowing for mesh manipulation.

IOW, they were stridently avoiding all of the giant, CPU-sucking things in Spring. The fact that every single one I looked at was using half-sized heightmaps... was rather striking, in fact.

Soooo... I'd like to propose a new map format, that will be relatively easy to implement. Here are the features:

1. Map format is based on technology of SMF. This is not a proposal for an entirely new technology.

2. Heightmap, and all features of SMF that pertain to heightmap would be half their current size.

3. Dynamic LOD code and terrain-deformation code would be removed entirely, and calls to it from other functions would just return, if possible.

4. Pathing / LOS algorithms will act upon the heightmap as normal (I would think that it won't "know" that the heightmap it's looking at is bigger / smaller than "normal". This will cause some small borks, but I suspect not very bad. Just treat footprints as being half their usual size, etc.

So... basically, this would be a really stripped-down, purposefully feature-poor SMF. If it can be gotten to the point of working, then we can look at raw speed and see what benefits might be had from going any further.

[smoth]

However, the striking thing was where the CPU / GPU resources were being used. The engines weren't doing dynamic LOD (a couple of them I saw appeared to use a sector-based approach, where mesh sectors > some distance away were culled), nor were they allowing for mesh manipulation.

paging landscapes

2. Heightmap, and all features of SMF that pertain to heightmap would be half their current size.

some of us need little details. I know gundam needs them because I am doing ~70 ft tall ms.

So... basically, this would be a really stripped-down, purposefully feature-poor SMF. If it can be gotten to the point of working, then we can look at raw speed and see what benefits might be had from going any further.

Sounds interesting but it is a lot of work for experimentation. If we had the engine break all the terrain into tiny squares for paging landscapes that would be cool. the only thing I am unsure about is how would it perform when someone like me is using fps cam to look across the map?

[Argh]

Paging landscapes would have to include fog with pretty strict cutoffs, or a user setting to allow you to see farther, that also pushed the fog back, for people with rigs able to handle it. Or just not allow for FPS cam or angles > 45 degrees from straight down.

At any rate, that's an entirely different proposition, and would require a new heightmap renderer. I'm just proposing a test to see how much speed we'd get- all of these commercial devs clearly arrived at the same conclusions somewhat independently, and there's probably a really compelling reason for it.

As for detail... tbh, it just doesn't matter a lot. If it renders as fast and has as good of a CPU benefit as I suspect it will, then mappers can make larger maps with true-scale features, and use other techniques to convey scale and proportion, like what I'm doing with World Builder.

I don't think that this will be that much work, either. We're mainly talking about a new map-format loop... and a cut-down version of SMF with a switch in the pathfinder and LOS algorithms that says, "if using this, then multiply all sizes by 2 and then proceed normally", I think. I may be way off-base there, but I think that's pretty much what this would entail.

[smoth]

I guess but I cannot see this format being useful to me, interesting experiment but not something I would want to use.

[Tobi]

Trading off CPU time for GPU time seems perfectly fine however I'd do it on a per case basis in current SMF instead of introducing a new map format, which is essentially compatible with SMF (or the same even?), but just somewhat faster.

Note for example that SMF file format already has support for arbitrary scale heightmap I think, it's just that Spring is hardcoded to reject anything that doesn't have this paramaters set to their required values:

Code: Select all

        int squareSize;      ///< Distance between vertices. Must be 8
        int texelPerSquare;  ///< Number of texels per square, must be 8 for now
        int tilesize;        ///< Number of texels in a tile, must be 32 for now

(quoted from map format)

Setting squareSize to 16 would have the effect of having half size heightmap and texturemap if it were supported.

Setting squareSize to 16 AND texelPerSquare to 16 would have effect of having only half size heightmap, if it were supported.

[Cremuss]

Maybe a new map format will be an opportunity to make a water heightmap system ?
Also, think about how to manage that about the actual map format. Will the support be deleted and old maps not playable anymore ? etc..

You have a great potential to do it so it's sounds really great :)

[Argh]

Setting squareSize to 16 AND texelPerSquare to 16 would have effect of having only half size heightmap, if it were supported.

So, basically, we just need some tags and very minor changes to try this out?

[jK]

3. .. and terrain-deformation code would be removed entirely, and calls to it from other functions would just return, if possible.

Code: Select all

CMapInfo::ReadGlobal()
...
file.GetDef(disableMapDamage, "0", "GAME\\DisableMapDamage");

Code: Select all

IMapDamage::GetMapDamage()
...
	if (mapInfo->map.notDeformable)
		disable = true;
	else if (gameSetup && gameSetup->disableMapDamage)
		disable = true;

Code: Select all

bool CGameSetup::Init(const char* buf, int size)
...
	file.GetDef(disableMapDamage, "0", "GAME\\DisableMapDamage");

So what does that say us?
First maps can disable terrain deformation, second the host can disable map deformation (just no lobby supports it atm).

[Argh]

Ok, so basically we just need to be able to tweak those hardcoded values Tobi referred to, turn off map deformation at the map level, and voila. Can't jack up the size of the bitmap without something pretty radical involved, but meh, the first thing is to see what kind of real difference there is. What about the on-the-fly LOD system? Is that just a loop in the rendering code that could be bypassed?

[Tobi]

Realize that 'just' tweaking this hardcoded values means replacing this hardcoded values throughout the engine with a variable.

That in itself isn't even too bad (most places a #define SQUARE_SIZE is used), but I'm pretty sure you'll also have to search for the remaining places where 8 is hardcoded, and figure out whether it's ment to be SQUARE_SIZE or texelsPerSquare, or whatever other constant which happens to be 8.

(Also note that changing these hardcoded stuff with variable will mean (slight) performance decrease, because compiler can do less optimizations.)

[AF]

You'd need to modify the AI interfaces lua APIs and unitsync callouts since they all use heightmaps and changing the scales would mean the algorithms that assume a SQUARE_SIZE == 8 would all be wrong and there'd be buffer overflows and crashes left right and center.

[Tobi]

That too

[Pxtl]

Can mods disable terrain deformation? I've been using the brute-force approach of reducing all cratering to zero, but I'd rather have the optimizations available.