mapconv - fixing for linux, and other things

[enetheru]

The time where one could produce an excellent looking map by just covering different areas with different tiling textures without baking a lot of lighting is long past. The file size overhead is becoming negligible, as is the increased memory usage.

Make a tile take care of all parts of the map, height, texture, ssmf. that way you get your pretty plus tiling.

The way I would work it is to have your tiles pre-rendered, with an associated height, splat dist and splat textures.

use a map editor(bitmap paint program) to create an index of what tiles you want

when you compile the map it builds the height and ssmf maps from the tile resources.. the smt gets built seperately.

you lose a fair amount of flexibility due to the tiling nature, but you gain size of maps..

[Beherith]

Look at the tiles KaiserJ made for FrostRegen's SME.

[enetheru]

Look at the tiles KaiserJ made for FrostRegen's SME.

talking about this? http://springfiles.com/spring/other/hma ... oever-else
That's exactly what I was thinking. still a ways to go before I get close to making it more useful though.

sooo.... an update.

* I've removed dependencies on boost and devIL, dependencies are currently OpenImageIO and NVTT and TCLAP
* I've pulled everything into mapconv.cpp because all those classes and extra files were such a PITA when the programs functionality is so simple. I think there was some over engineering happening.
* if you feed it any files that aren't the correct dimensions it scales using nearest pixel values.
* everything is completely optional, except for --width --length --smf-ofn(output filename), it will generate a valid completely black map, but I want to make it a prettier color.. perhaps grey or green..

Original functionality thats missing, but soon to be re-implemented:
* multi smt
* similar tile detection
* features
* heightmap filtering
* heightmap invert

Thoughts:
* I have to implement some image filtering other than nearest, especially for heightmap.
* Similar tile detection will be fun , some sort of hashing, or histogram etc etc.. I've been reading up on it, but haven't settled on a direction to head.
* splitting out the maptile generation to separate program tileconv
* map decompiling as a separate program.
* turning it into a shared lib, perhaps then we wont need separate programs at all and it can be added to spring proper.
* remembering how to spell separate properly

anyway, yeah. if you have requests or manage to compile and find bugs use this:
https://github.com/enetheru/MapConv/issues

PS: for laughs i compiled a 64x64 and a 256x16 map, and it worked fine. but without similar tile matching, its going to be too huge to be usable.

[enetheru]

what happens when you use nearest pixel when scaling up height maps?
MINECRAFT

[Beherith]

nice work on the minecraftification! seems like sidecycling is all the rage threse days :)

[enetheru]

multiple smt's implemented.

The index in the smf references them in the order they are listed in their relevant header section.

I can imagine some fancy tricks you could use with mapoptions and such to swap out portions of the map, but the effort may well not be worth it for the filesize savings. especially since the tiles really are only the diffuse colour.

Is it possible to reload tiles on the fly with lua? if so ground damage by texture, rather than decal could be an interesting use.

or some combination of height deforming aswell as texture changing.. would suffer from texture 'popping' I think.

[Beherith]

Spring.SetMapSquareTexture

(number texSqrX, number texSqrY, string luaTexName) -> boolean success

Works on tiles.

[FLOZi]

But there are drawbacks to using it on a large scale - ask jk

[gajop]

The drawback is huge GPU memory required for such a method, as textures aren't compressed. You will likely run out of VRAM on big maps.

[enetheru]

The drawback is huge GPU memory required for such a method, as textures aren't compressed. You will likely run out of VRAM on big maps.

Hence why I need a good tile matching algorithm so that we can reduce the texture requirements.

[enetheru]

soo update.

Currently bringing back the feature options, parsing lua files is a PITA without some sort of lib, csv is much easier and simpler.

Some things that cant be done easily is splatting of metal and geovent decals. Because the tiles are cut into 32x32 chunks, anything that is on the boundaries of those tiles will cause issue, not only for the blitting but for tile matching etc. I'll think of a robust solution as its a useful tool to place things onto the map. but its not by any means easy.

example cmd line and output messages

Code: Select all

$ ./mapconv -v --fast-dxt1 -w 16 -l 16 -y height.tif -c diffuse.jpg --metalmap metal.png --minimap minimap.jpg --grassmap grass.png -o charlie_in_the_hills

== Testing Arguments for validity ==
INFO: Typemap unspecified.
INFO: Featurelist unspecified.

== Creating charlie_in_the_hills.smt ==
INFO: Loaded diffuse.jpg (8192, 8192)*3
INFO: Processing tile 65536 of 65536

== Creating charlie_in_the_hills.smf ==
INFO: Adding grass extra header.
INFO: Processing and writing displacement.
INFO: Reading height.tif.
INFO: Processing and writing typemap.
INFO: Generating blank typemap.
INFO: Processing and writing minimap.
INFO: Reading minimap.jpg
WARNING: minimap.jpg is (1024,1024)3, wanted (1024, 1024)4 Resampling.
INFO: Processing and writing maptile index.
INFO: Adding charlie_in_the_hills.smt to smf
INFO: Processing and writing metalmap.
INFO: Reading metal.png
WARNING: metal.png is (512,512)3, wanted (512, 512)1 Resampling.
INFO: Processing and writing grass map.
INFO: Reading grass.png
WARNING: grass.png is (256,256)4, wanted (512, 512)1 Resampling.

[enetheru]

After Tearing my hair out for a couple of days, I finally got bilinear interpolation working when scaling up maps.

Next on my list is tile matching, without this there isn't much point to large maps as the file size is a deal breaker without it.

remember to feel free to lodge feature requests, or bug reports
https://github.com/enetheru/MapConv/issues?state=open

[jK]

...parsing lua files is a PITA without some sort of lib...

??? you execute them! and if adding lua dependencies is too much work, you can simply execute/parse them via unitsync. and yes you can execute arbitrary lua file through the unitsync, not just the expected ones (unitdefs, featuredefs, weapondefs, ...).

[enetheru]

...parsing lua files is a PITA without some sort of lib...

??? you execute them! and if adding lua dependencies is too much work, you can simply execute/parse them via unitsync.

The pre-existing code base parsed them, its a legacy feature that really should be removed in favour of featureplacer.

I'm sure that working on MapConv will be completely redundant as soon as spring proper can pull in images to do all this stuff.

Bet it can already and its just that nobody is using it yet.

[enetheru]

Rudimentary tile matching works, using the 3rd miplevel(4x4 pixels)dds as a 64bit checksum.

It does make a bunch of crappy matches at the moment.

I'm thinking of using this as a preliminary selection phase, then delving deeper to really figure out if a match is worth it.

It makes it easy that tiles are 32x32, there isnt a lot of data to match :D

[Beherith]

is 3rd mip really better than mean squared error of pixel values?

[jK]

is 3rd mip really better than mean squared error of pixel values?

he uses a database, with MSQE you need to compare each tile with all others (except 100% equal ones which can be sorted out before)

[enetheru]

is 3rd mip really better than mean squared error of pixel values?

I gots teh no ideas.. I guess we could calculate it.

It's just that I have the data right there anyway, I just store it for use when comparing.

I can go back and use mean squared error afterwards if wanted. or any other method. this was just the quickest to implement.

[enetheru]

is 3rd mip really better than mean squared error of pixel values?

he uses a database, with MSQE you need to compare each tile with all others (except 100% equal ones which can be sorted out before)

This is actually me:

[enetheru]

decompile works..(no diffuse yet, as that requires reconstructing the image from the tiles and I really cbf tonight).

and with that i'm going to remake another map.