mapconv - fixing for linux, and other things

[enetheru]

Hi guys,

I'm going to fix mapconv for linux. I've already gotten it to compile.

But I'm going to need some help from you to make sure I'm doing the right thing.

Primarily from windows developers to test my changes and make sure they haven't borked it for them.

I've forked it to my repo
https://github.com/enetheru/MapConv.git

[enetheru]

yess!! slowly getting used to this code base.

Now have texture image showing up on compiled map, it wasn't there before..

Height is only 8bpp though when i'm using a 16bpp tiff..

Really really need a windows developer to help out on this or I'm just going to mess it up.

[Beherith]

I said I would help, I just dont know what you need help with? Shall I keep compiling your repo after every commit to see if it still works under windows?

Also, only 16bit 1 channel intel byte order RAW is supported for 16bit. Adding 16bit tiff or png support would be epic!

[jK]

has GIMP finally 16bit support?

[enetheru]

Shall I keep compiling your repo after every commit to see if it still works under windows?

yes please,

I don't really have any questions regarding the code yet as I can figure out the purpose of most of it after a while.

I'm going to be refactoring for a bit to clean up its organisation. so yeah I wont know if I break windows compatibility till you tell me so.

I've upgraded tclap to the latest version 1.2.1
My OS uses boost 1.49 which was the cause of compile errors before I fixed them.

What do you use to compile? msvc, mingw?

Is is worth investigating CMake as a build system? ie will you use the project files it spits out or continue to maintain a seperate msvc project file?. if its unlikely to get support I will instead replace the existing makefiles with autotools.

I'm going to rename all the files to lowercase at come point, will that be a problem?

[Beherith]

MSVC all the way for me, I have all flavors from 5 to 11.

[enetheru]

OK this change is going to require a little work.

Basically I've ripped out the existing dxt1 compression stuff because it was making system calls to executables and copying files all over the place.

Now it uses nvtt library builds the raw dxt1 files and removes a bunch of platform specific stuff.

Can probably remove dependency on ddraw, and the extra texcompress binaries. they are duplicate work anyway since nvtt gives you a cmd line compression utility anyway(nvcompress).

I've chucked the nvtt download in the libs dir for you to use. not sure what changes are needed.

[enetheru]

Also, only 16bit 1 channel intel byte order RAW is supported for 16bit. Adding 16bit tiff or png support would be epic!

Done. and saying that I'm now going to proceed to rip out devIL and replace it with OpenImageIO.

[Beherith]

There used to be some bugs in DevIL:
Image origins were sometimes flipped for .tga images.
Sometimes, nvtt used to insert 1-bit alpha noise into dxt1 images.

[enetheru]

There used to be some bugs in DevIL:
Image origins were sometimes flipped for .tga images.
Sometimes, nvtt used to insert 1-bit alpha noise into dxt1 images.

sure, devIL is no longer an issue.
If that nvtt bug shows up again I will fix it.

[Beherith]

There is also a small issue with nvcompress (i might not be correct here, used to be like this): it used to be CUDA only, which is why nvdxt.exe was retained.

Thanks for all your hard work!

[enetheru]

There is also a small issue with nvcompress (i might not be correct here, used to be like this): it used to be CUDA only, which is why nvdxt.exe was retained.

Thanks for all your hard work!

sure, since I only have onboard intel graphics and its doing the work fine thats no longer a problem.

I no longer use any external binaries either, so no system calls. its all done using the nvtt lib.

[enetheru]

OK, what can I do to help get it compiling on windows? because unless I can get it on board my efforts aren't going to be merged back into the spring git repo and it will have been a waste of time.

[Beherith]

Could I ask you to run cmake and generate a new VS project file? Would make things a lot easier.

[enetheru]

Could I ask you to run cmake and generate a new VS project file? Would make things a lot easier.

Crap i was hoping you wouldn't say something like that. There isn't any cmake solution atm. so what i'll do is make one that works on linux.. then see what you can do with it

[enetheru]

http://youtu.be/IJfAu0lOt9w

[enetheru]

So dealing with this mapconv stuff is very enlightening.
The format is actually pretty flexible even if horribly designed.

in the smf there is a tilemap, which is just a bunch of indexes to the tile file.

so conceivably, you could create a "tileset" of images to cover the most common terrain transitions and directions, then just reference them.

Technically massive maps could be created with very little data.

The previous things I had seen is an attempt to auto detect similar tiles, discard one and reference the first. but its kind of a dodgy solution.

I could imagine a map editor now that uses tiles exclusively. you could make incredible things.. of course ssmf stuff doesnt mix well with this paradigm.

interesting to get into the nuts and bolts of the code.. i'll see about writing stuff in the wiki about it sometime.

this really lends itself to "tilesets" and multiple tilesets could be shared between lots of different maps. and auto generated maps would be much faster/less data.

still a lot of things to figure out.

[enetheru]

i had to get up and walk around...

so i'm thinking that a hybrid approach is best. primarily tiles based terrain with sections that are custom built.

could also build the ssmf splat maps using this approach.

gah.. paradigm shifts are a nuisance.. here i was just refactoring and WHACK... BAM... SPLAT... ideas

[jK]

this really lends itself to "tilesets" and multiple tilesets could be shared between lots of different maps. and auto generated maps would be much faster/less data.

It's even possible to link multiple smt.

PS: In the past someone got the idea to add a rotation index map for the tileset, so each square wouldn't just be an index to a dxt block in the smt, it would also allow to rotate it. This way even more dxt blocks could be merged and so reduce the smt filesize even more.
I always wondered how much it would really help.

[Beherith]

I also toyed with your ideas when I was playing around with mapconv.

You will often run into problems with large-scale gradients, especially on maps that have baked shadows, baked lighting and baked ambient occlusion. I can detect most of the compression artifacts produced by the tiling compression, which is why I default it to 0.

The time where one could produce an excellent looking map by just covering different areas with different tiling textures without baking a lot of lighting is long past.
The file size overhead is becoming negligible, as is the increased memory usage.