mapconv - fixing for linux, and other things
Moderator: Moderators
mapconv - fixing for linux, and other things
Hi guys,
I'm going to fix mapconv for linux. I've already gotten it to compile.
But I'm going to need some help from you to make sure I'm doing the right thing.
Primarily from windows developers to test my changes and make sure they haven't borked it for them.
I've forked it to my repo
https://github.com/enetheru/MapConv.git
I'm going to fix mapconv for linux. I've already gotten it to compile.
But I'm going to need some help from you to make sure I'm doing the right thing.
Primarily from windows developers to test my changes and make sure they haven't borked it for them.
I've forked it to my repo
https://github.com/enetheru/MapConv.git
Re: mapconv - fixing for linux, and other things
yess!! slowly getting used to this code base.
Now have texture image showing up on compiled map, it wasn't there before..
Height is only 8bpp though when i'm using a 16bpp tiff..
Really really need a windows developer to help out on this or I'm just going to mess it up.
Now have texture image showing up on compiled map, it wasn't there before..
Height is only 8bpp though when i'm using a 16bpp tiff..
Really really need a windows developer to help out on this or I'm just going to mess it up.
Re: mapconv - fixing for linux, and other things
I said I would help, I just dont know what you need help with? Shall I keep compiling your repo after every commit to see if it still works under windows?
Also, only 16bit 1 channel intel byte order RAW is supported for 16bit. Adding 16bit tiff or png support would be epic!
Also, only 16bit 1 channel intel byte order RAW is supported for 16bit. Adding 16bit tiff or png support would be epic!
Re: mapconv - fixing for linux, and other things
has GIMP finally 16bit support?
Re: mapconv - fixing for linux, and other things
yes please,Beherith wrote:Shall I keep compiling your repo after every commit to see if it still works under windows?
I don't really have any questions regarding the code yet as I can figure out the purpose of most of it after a while.
I'm going to be refactoring for a bit to clean up its organisation. so yeah I wont know if I break windows compatibility till you tell me so.
I've upgraded tclap to the latest version 1.2.1
My OS uses boost 1.49 which was the cause of compile errors before I fixed them.
What do you use to compile? msvc, mingw?
Is is worth investigating CMake as a build system? ie will you use the project files it spits out or continue to maintain a seperate msvc project file?. if its unlikely to get support I will instead replace the existing makefiles with autotools.
I'm going to rename all the files to lowercase at come point, will that be a problem?
Re: mapconv - fixing for linux, and other things
MSVC all the way for me, I have all flavors from 5 to 11.
Re: mapconv - fixing for linux, and other things
OK this change is going to require a little work.
Basically I've ripped out the existing dxt1 compression stuff because it was making system calls to executables and copying files all over the place.
Now it uses nvtt library builds the raw dxt1 files and removes a bunch of platform specific stuff.
Can probably remove dependency on ddraw, and the extra texcompress binaries. they are duplicate work anyway since nvtt gives you a cmd line compression utility anyway(nvcompress).
I've chucked the nvtt download in the libs dir for you to use. not sure what changes are needed.
Basically I've ripped out the existing dxt1 compression stuff because it was making system calls to executables and copying files all over the place.
Now it uses nvtt library builds the raw dxt1 files and removes a bunch of platform specific stuff.
Can probably remove dependency on ddraw, and the extra texcompress binaries. they are duplicate work anyway since nvtt gives you a cmd line compression utility anyway(nvcompress).
I've chucked the nvtt download in the libs dir for you to use. not sure what changes are needed.
Re: mapconv - fixing for linux, and other things
Done. and saying that I'm now going to proceed to rip out devIL and replace it with OpenImageIO.Beherith wrote:Also, only 16bit 1 channel intel byte order RAW is supported for 16bit. Adding 16bit tiff or png support would be epic!
Re: mapconv - fixing for linux, and other things
There used to be some bugs in DevIL:
Image origins were sometimes flipped for .tga images.
Sometimes, nvtt used to insert 1-bit alpha noise into dxt1 images.
Image origins were sometimes flipped for .tga images.
Sometimes, nvtt used to insert 1-bit alpha noise into dxt1 images.
Re: mapconv - fixing for linux, and other things
sure, devIL is no longer an issue.Beherith wrote:There used to be some bugs in DevIL:
Image origins were sometimes flipped for .tga images.
Sometimes, nvtt used to insert 1-bit alpha noise into dxt1 images.
If that nvtt bug shows up again I will fix it.
Re: mapconv - fixing for linux, and other things
There is also a small issue with nvcompress (i might not be correct here, used to be like this): it used to be CUDA only, which is why nvdxt.exe was retained.
Thanks for all your hard work!
Thanks for all your hard work!
Re: mapconv - fixing for linux, and other things
sure, since I only have onboard intel graphics and its doing the work fine thats no longer a problem.Beherith wrote:There is also a small issue with nvcompress (i might not be correct here, used to be like this): it used to be CUDA only, which is why nvdxt.exe was retained.
Thanks for all your hard work!
I no longer use any external binaries either, so no system calls. its all done using the nvtt lib.
Re: mapconv - fixing for linux, and other things
OK, what can I do to help get it compiling on windows? because unless I can get it on board my efforts aren't going to be merged back into the spring git repo and it will have been a waste of time.
Re: mapconv - fixing for linux, and other things
Could I ask you to run cmake and generate a new VS project file? Would make things a lot easier.
Re: mapconv - fixing for linux, and other things
Crap i was hoping you wouldn't say something like that. There isn't any cmake solution atm. so what i'll do is make one that works on linux.. then see what you can do with itBeherith wrote:Could I ask you to run cmake and generate a new VS project file? Would make things a lot easier.
Re: mapconv - fixing for linux, and other things
So dealing with this mapconv stuff is very enlightening.
The format is actually pretty flexible even if horribly designed.
in the smf there is a tilemap, which is just a bunch of indexes to the tile file.
so conceivably, you could create a "tileset" of images to cover the most common terrain transitions and directions, then just reference them.
Technically massive maps could be created with very little data.
The previous things I had seen is an attempt to auto detect similar tiles, discard one and reference the first. but its kind of a dodgy solution.
I could imagine a map editor now that uses tiles exclusively. you could make incredible things.. of course ssmf stuff doesnt mix well with this paradigm.
interesting to get into the nuts and bolts of the code.. i'll see about writing stuff in the wiki about it sometime.
this really lends itself to "tilesets" and multiple tilesets could be shared between lots of different maps. and auto generated maps would be much faster/less data.
still a lot of things to figure out.
The format is actually pretty flexible even if horribly designed.
in the smf there is a tilemap, which is just a bunch of indexes to the tile file.
so conceivably, you could create a "tileset" of images to cover the most common terrain transitions and directions, then just reference them.
Technically massive maps could be created with very little data.
The previous things I had seen is an attempt to auto detect similar tiles, discard one and reference the first. but its kind of a dodgy solution.
I could imagine a map editor now that uses tiles exclusively. you could make incredible things.. of course ssmf stuff doesnt mix well with this paradigm.
interesting to get into the nuts and bolts of the code.. i'll see about writing stuff in the wiki about it sometime.
this really lends itself to "tilesets" and multiple tilesets could be shared between lots of different maps. and auto generated maps would be much faster/less data.
still a lot of things to figure out.
Re: mapconv - fixing for linux, and other things
i had to get up and walk around...
so i'm thinking that a hybrid approach is best. primarily tiles based terrain with sections that are custom built.
could also build the ssmf splat maps using this approach.
gah.. paradigm shifts are a nuisance.. here i was just refactoring and WHACK... BAM... SPLAT... ideas
so i'm thinking that a hybrid approach is best. primarily tiles based terrain with sections that are custom built.
could also build the ssmf splat maps using this approach.
gah.. paradigm shifts are a nuisance.. here i was just refactoring and WHACK... BAM... SPLAT... ideas
Re: mapconv - fixing for linux, and other things
It's even possible to link multiple smt.enetheru wrote:this really lends itself to "tilesets" and multiple tilesets could be shared between lots of different maps. and auto generated maps would be much faster/less data.
PS: In the past someone got the idea to add a rotation index map for the tileset, so each square wouldn't just be an index to a dxt block in the smt, it would also allow to rotate it. This way even more dxt blocks could be merged and so reduce the smt filesize even more.
I always wondered how much it would really help.
Re: mapconv - fixing for linux, and other things
I also toyed with your ideas when I was playing around with mapconv.
You will often run into problems with large-scale gradients, especially on maps that have baked shadows, baked lighting and baked ambient occlusion. I can detect most of the compression artifacts produced by the tiling compression, which is why I default it to 0.
The time where one could produce an excellent looking map by just covering different areas with different tiling textures without baking a lot of lighting is long past.
The file size overhead is becoming negligible, as is the increased memory usage.
You will often run into problems with large-scale gradients, especially on maps that have baked shadows, baked lighting and baked ambient occlusion. I can detect most of the compression artifacts produced by the tiling compression, which is why I default it to 0.
The time where one could produce an excellent looking map by just covering different areas with different tiling textures without baking a lot of lighting is long past.
The file size overhead is becoming negligible, as is the increased memory usage.