New mapconv that works flawlessy on linux

New mapconv that works flawlessy on linux

Discuss maps & map creation - from concept to execution to the ever elusive release.

Moderator: Moderators

User avatar
tizbac
Posts: 136
Joined: 19 Jun 2008, 14:05

New mapconv that works flawlessy on linux

Post by tizbac » 22 Aug 2011, 16:46

I've created a new mapconv that uses DevIL and it does not store tons of intermediate files , it takes on my system ( intel i7 860 ) , 10 secs to compile 8x8 map.

You have only to care of main texture size that must be a multiple of 1024, all other textures will be rescaled if necessary
Textures can be .exr or .hdr too , this useful for heightfield specially.

it will output the files on the working directory with the specified suffix on -o .

Sources: https://springrts.com/wiki/MapConvNG

You need to install DevIL headers to compile it , then just do cmake . and make

If you have any questions / problem post here
At the time of writing , features are NOT supported



Suggested parameters:
-ct 2 : Compression Type , 1 is no compression at all , 2 - Compresses but comparing with limited set of tiles , 3 - Compares with ALL tiles , it can take ages to compile , you've been warned!

-th 0.03 : Compression Level , Higher compresses more , but adds more artifacts, 0.03 is a good compromise

-smooth : Does a bit of gaussian blur on HDR heightmap , it is especially useful when loading 8 bit images


Edit: With latest version , suggested -ct is 4 and -th is 0.8
Last edited by abma on 25 Sep 2014, 22:33, edited 4 times in total.
0 x

User avatar
koshi
Lobby Developer
Posts: 1058
Joined: 14 Aug 2007, 16:15

Re: New mapconv that works flawlessy on linux

Post by koshi » 22 Aug 2011, 17:10

suggestion: use boost::program_options, tclap or at least gnu getopt in your main
0 x

User avatar
tizbac
Posts: 136
Joined: 19 Jun 2008, 14:05

Re: New mapconv that works flawlessy on linux

Post by tizbac » 22 Aug 2011, 17:20

Thank for the suggestion , i already know and used boost program options , but here i didn't use it on purpose , i want to avoid having too many dependencies :)
0 x

User avatar
hoijui
Former Engine Dev
Posts: 4342
Joined: 22 Sep 2007, 09:51

Re: New mapconv that works flawlessy on linux

Post by hoijui » 22 Aug 2011, 18:10

nice nice! :-)

about program options:
i am pretty sure that the general user of mapconv would never even get the idea to compile it himself.
the people in spring-land that would compile it should usually already have boost, because of spring.
hmm... ok, as it is not a headers-only lib, it kind of makes sense here, not not use that.

how long does it take to compile the same map with the old mapconv on your system?
how does it compare in (peek-)memory usage?
0 x

User avatar
tizbac
Posts: 136
Joined: 19 Jun 2008, 14:05

Re: New mapconv that works flawlessy on linux

Post by tizbac » 22 Aug 2011, 18:25

hoijui wrote:nice nice! :-)

about program options:
i am pretty sure that the general user of mapconv would never even get the idea to compile it himself.
the people in spring-land that would compile it should usually already have boost, because of spring.
hmm... ok, as it is not a headers-only lib, it kind of makes sense here, not not use that.

how long does it take to compile the same map with the old mapconv on your system?
how does it compare in (peek-)memory usage?
I've made it cause i were unable to use mapconv at most , how i can compare it? :P
With 8x8 map it uses about 150MB , with 16x16 450MB, if i save the raw bitmap on disk i could get 20-30% less memory usage , but with the cost of ram memory today , it's not worth doing it
Especially considering that you have edited the texture somewhere so you must have the ram anyway :P
0 x

ljd
Posts: 7
Joined: 20 Aug 2011, 20:21

Re: New mapconv that works flawlessy on linux

Post by ljd » 23 Aug 2011, 21:30

Perfect timing! Just what I needed. I've been struggling to get textures to work with the main branch of mapconv on linux for the past 2 days. Thanks for putting this together, tizbac--it works great. Any plans on adding feature map support?

-ljd
0 x

User avatar
tizbac
Posts: 136
Joined: 19 Jun 2008, 14:05

Re: New mapconv that works flawlessy on linux

Post by tizbac » 23 Aug 2011, 22:35

i can do it in less than 5 hours i think
0 x

User avatar
tizbac
Posts: 136
Joined: 19 Jun 2008, 14:05

Re: New mapconv that works flawlessy on linux

Post by tizbac » 23 Aug 2011, 23:34

Done
to add features use -features [featurefile] parameter
Feature file is a simple text file , example following:
Name XPos YPos ZPos Orientation(Degrees)

Code: Select all

Rock1 6000 -500000 3000 0
Rock2 4000 200 1000 180
Rock1 will be placed on terrain surface
Rock2 will be placed @ 200
0 x

User avatar
tizbac
Posts: 136
Joined: 19 Jun 2008, 14:05

Re: New mapconv that works flawlessy on linux

Post by tizbac » 24 Aug 2011, 00:05

Win32 build, if it is recongnized as virus don't worry , some antiviruses may get alarmed cause executable is packed
Attachments
SpringMapConvNG_1.7z
(810.51 KiB) Downloaded 48 times
0 x

User avatar
Forboding Angel
Evolution RTS Developer
Posts: 14572
Joined: 17 Nov 2005, 02:43

Re: New mapconv that works flawlessy on linux

Post by Forboding Angel » 24 Aug 2011, 05:11

Don't use compression when compiling maps. I won't bother to explain why, just please understand that it's a fairly known unwritten rule among the experienced mappers here.

Also, you sir are awesome. I shall summon AF to get you a cookie.
0 x

ljd
Posts: 7
Joined: 20 Aug 2011, 20:21

Re: New mapconv that works flawlessy on linux

Post by ljd » 24 Aug 2011, 06:25

tizbac, your are amazing! This is great.

Some issues I had (mostly because I learned on mapconv.) I will document here so others don't make the same mistakes:

1) Typemap uses the luminance value instead of the red channel (so edit typemap in grayscale mode)
2) Metalmap and Typemap are expected to be 1/16 the size of the texturemap (versus of 1/8 plus 1 pixel in mapconv)
3) I had to remove the new height clamping code (lines 149-177 in SMFMap.cpp)

Regarding the height clamping; it caused my terrain to "rotate". So my lowest terrain (black) became mid elevation. My terrain just under gray 128 became the highest peaks and terrain above 128 (128-255) became valleys delving below what should have been the lowest ground. Removing the code solved the problem and returned everything to normal.

Some questions:

a) How do I add geothermal vents?
b) How do I use the vegetation map?

Again, tizbac, I really appreciate the work you've done.

-ljd
Last edited by ljd on 24 Aug 2011, 15:39, edited 1 time in total.
0 x

User avatar
tizbac
Posts: 136
Joined: 19 Jun 2008, 14:05

Re: New mapconv that works flawlessy on linux

Post by tizbac » 24 Aug 2011, 10:13

Vegetation map is 0-1 , rgb 0,0,0 and rgb 1,1,1 , it's a bit hard to edit that way , i have to modify it , clamping will be removed.
0 x

User avatar
hoijui
Former Engine Dev
Posts: 4342
Joined: 22 Sep 2007, 09:51

Re: New mapconv that works flawlessy on linux

Post by hoijui » 24 Aug 2011, 10:25

isn't it quite bad to not support the old format, no matter how much better the new one is?
i mean.. there are .. a thousand maps or so, and most of them exist as source files on someones PC, and they would all have to be converted now? and if so, are not compatible with the old mapconv anymore?
and who would change all the tutorials and threads that explain how to create maps?
0 x

User avatar
tizbac
Posts: 136
Joined: 19 Jun 2008, 14:05

Re: New mapconv that works flawlessy on linux

Post by tizbac » 24 Aug 2011, 12:23

ljd wrote: 1) Typemap uses the luminance value instead of the red channel (so edit typemap in grayscale mode)
Typemap on spring it's just 0-255 value matrix , each value in 0.255 range is a terrain type
ljd wrote: 2) Metalmap and Typemap are expected to be 1/16 the size of the texturemap (versus of 1/8 plus 1 pixel in mapconv)
I've used sizes used internally by spring , i have no idea why mapconv takes bigger ones and then rescales

ljd wrote: a) How do I add geothermal vents?
GeoVent as feature name

hoijui wrote:isn't it quite bad to not support the old format, no matter how much better the new one is?
i mean.. there are .. a thousand maps or so, and most of them exist as source files on someones PC, and they would all have to be converted now? and if so, are not compatible with the old mapconv anymore?
and who would change all the tutorials and threads that explain how to create maps?
I'm not going to add the crap from mapconv , i'll write a program to take compatible maps and feature file from smf and smt
0 x

User avatar
tizbac
Posts: 136
Joined: 19 Jun 2008, 14:05

Re: New mapconv that works flawlessy on linux

Post by tizbac » 24 Aug 2011, 12:56

Forboding Angel wrote:Don't use compression when compiling maps. I won't bother to explain why, just please understand that it's a fairly known unwritten rule among the experienced mappers here.

Also, you sir are awesome. I shall summon AF to get you a cookie.
I've tweaked compression so that it creates no artifacts at all, tiles that are equal 100% even if compression is disabled they are reused
0 x

ljd
Posts: 7
Joined: 20 Aug 2011, 20:21

Re: New mapconv that works flawlessy on linux

Post by ljd » 24 Aug 2011, 15:37

tizbac wrote: Typemap on spring it's just 0-255 value matrix , each value in 0.255 range is a terrain type
Right. The old programs used the red channel from RGB colorspace and your program uses the luminance channel from, I assume, LUV colorspace. Either one works, but they are incompatible. I only used the red channel in my typemap written for mapconv, leaving green and blue set to 0. To make it work with your program I had to raise the green and blue channels equal to the red channel (hence, edit in grayscale.) Of course, if you use grayscale typemaps to begin with, they will work with both implementations. However, you eat up all 3 RGB color channels, preventing green and blue channels to be used in the future for other information.
tizbac wrote: I've used sizes used internally by spring , i have no idea why mapconv takes bigger ones and then rescales
I think you made the right decision here. You automatically rescale the incorrect image sizes, so you have achieved both backwards compatibility and a more efficient implementation. Kudos
tizbac wrote: GeoVent as feature name
I tried it this morning. Works great! Thanks.
tizbac wrote: I'm not going to add the crap from mapconv , i'll write a program to take compatible maps and feature file from smf and smt
I think hoijui's point is when a map maker wants to modify an old map, he will not be editing the smf and smt files directly, but editing the source files (texture.png, height.png or whatever images used.) The image resizing is not an issue, since your program does it automatically. But the typemap maybe an issue if the mapper only used the red channel. And the feature map will definitely be an issue as its a completely different format than mapconv (text vs image.)

Question:
In the featuremap, to force placement on the ground we use y = -500000. Why this exact value?

Other than the height clamping, everything works perfect. This really helped me out. Thanks.

-ljd
0 x

User avatar
tizbac
Posts: 136
Joined: 19 Jun 2008, 14:05

Re: New mapconv that works flawlessy on linux

Post by tizbac » 24 Aug 2011, 18:43

- Fixed a bug on compiling nonsquare map
- Added decompilation support that generates all files required to rebuild the map , makefile included
- Fixed clamping


Windows version still to build


Edit: there are some problems with heightmap loading from 16 bits formats , i'm working on it
0 x

User avatar
knorke
Posts: 7971
Joined: 22 Feb 2006, 01:02

Re: New mapconv that works flawlessy on linux

Post by knorke » 24 Aug 2011, 20:30

maybe you want to make a wiki entry on mapconv?
I think atm there only is http://springrts.com/wiki/Enetheru:mapconv
That page only offer as a Windows download and is orphaned.
(not linked to from any other page)
0 x

User avatar
tizbac
Posts: 136
Joined: 19 Jun 2008, 14:05

Re: New mapconv that works flawlessy on linux

Post by tizbac » 24 Aug 2011, 20:37

When it's 100% complete and working i'll write tutorials and maybe a graphical editor on top of it
0 x

User avatar
tizbac
Posts: 136
Joined: 19 Jun 2008, 14:05

Re: New mapconv that works flawlessy on linux

Post by tizbac » 24 Aug 2011, 21:55

Now it's completely fixed it seems , if you decompile a map and recompile it you get excatly same result

Windoze build attached
Attachments
SpringMapConvNG_2.7z
(973.88 KiB) Downloaded 184 times
0 x

Post Reply

Return to “Map Creation”

cron