2 most important questions :
are the sm2 files decompressable back to their original form ?
and when maps are compressed, is the compression lossless ? because the program says "lower quality".......
about the sm2 format......
Moderator: Moderators
Re: about the sm2 format......
no - the format is lossy - It would be possible to split the sm2 out into the 3 "source" bitmaps however these would not be identical to the originals.monohouse wrote:2 most important questions :
are the sm2 files decompressable back to their original form ?
and when maps are compressed, is the compression lossless ? because the program says "lower quality".......
a bit of blurb:-
We're all familiar with the tilesets in OTA, mapconv attempts to create a good tileset from the large original texture graphic.
The Spring maps are based on tiles that are 32x32 pixels. When the Mapconv is processing your map texture it looks at the tiles one by one and calculates a "uniqueness" figure. It then builds a tile set by throwing away tiles that are very similar (<compression). The format is lossy, and when run at high compression you can see artifacts (especially at tile edges) as a similar tile has been substituted for the original graphic.
The compression works quite well, however a couple of things will help with the compressability: use texture sizes that are multiples of 32 and use smaller textures (==more repeating graphics == less tiles needed). You can take the tiling futher by aligning features on a 32x32 pixel grid - maybe you could have a mountain appear in a number of places. This saves tiles as the 1st mountain tiles are reused.
-
monohouse
-c 0.0 is not lossless then ?
can the data be made lossless with or without compression in the sm2 ? as in disable compression, or maybe lossless compression mode ?
and that data can be fully restored into the original 3 files form ?
or may as an addition can spring be allowed to read the 3 combining files of the map directly without sm2 (as in "TNT") ?
ymf is a lossless format, very effective too, too bad it wasn't developed far enough....
can you make some modifications the code to bypass compression ?
can the data be made lossless with or without compression in the sm2 ? as in disable compression, or maybe lossless compression mode ?
and that data can be fully restored into the original 3 files form ?
or may as an addition can spring be allowed to read the 3 combining files of the map directly without sm2 (as in "TNT") ?
ymf is a lossless format, very effective too, too bad it wasn't developed far enough....
can you make some modifications the code to bypass compression ?
Its easy to assume -c 0.0 to be lossless. however it depends how the uniqueness values (faststats) are worked out.
The amount of information in the tile graphic is reduced from 32x32x24 bits (24576 bits) to a faststat which is 9 integers (288 bits) so its possible that some tiles might have the same faststat value.
Its possible (anythings possible) to create a sm2 which is lossless - however the memory allocation required on you graphics card would probably cripple the framerate. a 8192x8192 x24 bitmap is about 200MB (my GF6600gt is 128mb). Its unrealistic to try to get spring to load uncompressed maps.. Why would you want to do this? Is it so that you can edit maps easier?
The amount of information in the tile graphic is reduced from 32x32x24 bits (24576 bits) to a faststat which is 9 integers (288 bits) so its possible that some tiles might have the same faststat value.
Its possible (anythings possible) to create a sm2 which is lossless - however the memory allocation required on you graphics card would probably cripple the framerate. a 8192x8192 x24 bitmap is about 200MB (my GF6600gt is 128mb). Its unrealistic to try to get spring to load uncompressed maps.. Why would you want to do this? Is it so that you can edit maps easier?
-
monohouse
obviously you need to match the software to the hardware as always, but if you also can tune the data to the hardware - that would yield the best results, id like to be able to define the quality myself, so if I want a map compressed I compresse it, but the SOURCE data must be lossless, have you heard of doom3 ? doom3 textures not compressed, and if the user requires (as in : out of video ram for high quality textures) the user can compress them, doom3 is good example of good compression implementation.
no, it is to preserve quality, doom3 looks good with compressed textures, but it looks AMAZING with uncompressed ones.....
which would be the same for spring if the data would be lossless in origin, what if tomorrow you get a new video card ? tell spring to reduce the compression and thus get better image quality, but if the data is already lost, there is nothing you can do, and no way to get improved image quality.....
and about the compression, the compression is bad, the 39 maps from fileuniverse for ta spring I compressed from 520 MB into 221 MB with 7-zip on 16 MB/64 word size and ultra mode, so if I compressed them if they were not "precompressed" I would get even higher ratio = smaller files AND higher quality.....
it's the quality :), it seems that spring has this file called "zlib.dll" I wonder what it is......you could use 7-zip compression in your program provided the license permits, but if not you can still use bzip2 as it is free for sure, and it would still be better than the current compression method used in the sm2 format, and on top of all that - it doesn't loose any data and thus quality :)
you can always compress the data some more, but if you already lost all the data what is there to compress ? there is no way to improve quality from a lossy format, but if the data is lossless you can compress to whatever you want, so if a user wants to compress the data he can do it without a problem anytime when you lose quality it is gone forever cannot restore - ONLY if it lossless you can re-lose as many times as you want - I think you know what I meen.
no, it is to preserve quality, doom3 looks good with compressed textures, but it looks AMAZING with uncompressed ones.....
which would be the same for spring if the data would be lossless in origin, what if tomorrow you get a new video card ? tell spring to reduce the compression and thus get better image quality, but if the data is already lost, there is nothing you can do, and no way to get improved image quality.....
and about the compression, the compression is bad, the 39 maps from fileuniverse for ta spring I compressed from 520 MB into 221 MB with 7-zip on 16 MB/64 word size and ultra mode, so if I compressed them if they were not "precompressed" I would get even higher ratio = smaller files AND higher quality.....
it's the quality :), it seems that spring has this file called "zlib.dll" I wonder what it is......you could use 7-zip compression in your program provided the license permits, but if not you can still use bzip2 as it is free for sure, and it would still be better than the current compression method used in the sm2 format, and on top of all that - it doesn't loose any data and thus quality :)
you can always compress the data some more, but if you already lost all the data what is there to compress ? there is no way to improve quality from a lossy format, but if the data is lossless you can compress to whatever you want, so if a user wants to compress the data he can do it without a problem anytime when you lose quality it is gone forever cannot restore - ONLY if it lossless you can re-lose as many times as you want - I think you know what I meen.
The other side to this problem is load times - the majority of data in the sm2 is in a DXT1 compressed texture format - which can be loaded directly into opengl or DirectX so is reasonably fast.
If we could store the sm2's in a decent, fast to extract zip file (or similar) then it'd slow the loading a little but the file sizes would be a bit smaller. I think Buggi's working on a HPI format which would do this sort of thing.
I've played Doom3 but not familiar with the texture handling, so I can't comment.
I see your point about having the source bitmaps and choosing how much compression you want for your machine - but do you think people will want to download a huge file and then wait for it to be compressed so they can play it? There are some people here doing 16384x16384 or larger - compressing a map like that takes quite a bit of time. I suppose they can always upgrade their machine!
If we could store the sm2's in a decent, fast to extract zip file (or similar) then it'd slow the loading a little but the file sizes would be a bit smaller. I think Buggi's working on a HPI format which would do this sort of thing.
I've played Doom3 but not familiar with the texture handling, so I can't comment.
I see your point about having the source bitmaps and choosing how much compression you want for your machine - but do you think people will want to download a huge file and then wait for it to be compressed so they can play it? There are some people here doing 16384x16384 or larger - compressing a map like that takes quite a bit of time. I suppose they can always upgrade their machine!
-
monohouse
roflmaooflol........ obviously not, and people don't have to download a huge file, only those interrested like me, but id love to be ABLE to do it .......just to allow it, so that you CAN, textures without compression look great, and if you CAN use them too, the rest is up to the users :)
as long as the program permits there are no problems, the user CAN make his graphics be better (MUCH) but if you cant.....there is nothing to talk about....... it is a matter of option, what about that "HPI" thing ? is it supposed to replace sm2 ? or is it just a compression for sm2 ?
if doom3 managed to get so many high-quality textures into 1.5 GB of data, I don't see how spring can't.
lossless is essential if you want high quality graphics for spring.
the huge file does not have to be huge, if there is a lossless texture compression format, using image compression, it can be almost as small as the sm2 with compressed textures.
then it can be loaded into the video card, it can also be precompressed before loading with a smaller video card, can't be taking THAT long, the compression from the lossless format is somewhere around 40% of original size, maybe more, that would bring it around 120 MB for a 300 MB texture, far more worth the quality :)
at the very least you could use DXT3 or some other higher-quality compression for textures, couldn't you ?
as long as the program permits there are no problems, the user CAN make his graphics be better (MUCH) but if you cant.....there is nothing to talk about....... it is a matter of option, what about that "HPI" thing ? is it supposed to replace sm2 ? or is it just a compression for sm2 ?
if doom3 managed to get so many high-quality textures into 1.5 GB of data, I don't see how spring can't.
lossless is essential if you want high quality graphics for spring.
the huge file does not have to be huge, if there is a lossless texture compression format, using image compression, it can be almost as small as the sm2 with compressed textures.
then it can be loaded into the video card, it can also be precompressed before loading with a smaller video card, can't be taking THAT long, the compression from the lossless format is somewhere around 40% of original size, maybe more, that would bring it around 120 MB for a 300 MB texture, far more worth the quality :)
at the very least you could use DXT3 or some other higher-quality compression for textures, couldn't you ?
I think you may be slightly overestimating the game there monohouse.
Doom 3 came on a DVD. Theres lots of room to put uncompressed files on it. Then they are all compresssed on the fly befor play depending on what settings you told it to compress to.
This game however is all downloaded. If you wanna download a DVD's worth of textures thats fine, but u'll have to respect the fact that even with broadband you only get 256 mbs upstream bandwidth and some map makers (actully all of us id think) wont want to sit there for a few hours uploading our uncompressed textures just for your benifit. Plus if you pay by the gig transfered, it would cost us map makers money to.
The maps all look fine and dandy as they are. Sure maybe some should be a little less compressed, and they might look very marginly nicer for it but its realy not worth the map makers time and effort.
aGorm
Doom 3 came on a DVD. Theres lots of room to put uncompressed files on it. Then they are all compresssed on the fly befor play depending on what settings you told it to compress to.
This game however is all downloaded. If you wanna download a DVD's worth of textures thats fine, but u'll have to respect the fact that even with broadband you only get 256 mbs upstream bandwidth and some map makers (actully all of us id think) wont want to sit there for a few hours uploading our uncompressed textures just for your benifit. Plus if you pay by the gig transfered, it would cost us map makers money to.
The maps all look fine and dandy as they are. Sure maybe some should be a little less compressed, and they might look very marginly nicer for it but its realy not worth the map makers time and effort.
aGorm
Spring uses 32x32 sized tiles on the map and DXT1 compression.
The tile replacing in mapconv can be turned off with compression level 0 or -1.
The DXT1 compression is a lossy image compression algorithm that has a compression ratio of 8:1 and can give some slight artifacts in the image (mostly colorloss or slight blockiness) these are however rarely noticable, especially not when mixed with a detail map like in spring.
The DXT1 compression is there to save system and video memory. If you want higher quality a better option than going for uncompressed image is usually to use higher resolution but still compressed data.
You can for an example create a spring map with 4 times the resolution, this would for most part look better than an uncompressed map in lower resolution and still only be half the size. (Diferent resolution maps isn't fully implemented in spring yet, while the format suports it it's mainly waiting for a better map editor).
DXT1 and DXT3 compression only differ in how they handle the alpha map and wouldn't make a difference for spring maps.
Spring map can allready be placed in a hpi file if you want a zlib type compression on them.
The tile replacing in mapconv can be turned off with compression level 0 or -1.
The DXT1 compression is a lossy image compression algorithm that has a compression ratio of 8:1 and can give some slight artifacts in the image (mostly colorloss or slight blockiness) these are however rarely noticable, especially not when mixed with a detail map like in spring.
The DXT1 compression is there to save system and video memory. If you want higher quality a better option than going for uncompressed image is usually to use higher resolution but still compressed data.
You can for an example create a spring map with 4 times the resolution, this would for most part look better than an uncompressed map in lower resolution and still only be half the size. (Diferent resolution maps isn't fully implemented in spring yet, while the format suports it it's mainly waiting for a better map editor).
DXT1 and DXT3 compression only differ in how they handle the alpha map and wouldn't make a difference for spring maps.
Spring map can allready be placed in a hpi file if you want a zlib type compression on them.
I think you have to differentiate between textures that used to build things out of, and a single giant image used to texture the entire map. You probably know this already, but for the benefit of other listeners: the graphics in first person games like the Doom series work because the scenes you see on screen are made up from lots of small textures - and crucially you only get to see a fraction of them in a scene at one time. Spring works by breaking the big map texture down into small textures so that it can be treated in the same way.monohouse wrote:no, it is to preserve quality, doom3 looks good with compressed textures, but it looks AMAZING with uncompressed ones.....
I have to agree with you that it is a bit counter-productive producing a map with a single massive lush terrain texture, when the compression gives you this (32x32) "pixelated" effect. It would look more lush if it had a uniform tile set which slotted together seamlessly.
If you have a good basic set of tiles that fit together seamlessly and you use lossless compression on those tiles then you get a nice looking seamless map that takes little memory. Most of the OTA maps are like this. The Spring Crosspaths map is a good example of a Spring map like this (< 1MB).
I think the danger is people try to produce lush looking maps and end up with stuff that doesn't look as good as something simple based on a decent tile set... because of the visible joins between the tiles.
I thought AfricaCoast was a great example of a lush map that worked because it uses repeated patterns to make it efficiently compressible but still has gorgeous textures. There are some tiny compression artefacts, but mainly they're not noticeable unless you look hard. Unfortunately it makes for games that crash, so something needs fixing!
Still, at the end of the day TA is all about excellent gameplay. The reason we're still playing a game that came out in 1997 isn't really anything to do with the graphics.
Just my tuppence
Munch
-
monohouse
