Static libs - can they be removed?

[gilboa]

so it is enough if i do in our configure/build script:

Code: Select all

if (systemHasSharedLib(7z))
   linkToShared();
else
   compileStatic();
   linkToStatic();

?

Yes.
If it's too much trouble, simply add --system-minizip --system-7z and --system-md5.

For the other libs (not 7z. md5. minizip), the reason is that we require changes to these libs to be able to keep synced (and in the case of Lua, for optimization/threading related reasons). these changes would not make sense for other users of these libraries.

OK. Good enough for me.
I'll attach a link to this thread to my approval request.

Thanks again.

- Gilboa

[lurker]

Is there a base64 lib?

[gilboa]

Is there a base64 lib?

OpenSSL seems to be a safe bet.
http://devenix.wordpress.com/2008/01/18 ... openssl-2/

- Gilboa

[jK]

Is there a base64 lib?

perhaps we should switch to the boost one.

Btw loading OpenSSL just for base64 doesn't seem to be a good idea (maybe only as a fallback solution).

[abma]

maybe minizip can be replaced with lib7z?

i'm unsure where the zip/7z functionality is used, replacing this with physfs (http://icculus.org/physfs/) should be possible, too.

[lurker]

Fallback should be the paragraph of code we have right now. Using the version in Boost sounds good assuming this isn't going to require linking an additional library; I'm not really sure how to check. Okay, maybe even if it does. But not an entire nonoptional dependency.

[gilboa]

Hello,

I'm resurrecting this thread in-order to save the need to summarize the previous discussion (beyond the following one liner)

I've managed to get the Fedora governing body to allow spring to include a number of static libraries [1]. (E.g. modified lua)
Beyond that, there's an effort within Fedora to package lzma-sdk and streflop. Unless something major happens, both should land before F16 is released.

1. Would it be possible to make spring build-sys detect existing lzma-sdk and streflop at compile time?
2. In the mean time, any possible issues with me, patching the spring build-sys to use the Fedora supplied libraries instead of the built-in ones?

Thanks,
- Gilboa
[1] https://fedorahosted.org/fpc/ticket/105

[SpliFF]

You may need to widen the discussion to include assimp (http://assimp.sourceforge.net/) which was recently added to spring for loading models. As far as I can see only Debian (http://packages.debian.org/sid/libassimp2) and ArchLinux (http://aur.archlinux.org/packages.php?ID=51058) provide shared library packages at this point however it's probable this will change in the future as assimp grows in usage.

Currently assimp is statically built with Spring but in theory it should work as a shared lib. The main consideration is that all distros (including windows/macosx builds) must provide identical results for model geometry (including postprocessing steps and transforms) or multiplayer games will desynchronise.

In the past I have spoken with assimp devs regarding this and they were fairly certain the results returned should reflect the hardcoded numbers in the model files in all cases however they weren't able to give me a definite. The situation would be more unclear if an assimp module were to transform the model using non-streflop maths operations. To combat this I originally replaced assimps maths functions with streflop:: versions and forced assimp to be built using the --fsingle-precision-constant argument to GCC.

Whether these things would be required in a system lib or wether they were requirements of being statically built with Spring is unclear.

EDIT: Found a request for including libassimp in Fedora: http://fossplanet.com/f13/%5Bbug-635511 ... ons-54074/

[cleanrock]

For compression you may wanna check out libarchive:
http://code.google.com/p/libarchive/

[hoijui]

just recently, we did the same for minizip because of gentoo.
https://github.com/spring/spring/commit ... b374fb1d30

[jK]

AFAIK it's not possible to use system lzma-sdk. lzma lib is a pain in the ..., I already tried to update our multiple years old one. Just to found out that the official one isn't posix compatible, instead each distro patch it to get it working. Additionally the API is everything else than stable and changes with each release.

[hoijui]

hmm :/
sounds like big management fail on behalf of the LZMA guys then.
why would they not accept patches into the main source trunk?
sounds to me like any distro guys, like those of Fedora, or at best those of all distros combined, should urge the LZMA guys to include their fix, and if they refuse, they should create a fork project, which would then be used by all linux distros (and therefore most open source devs, most likely), which would in a way mean death for the original project.

seems to me that linux distro maintainers should rather put effort into such stuff, instead of caring about how libs are used by end-user software like spring.

where is my cerviziaaa!!!!

[jK]

sounds like big management fail on behalf of the LZMA guys then.
why would they not accept patches into the main source trunk?

It's a single guy and it is very old. So it was post SVN/git times, and he never merged to it, so it is impossible to write patches and/or follow the development.

[hoijui]

(i assume you mean pre instead of post)

so there is no SCM for the LZMA lib, only tarballs?
this definitely cries for a fork then.

or is there an SCM, but all linux distros use old version of LZMA? (if so, why? are newer ones even less posix compatible?)

.. just trying to find out what would be the right thing to do, so we might try to get some distro guys together and do it.

[SpliFF]

http://tukaani.org/xz/

Fork of liblzma. I just built it on Gentoo and it's a small package with several nice CLI tools equivalent to those provided by bzip/gzip (eg, xzcat, xzless). More importantly though it provides an up-to-date posix liblzma.so, compiles and runs on almost anything, and development follows the usual OSS conventions (forums, git repo, mailing list, man pages, etc). Depending on this package rather than 7z package is probably the way to go.

[hoijui]

sounds perfect!
nice find SpliFF!
The page states it is targeted mostly at POSIX systems, but it also compiles on windows.

[jK]

xz is not compatible with lzma (means fileheaders are changed).
It was started by the GNU ppl, which indicates problems between them and the lzma dev (don't think they like to reimplement stuff), but possibly it was just a license thing.
They also had a `lzma utils` project, but stopped it years ago.

[hoijui]

mm :/
so the best option would be, to take up LZMA Utils again?
should we support LZMA2 aswell?

[SpliFF]

@jK: That's not my understanding of the situation

AFAIK the original 7-zip dev(s) were win-centric and GUI oriented. "lzma utils" was a fork to remedy this by moving to a posix codebase. The "lzma utils" was not stopped or abandoned, it evolved/became "xz utils" which added a new archive format "xz" and tools but still contains the "lzma utils" and libraries (ie, it installs liblzma, lzma, lzcat, lzgrep, etc as well as xzcat, xzgrep, etc). I can't think of any reason why the "lz" parts wouldn't be compatible with existing 7z archives. It basically says all that on the xz site and most of the tools are sym.

Now I can't guarantee the lzma tools are 7-zip compatible. All I know is xz/lzma tools use the same underlying library and the lzma / xz manpage says: "lzma is equivalent to xz --format=lzma" and "The native file format is the .xz format, but the legacy .lzma format used by LZMA Utils and raw compressed streams with no container format headers are also supported." also this link seems to use the terms LZMA and 7-zip interchangeably.

@hoijoi, It's highly probable that linking to the "xz utils" version of liblzma.so gives you LZMA2 support "out of the box" with minimal code changes. It sounds like most details of LZMA2 are internal. Wikipedia simply says: "LZMA2 - modified version of LZMA providing better multithreading support and less expansion of incompressible data ... Compared to LZMA1, LZMA2 adds support for LZMA_SYNC_FLUSH, uncompressed chunks (smaller expansion when trying to compress uncompressible data), possibility to change lc/lp/pb in the middle of encoding, and some other internal improvements."

[hoijui]

mm
yeah looks like they support both LZMA and LZMA2.
There are also topics on their forum that support that, like for example the following one, where a user compares archives compressed on linux with XZ with ones compressed on linux with 7-zip, and also interchanges them to see if they are compatible.
https://sourceforge.net/projects/lzmaut ... ic/4074496

i guess we should make a feature request on mantis, to replace 7zip with xz.