I propose that we have hosting sites, as well as the internal spring file sharing system done by torrents, use a binary difference model. Whenever someone is interested in downloading a new mod (through a spring client), a local check is done to see what previous versions of the mod he has. In case he has none, the regular full download is done.
In case the user has an existing version, he sends the hash of the file to the system. The system then responds by giving the URL of the patch, which can be a torrent of the patch, or the regular http/ftp file download link.
This can be applied to initial file uploads as well, where the mod uploader would do a local diff and send only the file difference to the site.
Now as far as tools are concerned, doing a quick search on linux packages gives me bsdiff (http://www.daemonology.net/bsdiff/), a generic binary diff/patch program, which seems to have a Windows port available as well.
A couple of examples on it's efficiency and usage are shown below:
Code: Select all
Patches:
19M 731_750
485K 750_760
17M 760_762
148K 762_763
Full files:
21M ba731.sdz
21M ba750.sdz
21M ba760.sdz
21M ba762.sdz
21M ba763.sdz
An example creation and application of a patch
Code: Select all
#patch creation
$ bsdiff ba762.sdz ba763.sdz 762_763
#patch application
$ bspatch ba762.sdz ba763_patched.sdz 762_763_diff
#produces nothing, they're the same
$ diff ba763_patched.sdz ba763.sdz
PS: All the above should be done "behind the scene", which means user should be doing the download using a spring client (springlobby f.e) which does the process for him - he just has to specify what he wants to download. In case when a user goes to a site to download mods, it can be done by downloading a (small) file which will complete the download process in one of the spring clients (similar to how .torrent files work).