XML-RPC searchapi on springfiles.com
Moderator: Moderators
Re: XML-RPC searchapi on springfiles.com
If you changed interface or renamed files randomly in repositiories it might cause it, i will try to debug that when i have time. Do you get requests for the file?
Re: XML-RPC searchapi on springfiles.com
yes, now it works... i don't know where the ~3 hours of delay come from.
i improved partly the cron job (for uploading / indexing new files) so it is run soon when new files are uploaded or at the given schedule (currently every two hours) so we hopefully have only a delay of a few minutes.
i improved partly the cron job (for uploading / indexing new files) so it is run soon when new files are uploaded or at the given schedule (currently every two hours) so we hopefully have only a delay of a few minutes.
Re: XML-RPC searchapi on springfiles.com
plasma lists this url as mirror:
"http://www.darkstars.co.uk/downloads/vi ... nculta.sd7", this is also a duplicate url of http://jj.darkstars.co.uk/c2/Inculta.sd7
but this file is damaged :-/ (md5sum incorrect)
this currently breaks downloading for springlobby
please do something against listing urls not returned by xml-rpc interface :-/
"http://www.darkstars.co.uk/downloads/vi ... nculta.sd7", this is also a duplicate url of http://jj.darkstars.co.uk/c2/Inculta.sd7
but this file is damaged :-/ (md5sum incorrect)
this currently breaks downloading for springlobby
please do something against listing urls not returned by xml-rpc interface :-/
Re: XML-RPC searchapi on springfiles.com
Plasma server returns torrent file to verify hashes for each piece .. all downloaders should always verify downloaded piece.
I recommend deleting broken file.
There are more mirrors than those tracked by springfiles, including old web repositories which are simply remotely checked for presence of correctly sized file.
I cannot turn off this ability because its needed for some things which are not trackedby SF, like missions or builds of rapid mods for springlobby.
I also dont want to be 100% dependent on springfiles because it had to be shutdown many times in the past.
Please return "banlist" instead - mirrors which should be avoided.
I recommend deleting broken file.
There are more mirrors than those tracked by springfiles, including old web repositories which are simply remotely checked for presence of correctly sized file.
I cannot turn off this ability because its needed for some things which are not trackedby SF, like missions or builds of rapid mods for springlobby.
I also dont want to be 100% dependent on springfiles because it had to be shutdown many times in the past.
Please return "banlist" instead - mirrors which should be avoided.
Re: XML-RPC searchapi on springfiles.com
i have no access to this file, i can't delete it :-/
where to return the banlist?
the banlist would be all links that are not listed in the result... the xml-rpc doesn't know links outside of the mirror-system.
so: i had to query plasma-service to get all urls + compare with own list -> make banlist :-/
imo you can delete all urls with ? or & in its url.
where to return the banlist?
the banlist would be all links that are not listed in the result... the xml-rpc doesn't know links outside of the mirror-system.
so: i had to query plasma-service to get all urls + compare with own list -> make banlist :-/
imo you can delete all urls with ? or & in its url.
Re: XML-RPC searchapi on springfiles.com
http://jj.darkstars.co.uk was managed by JJ. He did not comply with requests and thus I had to do it myself, requiring I change the password as I didn't have the original email containing it. JJ never asked for the new password despite it being offered. All I have done since is remove htaccess files and change file permissions to make things more open
The details for the account are stored on my main machine ( which recently broke and is being replaced ). It would be a few days minimum before I can regain access to that particular subdomain. I would recommend you amend your system so that host involvement is not necessary ( if only because I can now launch a makeshift attack by modifying the files on my servers, and you would require the cooperation of the attacker to fix it )
The details for the account are stored on my main machine ( which recently broke and is being replaced ). It would be a few days minimum before I can regain access to that particular subdomain. I would recommend you amend your system so that host involvement is not necessary ( if only because I can now launch a makeshift attack by modifying the files on my servers, and you would require the cooperation of the attacker to fix it )
Re: XML-RPC searchapi on springfiles.com
abma wrote:i have no access to this file, i can't delete it :-/
where to return the banlist?
the banlist would be all links that are not listed in the result... the xml-rpc doesn't know links outside of the mirror-system.
so: i had to query plasma-service to get all urls + compare with own list -> make banlist :-/
imo you can delete all urls with ? or & in its url.
But in the case of evolution you know that right? You could ban those..
These are mirrors it uses:
http://www.tasdownloads.com/uploads/%t/
http://www.evolutionrts.info/%t/%f

http://spring.vsync.de/%t/%f

http://www.jobjol.com/spring/%t/%f

http://www.ta-spring.com/%t/%f

http://www.fileuniverse.com/spring/%t/%f

http://packages.caspring.org/builds/%f

http://packages.springrts.com/builds/%f

It also used in the past:
http://www.springfiles.com/system/files ... g/games/%f
http://www.springfiles.com/system/files ... ng-maps/%f
Re: XML-RPC searchapi on springfiles.com
I could give you edit access to this list but deleting old mirror links means deleting it from database.
Even if I remove mirror site from the list, if the file is still present it will keep usingit.
Even if I remove mirror site from the list, if the file is still present it will keep usingit.
Re: XML-RPC searchapi on springfiles.com
i think deleting by hand makes no sense, a blacklist is bad, too.
as springlobby + zero-k (i hope so it does) checks for valid files for now, that shouldn't be a problem.
i made a small update/fix on the searchapi:
the comparison of springname is now binary + underscore is now threated as underscore and not as wildcard. I found that because when searching for "Lost_v2" two results where returned. "Lost-v2", exists, too.
results should be now more accurate.
as springlobby + zero-k (i hope so it does) checks for valid files for now, that shouldn't be a problem.
i made a small update/fix on the searchapi:
the comparison of springname is now binary + underscore is now threated as underscore and not as wildcard. I found that because when searching for "Lost_v2" two results where returned. "Lost-v2", exists, too.
results should be now more accurate.
Re: XML-RPC searchapi on springfiles.com
They should both check that file content is valid, however tasclient probably does not do that, so it could download faulty file.
Sometimes all mirrors are "corrupted" meaning torrent is for another file than in the mirror system. In such cases i delete the file from the system and it gets back with correct torrent usually.
Sometimes all mirrors are "corrupted" meaning torrent is for another file than in the mirror system. In such cases i delete the file from the system and it gets back with correct torrent usually.
Re: XML-RPC searchapi on springfiles.com
now "name" and "version" are additionally returned, torrent data will be returned when "torrent" is set in request. (see doc/exampleclient.py)
Re: XML-RPC searchapi on springfiles.com
added "tag" to search + added "sdp" to the result.
./exampleclient.py "zk:stable"
[...]
'tags': ['zk:revision:3767', 'zk:stable'],
'sdp': 'http://packages.springrts.com/packages/ ... 087aa0.sdp',
[...]
note:
tags are returned when they are on rapid _and_ springfiles.com.
the info is added by a script that is currently run daily by cron:
./exampleclient.py "zk:stable"
[...]
'tags': ['zk:revision:3767', 'zk:stable'],
'sdp': 'http://packages.springrts.com/packages/ ... 087aa0.sdp',
[...]
note:
tags are returned when they are on rapid _and_ springfiles.com.
the info is added by a script that is currently run daily by cron:
Re: XML-RPC searchapi on springfiles.com
major update:
http://api.springfiles.com/, the page allows testing of input to the xmlrpc interface and shows results.
please use http://api.springfiles.com/xmlrpc.php as soon as possible, the old url will be switched off soon.
its still a work in progress, but should work nicely.
changes:
- result returns links to splash / map images
- added metadata in results
- sdp hash for all files is calculated
- its much more faster than the old one!
- search for tags works (ba:latest for example), currently only for files that are also known on springfiles.com
- stuff i forgot...
- all source can be found at http://github.com/springfiles/upq (moved many stuff from the drupal module into upq)
http://api.springfiles.com/, the page allows testing of input to the xmlrpc interface and shows results.
please use http://api.springfiles.com/xmlrpc.php as soon as possible, the old url will be switched off soon.
its still a work in progress, but should work nicely.
changes:
- result returns links to splash / map images
- added metadata in results
- sdp hash for all files is calculated
- its much more faster than the old one!
- search for tags works (ba:latest for example), currently only for files that are also known on springfiles.com
- stuff i forgot...
- all source can be found at http://github.com/springfiles/upq (moved many stuff from the drupal module into upq)
Re: XML-RPC searchapi on springfiles.com
small update (maybe major...):
i normalized all filenames, new files filenames will be normalized, too:
the name is now always:
<lowercase springname>[-version].<suffix>
also: duplicate filenames and version-numbers aren't possible any more on the springfiles.com mirror-system. currently only files on api.springfiles.com are normalized, soon i'll normalize all filenames on the mirrors, too.
broken files (files that cant't be loaded by unitsync) are moved to http://api.springfiles.com/broken/ , maybe this way we get a higher quality of files that are downloadable...
note: i hopefully have soon the courage to activate auto-creating of pages for files on springfiles.com. the code is ready and partly tested, but i fear a bit, that it does stupid things
i normalized all filenames, new files filenames will be normalized, too:
the name is now always:
<lowercase springname>[-version].<suffix>
also: duplicate filenames and version-numbers aren't possible any more on the springfiles.com mirror-system. currently only files on api.springfiles.com are normalized, soon i'll normalize all filenames on the mirrors, too.
broken files (files that cant't be loaded by unitsync) are moved to http://api.springfiles.com/broken/ , maybe this way we get a higher quality of files that are downloadable...
note: i hopefully have soon the courage to activate auto-creating of pages for files on springfiles.com. the code is ready and partly tested, but i fear a bit, that it does stupid things
Re: XML-RPC searchapi on springfiles.com
ehhh that sound good! especially the broken files thing.
Re: XML-RPC searchapi on springfiles.com
As soon as unitsync SPRING_DATADIR is fixed (it ignores it), i will do that tool to auto regiser all new content so they will be instantly available on autohosts.
Re: XML-RPC searchapi on springfiles.com
i tapped into the same issue on linux and but made a work-around by setting HOME... don't know if that works on win32...Licho wrote:As soon as unitsync SPRING_DATADIR is fixed (it ignores it), i will do that tool to auto regiser all new content so they will be instantly available on autohosts.
can't we bundle manpower and don't write the same stuff twice? it feels stupid, to code it in python and .net, one language should be enough?!
Re: XML-RPC searchapi on springfiles.com
Sure!
Hmm setting home.. i can try it but i doubt it wil use it on win.
Hmm setting home.. i can try it but i doubt it wil use it on win.
Re: XML-RPC searchapi on springfiles.com
added the parameter "limit" which allows to restrict amount of results returned. defaults to 10 when not set (as before)
now for example a list of all engine versions can be shown:
http://api.springfiles.com/?category=en ... ws&limit=0
(the parameter works for the xml-interface, too)
please don't abuse this parameter, as in 99% of cases you don't need a whole list: cache results as they shouldn't change very often.
https://github.com/springfiles/upq/comm ... b7d3bda669
now for example a list of all engine versions can be shown:
http://api.springfiles.com/?category=en ... ws&limit=0
(the parameter works for the xml-interface, too)
please don't abuse this parameter, as in 99% of cases you don't need a whole list: cache results as they shouldn't change very often.
https://github.com/springfiles/upq/comm ... b7d3bda669