View topic - XML-RPC searchapi on springfiles.com



All times are UTC + 1 hour


Post new topic Reply to topic  [ 79 posts ]  Go to page Previous  1, 2, 3, 4  Next
Author Message
PostPosted: 28 Jan 2011, 15:16 
Spring Developer

Joined: 31 May 2009, 23:08
you didn't understand: http://www.springfiles.com/download/3668/5 does send a HTTP 301 as response, if you want to get that file, you've to follow to that new location and request the file there with a range-request. So resources aren't wasted...

currently client + server implementation don't understand each other... one side has to be fixed, imo the server side. but i can't do anything, because i don't have access to springfiles, only to a dev-server.

jj wants accurate stats, but currently this is a bit broken because of this problem... let's wait what he says about this.

edit: i've fixed it on the dev-server it returns now the real url. jj please upload it to springfiles.com (https://github.com/springfiles/searchap ... api.module)


Top
 Offline Profile  
 
PostPosted: 28 Jan 2011, 16:04 
User avatar

Joined: 01 Apr 2006, 18:03
Location: The Netherlands
Licho, the files are not served by php, it's just a redirect, not a php- download script.

I dont like the approach to serve the direct url.
How can we check the used bandwidth for mirror owners?

But i will put abma's new code online, as soon as i am home (tonight)
That will solve all the problems.

ps. nice work abma!


Top
 Offline Profile  
 
PostPosted: 28 Jan 2011, 18:48 
Spring Developer

Joined: 31 May 2009, 23:08
uh... forgot to say, size and timestamp of the file is now returned, too.


Top
 Offline Profile  
 
PostPosted: 29 Jan 2011, 00:59 
Server Owner & Developer
User avatar

Joined: 19 May 2006, 18:13
Location: Brno, Czech rep., EU, Terra, Sol, Orion arm, Milky way, Virgo supercluster
Why the hell you care about used bandwith, i pay for the server and it has unlimited one. You cannot measure it this way anyway unless you process range headers correctly or measure it lower level.

I care about stable and reliable downloads for users..

And no Abma, you are mistaken, it does not return 302 for all URLs. Example:

Request
Code:
GET /system/files/downloads/spring/spring-maps/DeltaSiegeDry.sd7 HTTP/1.1
Host: www.springfiles.com
Range: bytes=262144-524288



Response
Code:
HTTP/1.1 200 OK
Date: Fri, 28 Jan 2011 23:54:43 GMT
Server: Apache/2.2.12 (Ubuntu)
X-Powered-By: PHP/5.2.10-2ubuntu6.5
Expires: Sun, 19 Nov 1978 05:00:00 GMT
Cache-Control: post-check=0, pre-check=0, private
Content-Disposition: attachment; filename="DeltaSiegeDry.sd7"
Set-Cookie: SESS91c96e8a48be43d5eb7b4b5d71a52400=cff809990db8c91808f0ad7af7f11b14; expires=Mon, 21-Feb-2011 03:28:03 GMT; path=/; domain=.springfiles.com
Last-Modified: Fri, 28 Jan 2011 23:54:43 GMT
Content-Length: 17640007
Content-Type: application/octet-stream; name="DeltaSiegeDry.sd7"


If this isnt php served file then what is it jj and why it ignores range?


Top
 Offline Profile  
 
PostPosted: 29 Jan 2011, 02:36 
AI Coder
User avatar

Joined: 14 Sep 2004, 10:32
Location: Cookieland
If we want bandwidth measured isn't that something that can be done via apache or some other method rather than php?


Top
 Offline Profile  
 
PostPosted: 29 Jan 2011, 20:28 
User avatar

Joined: 01 Apr 2006, 18:03
Location: The Netherlands
Abma has updated the code, so the problem with range headers will be gone.

Licho, i wanted to measure the dl stats because some mirrors have bandwidth limitations. Now lots of the traffic will be unmanaged which can be bad for particular mirrors. It has nothing to do with your server.


Top
 Offline Profile  
 
PostPosted: 30 Jan 2011, 12:34 
Spring Developer

Joined: 31 May 2009, 23:08
@licho: fixed, now only plain urls are returned


Top
 Offline Profile  
 
PostPosted: 30 Jan 2011, 13:28 
Spring Developer

Joined: 31 May 2009, 23:08
hm.. one issue i found is:

when watching the logs of springfiles, i saw some accesses like this:

GET /system/files/downloads/spring/games/Tech Annihilation.sd7 HTTP/1.1" 404 10643

this looks like some client's don't url-encode the request. maybe i should do this server-side in the xml-rpc result? (for example replace space with %20)


Top
 Offline Profile  
 
PostPosted: 30 Jan 2011, 16:33 
Server Owner & Developer
User avatar

Joined: 19 May 2006, 18:13
Location: Brno, Czech rep., EU, Terra, Sol, Orion arm, Milky way, Virgo supercluster
You could, but server has even more mirror sources so i should encode it there too.

And ideally clients should do that themselves but fastest fix is for me to url encode mirrors provided to lobbies.


Top
 Offline Profile  
 
PostPosted: 30 Jan 2011, 19:58 
User avatar

Joined: 16 May 2007, 17:33
Hey, nice work.

I don't like xmlrpc, but it is fine.
I implemented it in my Go package: https://github.com/Agon/go-slc/blob/mas ... archapi.go


Top
 Offline Profile  
 
PostPosted: 30 Jan 2011, 20:16 
User avatar

Joined: 16 May 2007, 17:33
Got a bot running which listens to following command:
"map mapname"
Prints out all mirrors which could be found.


Top
 Offline Profile  
 
PostPosted: 30 Jan 2011, 21:29 
Server Owner & Developer
User avatar

Joined: 19 May 2006, 18:13
Location: Brno, Czech rep., EU, Terra, Sol, Orion arm, Milky way, Virgo supercluster
Every springie autohost already does that but lists more than just those mirrors, because it has its own list as well.


Top
 Offline Profile  
 
PostPosted: 30 Jan 2011, 22:12 
Server Owner & Developer
User avatar

Joined: 19 May 2006, 18:13
Location: Brno, Czech rep., EU, Terra, Sol, Orion arm, Milky way, Virgo supercluster
jj, are you 100% sure nor

http://springfiles.com/system/files/dow ... egeDry.sd7

nor

http://springfiles.com/sites/default/fi ... egeDry.sd7

run using php?

I dont want ANY big downloads running through web technology. Make sure there are none please.

Yesterday your cron script killed server again and I don't want to waste extra resources :(


Top
 Offline Profile  
 
PostPosted: 30 Jan 2011, 22:34 
Spring Developer

Joined: 31 May 2009, 23:08
system/files/... is using php
sites/default/files ... isn't!


Top
 Offline Profile  
 
PostPosted: 30 Jan 2011, 23:51 
Spring Developer

Joined: 31 May 2009, 23:08
next addition: "description" field added,
this is a url to the download page + descriptions of a map/game, please use it!

the direct link is for autodownloaders, end-users should use the webpage.


Top
 Offline Profile  
 
PostPosted: 31 Jan 2011, 13:28 
Spring Developer

Joined: 31 May 2009, 23:08
@licho:
can you delete all urls from plasma starting with

http://springfiles.com/system
http://springfiles.com/download
http://www.springfiles.com/

i hope these are all invalid/duplicate urls...


Top
 Offline Profile  
 
PostPosted: 31 Jan 2011, 13:41 
Server Owner & Developer
User avatar

Joined: 19 May 2006, 18:13
Location: Brno, Czech rep., EU, Terra, Sol, Orion arm, Milky way, Virgo supercluster
They will be auto deleted when those becoe inalid


Top
 Offline Profile  
 
PostPosted: 31 Jan 2011, 13:59 
Spring Developer

Joined: 31 May 2009, 23:08
http://springfiles.com/system urls aren't invalid, they don't support the-range stuff.

the others are duplicates and should all return a 301 to http://springfiles.com but are valid, too.


Top
 Offline Profile  
 
PostPosted: 31 Jan 2011, 14:00 
Server Owner & Developer
User avatar

Joined: 19 May 2006, 18:13
Location: Brno, Czech rep., EU, Terra, Sol, Orion arm, Milky way, Virgo supercluster
Hmm then http://springfiles.com/system
should be removed from springfiles (if they are linked from somewhere).
I still see them used by the site itself.


Top
 Offline Profile  
 
PostPosted: 31 Jan 2011, 14:15 
Spring Developer

Joined: 31 May 2009, 23:08
drupal is using it, that can't be removed.


Top
 Offline Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 79 posts ]  Go to page Previous  1, 2, 3, 4  Next

All times are UTC + 1 hour


Who is online

Users browsing this forum: No registered users and 0 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB® Forum Software © phpBB Group

Site layout created by Roflcopter et al.