WIP - unnamed lobby

WIP - unnamed lobby

Discuss development of lobby clients, server, autohosts and auto-download software.

Moderators: Moderators, Lobby Developers

Locked
MetalSucker
Posts: 98
Joined: 22 Sep 2014, 20:29

WIP - unnamed lobby

Post by MetalSucker » 22 Jun 2015, 13:42

It's not lua and it's not an in-engine lobby.

Image

Python + gtk3 + twisted on the latest Ubuntu, at this point in time it logs in, registers, saves user/password, handles agreement/motd, and just enough commands to hold the users and battles list in python dictionaries, update them and the UI. Also pings. It can't yet join battles.

The image is a GTK3 ListBox, with sorting, updating, events and all. Oh yes, and it will have CSS theming/branding.

There is no ETA.

The UI code is/will be reasonably separated from the toolkit so I can make it run on Windows at some point.

Source will come, if anyone is interested, when I get it out of the "embarrassing" stage.
0 x

User avatar
PicassoCT
Journeywar Developer & Mapper
Posts: 10235
Joined: 24 Jan 2006, 21:12

Re: WIP - unnamed lobby

Post by PicassoCT » 22 Jun 2015, 16:13

Usps?
0 x

MetalSucker
Posts: 98
Joined: 22 Sep 2014, 20:29

Re: WIP - unnamed lobby

Post by MetalSucker » 22 Jun 2015, 16:59

PicassoCT wrote:Usps?
locbjefnej!

dabu! zud zud!
0 x

User avatar
Silentwings
Moderator
Posts: 3589
Joined: 25 Oct 2008, 00:23

Re: WIP - unnamed lobby

Post by Silentwings » 22 Jun 2015, 17:46

Nice to see something happening - but I was expecting something more in the style of viewtopic.php?f=64&t=32563 ?
0 x

MetalSucker
Posts: 98
Joined: 22 Sep 2014, 20:29

Re: WIP - unnamed lobby

Post by MetalSucker » 22 Jun 2015, 17:53

Silentwings wrote:Nice to see something happening - but I was expecting something more in the style of viewtopic.php?f=64&t=32563 ?
It will get there, I need to get to the point where all games start right, this is one step towards that. I don't handle the downloads right yet, that's the next step, then either pass the action to chilli lobby or something else, but the foundation isn't fully there yet, it's in its infancy. (read: weee, it didn't crash in the last hours!)
0 x

User avatar
Jazcash
Posts: 5302
Joined: 08 Dec 2007, 17:39

Re: WIP - unnamed lobby

Post by Jazcash » 23 Jun 2015, 20:53

Very interested in the source for this
0 x

MetalSucker
Posts: 98
Joined: 22 Sep 2014, 20:29

Re: WIP - unnamed lobby

Post by MetalSucker » 07 Jul 2015, 04:27

I'm not sure who I'm supposed to ask, so I'll post here and hopefully someone knows the answer.

When new players download a game (they don't have any files yet), the lobby queries the available download systems. In the event that a full archive is not found (on upq/plasma), then rapid is queried and the download starts from there. The problem is the actual download speed.

While rapid is great for developers/handling multiple versions/updates, it can be quite slow for users that have no files to start with in the pool.

That creates a major usability issue, since the time to download from rapid could be much higher than simply downloading an archive from a web server, especially since there are a lot of files in the hundreds of bytes range.

While I'm currently doing sequential requests from rapid (on purpose), I can do a large number of simultaneous requests, but I'm afraid that would overload the servers.

Basically if the game file isn't available as an archive, players that just downloaded the game for the first time are stuck with sub-optimal download speeds - that's a major turn-off.

Dunno, maybe I'm missing something.
0 x

User avatar
Forboding Angel
Evolution RTS Developer
Posts: 14599
Joined: 17 Nov 2005, 02:43

Re: WIP - unnamed lobby

Post by Forboding Angel » 07 Jul 2015, 07:24

Rapid should be asked first. NEVER prioritize an sdz/sd7.
0 x

MetalSucker
Posts: 98
Joined: 22 Sep 2014, 20:29

Re: WIP - unnamed lobby

Post by MetalSucker » 07 Jul 2015, 07:31

Right, but for first time users, an archive is much needed.
0 x

User avatar
Forboding Angel
Evolution RTS Developer
Posts: 14599
Joined: 17 Nov 2005, 02:43

Re: WIP - unnamed lobby

Post by Forboding Angel » 07 Jul 2015, 07:41

Rapid files aren't going to download at a different rate than an archive, it's the same server. It's not uncommon for a game to be over 300mb. So you're going to make them dl 300mb twice?
Last edited by Forboding Angel on 07 Jul 2015, 09:32, edited 1 time in total.
0 x

User avatar
PicassoCT
Journeywar Developer & Mapper
Posts: 10235
Joined: 24 Jan 2006, 21:12

Re: WIP - unnamed lobby

Post by PicassoCT » 07 Jul 2015, 08:23

Nothing to motivate one more then some insults in the early morning.
0 x

MetalSucker
Posts: 98
Joined: 22 Sep 2014, 20:29

Re: WIP - unnamed lobby

Post by MetalSucker » 07 Jul 2015, 08:29

Not even with http pipelining and spdy put together would separately archived files needing separate sequential requests and separate fwrites and md5 checks be faster than one single archive containing multiple files, over one connection with one md5 hash, be it the same server.

Maybe a daily pool snapshot would be an answer.

Again, maybe I'm missing something, it might have to do with the stable tag(s) and its (mis)use. I could of course pull from git and convert that to a pool, but I don't think it's ideal.

Fortunately, the target audience for this is made up of the people not currently playing spring, thus unable to issue opinions.
Last edited by MetalSucker on 07 Jul 2015, 08:43, edited 1 time in total.
0 x

dansan
Server Owner & Developer
Posts: 1192
Joined: 29 May 2010, 23:40

Re: WIP - unnamed lobby

Post by dansan » 07 Jul 2015, 08:42

I'm a player too, and I love rapid, because when a new version of *A is released, I can download it in seconds and it doesn't waste space on my SSD.

IMO not all springfiles mirror are also rapid servers, but I'm not sure. So there might be a bandwidth difference.

I think you should ask abma if it makes sense to create threads for 2 concurrent rapid downloads. More doesn't make sense with so few mirrors, but 2 may help in case of a single connection gone slow/broken/bad route. In my experience springlobby does a good job at downloading (not displaying it though). So abma can probably answer your questions best.
0 x

MetalSucker
Posts: 98
Joined: 22 Sep 2014, 20:29

Re: WIP - unnamed lobby

Post by MetalSucker » 07 Jul 2015, 11:58

git clone --depth 1 https://github.com/EvolutionRTS/Evolution-RTS

(total number of files)
find | wc -l
5574

(files smaller than 1k)
find . -size -2k -ls |wc -l
1715

(files smaller than 4k)
find . -size -5k -ls |wc -l
3287

The large number of very small files is slowing rapid since each one of them generates a request.

Since a new player that doesn't have any spring game installed needs to get all of them, it takes a lot of time. It's not the same as downloading an archive. Git itself is optimized for this type of small file transfer.

So, for the last time:
I am not talking about existing spring users.

I am talking about new ones, the kind that you don't want to get the wrong impression the first time they try to launch a game.



Ok, for the last last time:

NEW people coming to spring (from wherever) - rapid will be much slower for them then it is for you! And that's bad, because they get bored with installing the game and move on to something else.


Last last last time....
0 x

User avatar
Anarchid
Posts: 1380
Joined: 30 Nov 2008, 04:31

Re: WIP - unnamed lobby

Post by Anarchid » 07 Jul 2015, 12:16

The large number of very small files is slowing rapid since each one of them generates a request.
Are you certain of this?
rapid will be much slower for them then it is for you
I've redownloaded both evo and zk with a rm -rf'd pool folder several times and i didn't experience that "much slower", especially not any slower than manually downloading the archive or making a shallow clone from git.
he kind that you don't want to get the wrong impression the first time they try to launch a game
Why are you doing a whole new lobby in this case? It would be better to adapt an already working one, i bet, unless you're playing into the hands of not invented here.
0 x

MetalSucker
Posts: 98
Joined: 22 Sep 2014, 20:29

Re: WIP - unnamed lobby

Post by MetalSucker » 07 Jul 2015, 12:18

There is too much hostility here. I will stop using this forum.
0 x

gajop
Moderator
Posts: 3023
Joined: 05 Aug 2009, 20:42

Re: WIP - unnamed lobby

Post by gajop » 07 Jul 2015, 14:39

Anarchid wrote:
The large number of very small files is slowing rapid since each one of them generates a request.
Are you certain of this?
Is this really true? I played a bit with the pr downloader code before, and I was pretty sure that it would only download one file (i.e. one dependency) per game download.

If so, it would make sense to download an archive instead.
0 x

User avatar
Silentwings
Moderator
Posts: 3589
Joined: 25 Oct 2008, 00:23

Re: WIP - unnamed lobby

Post by Silentwings » 07 Jul 2015, 15:53

The large number of very small files is slowing rapid since each one of them generates a request. .. NEW people coming to spring (from wherever) - rapid will be much slower for them then it is for you!
My experience of (often, for reasons that would be off-topic here) wiping my pr-dl cache and starting again from scratch also suggests that this is not true. Do you have any evidence?
0 x

MetalSucker
Posts: 98
Joined: 22 Sep 2014, 20:29

Re: WIP - unnamed lobby

Post by MetalSucker » 08 Jul 2015, 03:36

To clarify things: implementing rapid downloads, using your documentation, following the link at https://springrts.com/wiki/Rapid leads to reading a sdp binary file then grabbing md5-named files over http (pipelined, yes, but individually) - and I did just what it said there.

It's not my fault that following your docs doesn't mention the streamer.cgi (it's mentioned under server side tools on another page) - so it's not obvious it exists, why would I go on to "server side tools" after following the Rapid page to the bottom?

Even so, individually zipped files sent in a sequence will be larger/slower than one archive.

It's absolutely ridiculous that Forb used the language he did while having no idea what he's talking about, and I am asked to bring proof, that the most vocal members of the forum are not only clueless about the lower level systems but pretty much have a behaviour that can only be described as "get off my lawn". That's either insecurity or adolescence.

I'm sorry for making a lobby development related post that clearly mentioned my dilemma, where I clearly wrote "i might be missing something", a post made in the lobby development section of the forum. I guess I was off topic.

Please remove my account here, development might continue on github.
0 x

gajop
Moderator
Posts: 3023
Joined: 05 Aug 2009, 20:42

Re: WIP - unnamed lobby

Post by gajop » 08 Jul 2015, 03:57

Seeing as the OP wants to end his presence on the forum I'm locking this thread.
0 x

Locked

Return to “Lobby Clients & Server”