WIP - unnamed lobby

[MetalSucker]

It's not lua and it's not an in-engine lobby.



Python + gtk3 + twisted on the latest Ubuntu, at this point in time it logs in, registers, saves user/password, handles agreement/motd, and just enough commands to hold the users and battles list in python dictionaries, update them and the UI. Also pings. It can't yet join battles.

The image is a GTK3 ListBox, with sorting, updating, events and all. Oh yes, and it will have CSS theming/branding.

There is no ETA.

The UI code is/will be reasonably separated from the toolkit so I can make it run on Windows at some point.

Source will come, if anyone is interested, when I get it out of the "embarrassing" stage.

[PicassoCT]

Usps?

[MetalSucker]

Usps?

locbjefnej!

dabu! zud zud!

[Silentwings]

Nice to see something happening - but I was expecting something more in the style of viewtopic.php?f=64&t=32563 ?

[MetalSucker]

Nice to see something happening - but I was expecting something more in the style of viewtopic.php?f=64&t=32563 ?

It will get there, I need to get to the point where all games start right, this is one step towards that. I don't handle the downloads right yet, that's the next step, then either pass the action to chilli lobby or something else, but the foundation isn't fully there yet, it's in its infancy. (read: weee, it didn't crash in the last hours!)

[Jazcash]

Very interested in the source for this

[MetalSucker]

I'm not sure who I'm supposed to ask, so I'll post here and hopefully someone knows the answer.

When new players download a game (they don't have any files yet), the lobby queries the available download systems. In the event that a full archive is not found (on upq/plasma), then rapid is queried and the download starts from there. The problem is the actual download speed.

While rapid is great for developers/handling multiple versions/updates, it can be quite slow for users that have no files to start with in the pool.

That creates a major usability issue, since the time to download from rapid could be much higher than simply downloading an archive from a web server, especially since there are a lot of files in the hundreds of bytes range.

While I'm currently doing sequential requests from rapid (on purpose), I can do a large number of simultaneous requests, but I'm afraid that would overload the servers.

Basically if the game file isn't available as an archive, players that just downloaded the game for the first time are stuck with sub-optimal download speeds - that's a major turn-off.

Dunno, maybe I'm missing something.

[Forboding Angel]

Rapid should be asked first. NEVER prioritize an sdz/sd7.

[MetalSucker]

Right, but for first time users, an archive is much needed.

[Forboding Angel]

Rapid files aren't going to download at a different rate than an archive, it's the same server. It's not uncommon for a game to be over 300mb. So you're going to make them dl 300mb twice?

[PicassoCT]

Nothing to motivate one more then some insults in the early morning.

[MetalSucker]

Not even with http pipelining and spdy put together would separately archived files needing separate sequential requests and separate fwrites and md5 checks be faster than one single archive containing multiple files, over one connection with one md5 hash, be it the same server.

Maybe a daily pool snapshot would be an answer.

Again, maybe I'm missing something, it might have to do with the stable tag(s) and its (mis)use. I could of course pull from git and convert that to a pool, but I don't think it's ideal.

Fortunately, the target audience for this is made up of the people not currently playing spring, thus unable to issue opinions.

[dansan]

I'm a player too, and I love rapid, because when a new version of *A is released, I can download it in seconds and it doesn't waste space on my SSD.

IMO not all springfiles mirror are also rapid servers, but I'm not sure. So there might be a bandwidth difference.

I think you should ask abma if it makes sense to create threads for 2 concurrent rapid downloads. More doesn't make sense with so few mirrors, but 2 may help in case of a single connection gone slow/broken/bad route. In my experience springlobby does a good job at downloading (not displaying it though). So abma can probably answer your questions best.

[MetalSucker]

git clone --depth 1 https://github.com/EvolutionRTS/Evolution-RTS

(total number of files)
find | wc -l
5574

(files smaller than 1k)
find . -size -2k -ls |wc -l
1715

(files smaller than 4k)
find . -size -5k -ls |wc -l
3287

The large number of very small files is slowing rapid since each one of them generates a request.

Since a new player that doesn't have any spring game installed needs to get all of them, it takes a lot of time. It's not the same as downloading an archive. Git itself is optimized for this type of small file transfer.

So, for the last time:
I am not talking about existing spring users.

I am talking about new ones, the kind that you don't want to get the wrong impression the first time they try to launch a game.



Ok, for the last last time:

NEW people coming to spring (from wherever) - rapid will be much slower for them then it is for you! And that's bad, because they get bored with installing the game and move on to something else.


Last last last time....

[Anarchid]

The large number of very small files is slowing rapid since each one of them generates a request.

Are you certain of this?

rapid will be much slower for them then it is for you

I've redownloaded both evo and zk with a rm -rf'd pool folder several times and i didn't experience that "much slower", especially not any slower than manually downloading the archive or making a shallow clone from git.

he kind that you don't want to get the wrong impression the first time they try to launch a game

Why are you doing a whole new lobby in this case? It would be better to adapt an already working one, i bet, unless you're playing into the hands of not invented here.

[MetalSucker]

There is too much hostility here. I will stop using this forum.

[gajop]

The large number of very small files is slowing rapid since each one of them generates a request.

Are you certain of this?

Is this really true? I played a bit with the pr downloader code before, and I was pretty sure that it would only download one file (i.e. one dependency) per game download.

If so, it would make sense to download an archive instead.

[Silentwings]

The large number of very small files is slowing rapid since each one of them generates a request. .. NEW people coming to spring (from wherever) - rapid will be much slower for them then it is for you!

My experience of (often, for reasons that would be off-topic here) wiping my pr-dl cache and starting again from scratch also suggests that this is not true. Do you have any evidence?

[MetalSucker]

To clarify things: implementing rapid downloads, using your documentation, following the link at https://springrts.com/wiki/Rapid leads to reading a sdp binary file then grabbing md5-named files over http (pipelined, yes, but individually) - and I did just what it said there.

It's not my fault that following your docs doesn't mention the streamer.cgi (it's mentioned under server side tools on another page) - so it's not obvious it exists, why would I go on to "server side tools" after following the Rapid page to the bottom?

Even so, individually zipped files sent in a sequence will be larger/slower than one archive.

It's absolutely ridiculous that Forb used the language he did while having no idea what he's talking about, and I am asked to bring proof, that the most vocal members of the forum are not only clueless about the lower level systems but pretty much have a behaviour that can only be described as "get off my lawn". That's either insecurity or adolescence.

I'm sorry for making a lobby development related post that clearly mentioned my dilemma, where I clearly wrote "i might be missing something", a post made in the lobby development section of the forum. I guess I was off topic.

Please remove my account here, development might continue on github.

[gajop]

Seeing as the OP wants to end his presence on the forum I'm locking this thread.