AlphaLobby Alpha 0.06.10

[Axiomatic]

http://springfiles.com/system/files/dow ... 0.04.8.exe
Has too many problems to be worth using.

[Axiomatic]

http://springfiles.com/system/files/dow ... 0.04.9.exe

Had a major bug last time, was crashing depending on how memory was arranged.
Progress bar is also mostly fixed.Main problem now is assuming compressed file size is proportional to actual size, so it's inaccurate. But it gives shows that things are progressing so its much better.

Last post I compared rapid download speeds, I deleted it because I realized I was cheating; Alphalobby got the benefit of hard drive caching which cut 27 seconds off time. I could recover these seconds by sending requests earlier; but here is fairer results:

Alphalobby: 113 seconds
Zero-K: 160 seconds
Rapid: 187 seconds

test was time to download Zero-K 0.7.3 after "spring/packages" and "spring/pool/0*" were removed.

still to do:
Speed up downloading. Improve progress bar.

[very_bad_soldier]

Are you sure your speed tests are reasonable (same server, same time, etc.)? I mean, its just http/ftp-downloading (I assume), how do you gain such major speed advantages over ZeroK?

[Axiomatic]

Are you sure your speed tests are reasonable (same server, same time, etc.)? I mean, its just http/ftp-downloading (I assume), how do you gain such major speed advantages over ZeroK?

You try to compare things for yourself if you want; my "benchmark" could be flawed. I wrote 3 different implementations of rapid, using winhttp, libcurl and wininet. For a while I thought wininet was fastest, but I found out that it was just caching downloads. After I flushed tmp files the winhttp implementation was fastest by about 20%. I could be accidentally cheating, but I aren't intending to be misleading.

Honestly, results did surprise me. Though I always profile my code and I've learnt to expect to be surprised. I did repeat tests many times and got a similar outcome each time. I haven't looked at ZeroK source so I could only guess why it's slower, but I'd assume it uses System.net, which is completely different until the socket level. The issue with rapid is its downloading a lot of small files; so performance depends a lot on how quickly requests can be made. IE it is significantly effected by implementation, rather than just bandwidth.

I also spent a lot of time performance tuning; its surprising just how much room for optimization there is. Currently my implementation spends about 25 seconds reading the hard drive before it can download files, just today I cut time down by 15 seconds by starting downloading before the package file is fully parsed (improvement would be much smaller on an SSD). I think I can cut an extra 5-10 seconds by queuing newly downloaded files in memory before writing them to disk, so the downloading thread isn't tied up in IO.

PS: Alphalobby's map downloader hasn't been optimized at all. I expect to cut time significantly when I get time to work on it.

[slogic]

Window size is not remembered on exit, only position.

In single player tabe "leave" button is redundant. You can close tab by "X" button on the tab itself or by Ctrl+W combination. Freed space worth to be used for comboboxes (map & game selectors).

[Axiomatic]

Window size is not remembered on exit, only position.

In single player tabe "leave" button is redundant. You can close tab by "X" button on the tab itself or by Ctrl+W combination. Freed space worth to be used for comboboxes (map & game selectors).

Window position is saved for me, maybe you aren't using latest version. I made many changes how config files are handled recently (mainly to reduce IO), it might be fixed already.

IMO, leaving a battle isn't just terminating a window, its an action, so it belongs on the bottom of the screen. Think of how a "print" dialog works. If I had to choose between an 'X' on the tab and the button, I would use only the button. The 'X' uses negligible space so there is no point removing it though.

I think the real issue is that the combo boxes can get kind of narrow sometimes. I agree, it would be good if they moved above the "leave/start/add bot" buttons when they didn't fit. The interface is unpolished pretty much everywhere. UI is not my strength. I know things look bad but it can hard to identify why.

I really appreciate the feedback, I'm writing this stuff down for when I get around to the GUI again. It won't be until I finish my work on the downloader, my brain only fits so much in it at once and its easier to focus on one area at a time.

And a thought about the download speeds; I'm in Australia so I have 700-800ms latency with servers (compared to <250ms for Europe) , when you are downloading >200 30kB files, round-trip latency really adds up. (Also this is why I hate Dguns, European commanders do 3x DPS with 3x accuracy)

[knorke]

in SP it sometimes seems to remove the wrong bot.
ie you click on the [3] thing on the map of a bot but bot[4] gets deleted.
(couldnt find a way to reliable reproduce)
one way to reproduce something similiar is to put multiple bots on the same ID, then delete one: it will delete all bots, not just one.

when selecting/deselecting spectate i would expect that one startposition marker disappears? (the one where human player starts)

colored starmarkers etc would be nice but i figure the buttons are just stand-in.

In the context menu, it says "Set team" but it actually sets the ally team.
As to avoid confusion with team/ally team i would rename that option to "Set Allyteam"

Tools -> Open Replay freezes the lobby for a few moments. (does not react, looks like hung up)
Hm just noticed I have 2600+ replays thats probally why. (should delete some old)
Still if you want to make it a seperate thread or show progress or w/e...

[det]

You should be using streamer.cgi for rapid downloading, HTTP GET per file is really inefficient. See http://springrts.com/phpbb/viewtopic.ph ... 21#p414921

[Axiomatic]

Problems with SP seem to be related to changing the team/ID of players, I'll fix it soon but not while I have a half-working downloader. Its too much effort to fix bugs twice at this stage.

Will definitely look into using the streamer, I was working from http://springrts.com/wiki/Dev:LobbyDownloadSystems and it doesn't really give much info.

To make sure I understand it correctly, say the package had 5 files in it and the client already had the 2nd and 5th, then the client would send "10110" to the server (gzipped ofc). nvm I understand what you meant now.

[Axiomatic]

I think I'll be able to get the streamer to download in 5-10 seconds less time than the old method of spamming http requests. Its not going to be a huge difference. Http actually doesn't have much bandwidth overhead, its main problem is latency, but since I was already sending requests in parallel, the latency isn't a huge problem.

Currently I have a target for my own benchmark of 80 seconds with warm hdd cache, 90 seconds on fresh boot. ZeroK does it in 140 and 160 respectively.

[det]

Depending on your latency and bandwidth, you may need something like 50-100 concurrent HTTP connections to max it out. streamer.cgi is a lot nicer than that IMO :)

Feel free the contact me in the lobby if you have any rapid questions (det[THE]dave)

[Axiomatic]

Thanks, your forum post pretty much covered everything, its done already. I got download time down to 77 seconds, from ~90 seconds.

My main focus now is compensating for hard drive time, it takes 25 seconds to scan pool directory. Even when I start downloads earlier, it seems like network performance is degraded by hard drive usage, so I can only recover maybe 10 seconds.

[aegis]

if you assume pool files haven't changed based on filesize and modification time you can just list the directory and pull from a cache to load rapid

[Axiomatic]

Listing a directory is a fairly expensive operation and would take longer than just checking whether the files in a package exist locally. (which is all I currently do. I think its safe to assume that if a file exists, then it's uncorrupted)

And yes I did test this. Listing the contents of a directory with 5000 files took 15 seconds. Checking whether 5000 files exists took 25. If I run again without rebooting, time is ~250ms for both.

The most expensive step (other than downloading) is checking whether pool files exist locally. Caching the contents of "pool/*" would reduce this time dramatically, except there isn't an fast way to check if a directories contents has been modified by some other program.

Its only 25 seconds on my slow 6 year old hard drive anyway, 10-15 if i start fetching files as early as possible; so I guess it isn't a huge problem. Would probably take half as long on a modern PC and almost instant on an SSD.

[Axiomatic]

http://springfiles.com/system/files/dow ... 0.05.1.exe

Rapid is slightly faster now, I think I'm pretty much at the limit unless I did something insane like manipulate sockets directly. I got an extra 5 seconds or so when HDD is bottleneck at the cost of 100MB RAM, not sure if its worth it and I might revert it. Problem with testing network code is results tend to vary by +/- 10% so its hard to fine tune.

I looked briefly at Zero K source for package download http://zero-k.googlecode.com/svn/trunk/ ... ownload.cs. I think the main reason it's slow is it only makes a single request.

Got some fixes in for single player, removed "Change ID" field because I can't see any reason why coop would be useful in single player.

@knorke, I've consistently referred to ally as "team", and team/id as "ID", because to most people, "team" and "ally" are synonyms. Would prefer not to break convention, but this case is exceptionally confusing if I don't.

Yes replay list needs to be threaded, but it also means I need to worry about synchronization. Replay list was just a quick and dirty hack so there wouldn't be a gap in features, its a low priority and not going to be cleaned up for a while.

Start positions are colored now, but don't show anything else. Win32 is a nightmare after you go past the limited functionality provided by common controls. I would have to manually draw each start position to do anything that looks half decent.

Going to focus on all the small bugs for a while, people have reported some embarrassingly obvious problems that I should have noticed if i spent more than 2 minutes testing.

[slogic]

Still window size is not remembered. Windows size is width & height, not left corner position.

[Axiomatic]

Still window size is not remembered. Windows size is width & height, not left corner position.

Works for me. It is strange that the window position is remembered but not size, because both are handled together. Would be helpful to know OS. I've only tested stuff in XP, though I can access win 7 if i need to.

Assuming other settings are saved (to "My Documents\My Games\Spring\alphalobby.conf"), I honestly can't see any reason window size wouldn't be remembered too.

I doubt you would bring this up if you didn't have a problem with the default size. Next release I'm going to let windows choose window size on first run so it shouldn't end up partially off-screen.

[slogic]

I've tested both in 7 and XP. Looks like "alphalobby.conf" is not created at all. I changed default "My documents" folder to "D:\docs.private" on XP (under 7 also changed). "D:\docs.private\My Games\Spring" folder also exists containing pool data & other Spring crap. So, i think you're using improper method of getting "My documents" folder and/or bad error control when "alphalobby.conf" should be created.

[Axiomatic]

I think the problem is that temp file I write to is on a different partition, so the move fails (copy and delete behavior should really be automatic, but its not). The path to "my documents" isn't hard coded; I'm pretty sure I am using the correct method to generate it.

Here's a fix.
http://springfiles.com/system/files/dow ... 0.05.2.exe

[slogic]

Ok, looks like it's fixed now. Under XP at least it is working.