Elo, Glicko and Trueskill ratings on replays.springrts.com

[Silentwings]

Interesting - so maybe in the new SPADS balance algorithm it would make sense to allow +-5% difference as though its no difference, but bias heavily against >10% difference. That way you get more randomness (= more variety and fun) than always picking the balance that gets you closest to 0%.

[Silentwings]

A couple of bugs on the replay site:
(1) The faction it reports for each player is the one from the lobby, whereas most people chose their side ingame when placing startpoint nowadays. (Result is the replay site makes it look at though 90% of people use arm!)
(2) The rank reported for each player is the one reported by the lobby and not the chrank used by the autohost in its balancing.

[dansan]

A couple of bugs on the replay site:
(1) The faction it reports for each player is the one from the lobby, whereas most people chose their side ingame when placing startpoint nowadays. (Result is the replay site makes it look at though 90% of people use arm!)

currently runtime data is not interpreted, work is scheduled for winter/spring

(2) The rank reported for each player is the one reported by the lobby and not the chrank used by the autohost in its balancing.

That information is not available from the replay. Unfortunately that makes the comparision between lobby-rank and trueskill/elo-rank | balancing wrong :(
There is currently no way to get this information. Crazy idea: someone (not me) writes a (IRC|lobby)bot, that PMs all autohosts and requests the info for each playeraccount :D

[Silentwings]

I thought it would be something like that

I would say not to bother solving those bugs for now - because largely down to yours and vbs awesome work we will one day have trueskill balancing in spads, making both those bugs irrelevent. For what it's worth I think the replay site has already improved balancing because it gives us very handy data when doing chranks.

[klapmongool]

Hmm trueskill is weird? It goes down while I won matches and up when I lost?

It was a series of five 1v1 matches. I won the first three and lost the last two. The first match trueskill went up, the 2nd and 3rd it went down and the 4th and the 5th it went up again.

http://replays.springrts.com/player/3780/

[dansan]

lol - that sounds reversed :D

I never observed TS for 1v1, just Elo. So it's practically "untested"... maybe I did reverse some calculation... I'll check it out later.

[Silentwings]

I've checked through quite a lot of the trueskill data for big games whilst making chranks and I've never seen it behave badly on those (but I also looked through the games klap mentions and he is right that it is behaving wrongly in those - so maybe this issue is restricted to 1v1s).

[HectorMeyer]

awesome

[muckl]

(2) The rank reported for each player is the one reported by the lobby and not the chrank used by the autohost in its balancing.

why not the same way like the trueskill values are sent to the spads?
there could be a feedback from spads

over IRC bridge it would go the wrong way in my mind

[dansan]

Implementing lobby or irc protocol on the replays-site just to fetch the chranks (a workaround for s/t I consider broken) from the autohosts... to much work for something that was meant as a simple comparison and is broken bc I didn't consider chranks... there is a LOT of more important work queued up for the site :/

But... hmm... you want to use it to fix your (ch)ranks? That'd make sense.
... still... feels like a workaround of a workaround...

... please feel free to send in a patch, and I'll gladly add it :)

[muckl]

how is the data transfer done then?
that the spads gets the trueskill infos?

[dansan]

SPADS pulls the data from the replay site through a xmlrpc api

As chranks are local to autohosts and can change between matches, this information must be retrieved at the time of the creation of the replay.
IMO this can only be done by either having the replay-site pull it from the autohost via lobby/irc-protocol directly after receiving a replay, or by spads transmitting it together with the replay (extending the endGameCommand).

[muckl]

or by spads transmitting it together with the replay (extending the endGameCommand).

yep that direction i was thinking

or like to post the vars as parameters to an php script

but as u say.. there are sure more important things to do

[dansan]

If it were transmitted via endGameCommand as a cmdline switch to replay_upload.py it'd be using an existing interface. But php would work too.

Anyway: biggest work is to patch spads to collect the smurfs that participated in the [to-be-]uploaded match, and [extend endGameCommand | upload via php].

[bibim]

Dansan, if you want to receive the chrank values for each battle, I think the easiest way would be to patch SPADS to write them in the startscript, just like it writes the players account IDs currently.

[dansan]

How to get the start script? Hmm.. I read something about it not so long ago... cannot remember... You mean script.txt yes?

I'd be happy to show chranks on the replays-site, but I won't be patching spads - maybe muckl wants to?

[Silentwings]

Are there instructions on where to get the source and how to compile spads anywhere? I've never tried to do modify it before but i'd be happy to do simple stuff to it.

[bibim]

How to get the start script? Hmm.. I read something about it not so long ago... cannot remember... You mean script.txt yes?

The startscript is included in the replay, I think it's how you retrieve the account IDs currently?

Concerning smurf data, imo there are 2 ways to implement it:
1) relying on each SPADS individual smurf (ip address) database. Which means extending endGameCommand or adding a new interface just for this, like you explained.
2) relying on the central lobby database I'm working on. The smurf detection part is actually finished, and writing a daemon which would propagate the account unification/split events to your site shouldn't be hard.

Imo, 2) is much better than 1) for multiple reasons:
- the SPADS smurf database isn't available if the autohost admin didn't activate account data retention (disabled by default)
- false smurf positives (players sharing same IP) aren't detected by SPADS
- SPADS smurf databases are independent and configured by each autohost admin, and as such they could be conflicting or at least not as exhaustive as the central one
- with 2), you don't need to add several "trusted" entities (SPADS instances) which are allowed to unify/split accounts. Only the central database has to be trusted.

However, as we discussed earlier, we have to agree on a policy regarding smurfing, anonymity, and public ratings.
I'm also working on an automatic battle history site (not based on replays) since quite a long time, with ratings which could be used by SPADS for balancing. I'm not sure I will finish it one day, that's why I don't speak much about it, but I still have hope...
I chose to allow players to keep some anonymity (i.e., don't show alternate accounts) and stay away of competitive ratings (i.e., don't show precise rank publicly) by default. Each player would be able to change this behavior by modifying the corresponding option for his account.
These choices are totally open to discussion, however I think both sites should be consistent and use same policy.

[bibim]

Are there instructions on where to get the source and how to compile spads anywhere? I've never tried to do modify it before but i'd be happy to do simple stuff to it.

You don't need to compile it, the source scripts are directly interpreted.
Sources are available on SPADS repository, but the easiest way to get all required files locally is to follow the instructions in the INSTALL file of SPADS installation package.

Unfortunately source code is poorly documented, but don't hesitate to contact me if you have questions about it.

[dansan]

The startscript is included in the replay, I think it's how you retrieve the account IDs currently?

The start script at the start of a replay is written by spring. I never realized that this information is from spads. The info I see as springs start script is the one you see, if you run "parse_demo_file.py replay.sdf". Is this what you mean?
If that is it, then that would be just perfect :)