Elo, Glicko and Trueskill ratings on replays.springrts.com

[dansan]

It does kind of take prior performance into account, but without keeping a history. It just wants to know how correct the prior skill assessment was.

Input parameters to the algorithm are just two: the actual "mu" (the skill value) and "sigma" (a deviation parameter). Both change with each match:
Mu is adjusted depending on win/loss and relative to the opponents strength. How much it is adjusted depends on sigma.
Sigma basically indicated how "certain" the algo is of your skill value. If you have performed consistent with the estimated skill, then sigma is lowered, if your performance fluctuates sigma rises. This is the primary mechanism to converge on the correct value, but still be agile to adjust it if a players skill changes (long offline for example). It comes down to "your skill is between mu-sigma and mu+sigma". Elo has a fixed "k-factor" that indicates the deviation. It assumes a player always performs consistent... it was made for the chess league...

For us this means: if a manual chrank is made, mu will be set to that value and sigma should be medium to high (a newbies value), so the algo can quickly adjust, starting from the admins estimate. Hmm... or should it be low, trusting in the admins judgment? Hmm... I guess medium is safe :D

If the admin knows exactly who he's dealing with, than it's still important to make the account_unification().

[dansan]

OMG... my posts are so long... here the tl;dr:

If it can just relearn at any time from a given value

Yes, it always does, it keeps no record except current mu and sigma.

without wild oscillations or other screw ups then, sure.

low sigma -> low oscillation

[dansan]

XML-RPC interface to request and set ratings implemented

Code: Select all

get_rating_single_user(accountid, game, match_type) -> rating(float)
get_rating_multiple_users(accountids, game, match_type) -> [[accountid, rating], [accountid, rating], ...]
set_rating(accountid, game, match_type, rating, username, password, admin_accountid) -> rating(float)

If the functions don't like the input, they return a string: "-1 describing the problem"

Code: Select all

>>> import xmlrpclib
>>> XMLRPC_URL = "http://replays-test.springrts.com/xmlrpc/"
>>> rpc_srv = xmlrpclib.ServerProxy(XMLRPC_URL)
>>> rpc_srv.get_rating_multiple_users([168247, 65787, 194470], "BA", "T")
[[168247, 27.7860165582782], [65787, 25.0], [194470, 30.3310634043166]]
>>> rpc_srv.get_rating_multiple_users([168247, 65787, 194470], "BA", "F")
[[168247, 29.0536634947103], [65787, 25.0], [194470, 25.0]]
>>> rpc_srv.get_rating_multiple_users([168247, 65787, 194470], "BA", "1")
[[168247, 1482.97396552475], [65787, 1500.0], [194470, 1530.72668017126]]

>>> rpc_srv.set_rating(65787, "BA", "1", 1666, "Dansan", "xxxxxxxx", 130601)
1666.0
>>> rpc_srv.get_rating_single_user(65787, "BA", "1")
1666.0

Current "username"s allowed in set_rating(): "bibimDemos", "nixbot", "PrincessHost", "[ACE]FaFa_BOT", "[LOeT]upload", "[semprini]Replaybot", "[x]TheHost", "Dansan", "vbs" (the last two for testing & development). vbs: if you wish to use another account, please PM me.
"password" is their password
"admin_accountid" is the accountid of the admin that called the "!chrank" (for logging purposes, as I guess this feature will be used one day from different autohosts with lots of admins) -> list of manual changes made: last link on Rating-line in side panel.

For testing of set_rating() please use
XMLRPC_URL = "http://replays-test.springrts.com/xmlrpc/".

Account-unification is missing. It will be something like unify_accounts(accountid, accountid) -> accountid with the same restrictions as set_rating().

[dansan]

XML-RPC interface for account unification and rating transfer implemented

Code: Select all

unify_accounts(accountid1, accountid2, username, password, admin_accountid) -> accountid_of_oldest_account

Code: Select all

>>> import xmlrpclib
>>> XMLRPC_URL = "http://replays-test.springrts.com/xmlrpc/"
>>> rpc_srv = xmlrpclib.ServerProxy(XMLRPC_URL)
>>> rpc_srv.unify_accounts(125476, 65787, "Dansan", "xxxxxxxxxx", 130601)
65787
>>> rpc_srv.unify_accounts(125476, 198943, "Dansan", "xxxxxxxxxx", 130601)
65787

"Account unification history" page shows log -> click "Smurfs" on Player-line in side panel.

[Jools]

Yeah, that's good. Account joining should be transparent, otherwise some admin could join a person's account with some noob to get the skill down. Or something.

[Jools]

Spring Replays
Who needs a VCR?

This is funny.

Anyway, I didn't find myself in the list of players . Do I have to register or something to get there?

[knorke]

Jools:
http://replays.springrts.com/player/84842/
---
When searching for "xta" it says:

126 replays matching your search
...
list
...
Page 1 of 3
50 of 126 replays

But when I want to go to page 2 it says
0 replays matching your search
with a search formula under it.

[dansan]

Yeah, that's good. Account joining should be transparent, otherwise some admin could join a person's account with some noob to get the skill down. Or something.

Transparency is good vs both the smurf-problem and admin-abuse(-accusation) I hope... The ratings of the unified users are also unified, and always the highest value is used.

I didn't find myself

Sorting with non-alphanum characters is tricky... I didn't invest time into this... I think it makes more sense to make the search results better (issue 29), so you can just search for your name.

searching [..] when I want to go to page 2 it says 0 replays matching your search

lol - I never noticed, that the pagination of the search results doesn't work at all. The table-class uses GET for pagination, but I use POST for the search parameters... issue 30.
In the meantime you can use the game-type-link on the lower right side panel (XTA).

[Jools]

Ok, and just for the record: I don't think any of our admins would do such a thing, but it's good to have it transparent anyway. We have very good admins on the site.

The second issue: so it's a problem with the "[" character. But why are other PRO:s listed?

[dansan]

so it's a problem with the "[" character. But why are other PRO:s listed?

I see you on page 22 together with the other PROs.

[dansan]

New All Players page is quick and searchable :)

[Jazcash]

Blocks of text are so much prettier when the lines are justified ^_^

Btw, loving the progress of this. Even though I don't really play anymore, I check the site almost daily just to see what swanky new features and updates there are

[knorke]

just noticed the spoiler "show winner" function is a bit useless when it alreadys shows new and old elo rating

[dansan]

just noticed the spoiler "show winner" function is a bit useless when it alreadys shows new and old elo rating ;)

Oh right :) ... changed it.

[Jools]

As I wrote in PM, what to do in situations where user forfeits match (e.g. by DGunning opponent Commander) but where Spring still records him as winner?

Can the Engine maybe be improved better determine winner?
NB: I capitalise engine because of it's divine meaning

[gajop]

that's news to me, i thought dguning an opponent commander makes you lose the match, not win it

[knorke]

it is games job to determine winner/tie:
http://springrts.com/wiki/Lua_SyncedCtrl#Game_End

[Jazcash]

Elo, Glicko and Trueskill should be added to SPADS as optional balancing methods. Should be interesting to see how they fair up to ingame time ranks

[Silentwings]

Yes - all the BA autohost owners are keen to add it.

[dansan]

Elo, Glicko and Trueskill should be added to SPADS as optional balancing methods. Should be interesting to see how they fair up to ingame time ranks :lol:

vbs is working on that.

I haven't done any statistics yet, but if you're interested in a comparison, you can simply browse the games, and see if the team with the higher rating did win or not. When I'm bored I observe the RSS feed :D

My observation of team games: if the difference was >= 10% then >2/3 of the matches were won by the team with the higher rating. At 5% difference the chance is almost 1:1. In such a case it can happen, that if the team with the lower rating won, its sum of the ratings is the higher one afterwards. That means it was almost balanced?