Page 3 of 6
Re: Elo, Glicko and Trueskill ratings on replays.springrts.c
Posted: 20 Nov 2012, 09:08
by dansan
It does kind of take prior performance into account, but without keeping a history. It just wants to know how correct the prior skill assessment was.
Input parameters to the algorithm are just two: the actual "mu" (the skill value) and "sigma" (a deviation parameter). Both change with each match:
Mu is adjusted depending on win/loss and relative to the opponents strength. How much it is adjusted depends on sigma.
Sigma basically indicated how "certain" the algo is of your skill value. If you have performed consistent with the estimated skill, then sigma is lowered, if your performance fluctuates sigma rises. This is the primary mechanism to converge on the correct value, but still be agile to adjust it if a players skill changes (long offline for example). It comes down to "your skill is between mu-sigma and mu+sigma". Elo has a fixed "k-factor" that indicates the deviation. It assumes a player always performs consistent... it was made for the chess league...
For us this means: if a manual chrank is made, mu will be set to that value and sigma should be medium to high (a newbies value), so the algo can quickly adjust, starting from the admins estimate. Hmm... or should it be low, trusting in the admins judgment? Hmm... I guess medium is safe :D
If the admin knows exactly who he's dealing with, than it's still important to make the account_unification().
Re: Elo, Glicko and Trueskill ratings on replays.springrts.c
Posted: 20 Nov 2012, 09:12
by dansan
OMG... my posts are so long... here the tl;dr:
Silentwings wrote:If it can just relearn at any time from a given value
Yes, it always does, it keeps no record except current mu and sigma.
Silentwings wrote:without wild oscillations or other screw ups then, sure.
low sigma -> low oscillation
Re: Elo, Glicko and Trueskill ratings on replays.springrts.c
Posted: 23 Nov 2012, 00:29
by dansan
XML-RPC interface to request and set ratings implemented
Code: Select all
get_rating_single_user(accountid, game, match_type) -> rating(float)
get_rating_multiple_users(accountids, game, match_type) -> [[accountid, rating], [accountid, rating], ...]
set_rating(accountid, game, match_type, rating, username, password, admin_accountid) -> rating(float)
If the functions don't like the input, they return a string: "-1 describing the problem"
Code: Select all
>>> import xmlrpclib
>>> XMLRPC_URL = "http://replays-test.springrts.com/xmlrpc/"
>>> rpc_srv = xmlrpclib.ServerProxy(XMLRPC_URL)
>>> rpc_srv.get_rating_multiple_users([168247, 65787, 194470], "BA", "T")
[[168247, 27.7860165582782], [65787, 25.0], [194470, 30.3310634043166]]
>>> rpc_srv.get_rating_multiple_users([168247, 65787, 194470], "BA", "F")
[[168247, 29.0536634947103], [65787, 25.0], [194470, 25.0]]
>>> rpc_srv.get_rating_multiple_users([168247, 65787, 194470], "BA", "1")
[[168247, 1482.97396552475], [65787, 1500.0], [194470, 1530.72668017126]]
>>> rpc_srv.set_rating(65787, "BA", "1", 1666, "Dansan", "xxxxxxxx", 130601)
1666.0
>>> rpc_srv.get_rating_single_user(65787, "BA", "1")
1666.0
Current "username"s allowed in set_rating(): "bibimDemos", "nixbot", "PrincessHost", "[ACE]FaFa_BOT", "[LOeT]upload", "[semprini]Replaybot", "[x]TheHost", "Dansan", "vbs" (the last two for testing & development). vbs: if you wish to use another account, please PM me.
"password" is their password
"admin_accountid" is the accountid of the admin that called the "!chrank" (for logging purposes, as I guess this feature will be used one day from different autohosts with lots of admins) -> list of manual changes made:
last link on Rating-line in side panel.
For testing of set_rating() please use
XMLRPC_URL = "http://replays
-test.springrts.com/xmlrpc/".
Account-unification is missing. It will be something like
unify_accounts(accountid, accountid) -> accountid with the same restrictions as set_rating().
Re: Elo, Glicko and Trueskill ratings on replays.springrts.c
Posted: 23 Nov 2012, 12:07
by dansan
XML-RPC interface for account unification and
rating transfer implemented
Code: Select all
unify_accounts(accountid1, accountid2, username, password, admin_accountid) -> accountid_of_oldest_account
Code: Select all
>>> import xmlrpclib
>>> XMLRPC_URL = "http://replays-test.springrts.com/xmlrpc/"
>>> rpc_srv = xmlrpclib.ServerProxy(XMLRPC_URL)
>>> rpc_srv.unify_accounts(125476, 65787, "Dansan", "xxxxxxxxxx", 130601)
65787
>>> rpc_srv.unify_accounts(125476, 198943, "Dansan", "xxxxxxxxxx", 130601)
65787
"Account unification history" page shows log -> click
"Smurfs" on Player-line in side panel.
Re: Elo, Glicko and Trueskill ratings on replays.springrts.c
Posted: 23 Nov 2012, 13:24
by Jools
Yeah, that's good. Account joining should be transparent, otherwise some admin could join a person's account with some noob to get the skill down. Or something.
Re: Elo, Glicko and Trueskill ratings on replays.springrts.c
Posted: 23 Nov 2012, 13:36
by Jools
Spring Replays
Who needs a VCR?
This is funny.
Anyway, I didn't find myself in the
list of players . Do I have to register or something to get there?
Re: Elo, Glicko and Trueskill ratings on replays.springrts.c
Posted: 23 Nov 2012, 14:56
by knorke
Jools:
http://replays.springrts.com/player/84842/
---
When searching for "xta" it says:
126 replays matching your search
...
list
...
Page 1 of 3
50 of 126 replays
But when I want to go to page 2 it says
0 replays matching your search
with a search formula under it.
Re: Elo, Glicko and Trueskill ratings on replays.springrts.c
Posted: 23 Nov 2012, 15:39
by dansan
Jools wrote:Yeah, that's good. Account joining should be transparent, otherwise some admin could join a person's account with some noob to get the skill down. Or something.
Transparency is good vs both the smurf-problem and admin-abuse(-accusation) I hope... The ratings of the unified users are also unified, and always
the highest value is used.
Jools wrote:I didn't find myself
Sorting with non-alphanum characters is tricky... I didn't invest time into this... I think it makes more sense to make the search results better (
issue 29), so you can just search for your name.
knorke wrote:searching [..] when I want to go to page 2 it says 0 replays matching your search
lol - I never noticed, that the pagination of the search results doesn't work at all. The table-class uses GET for pagination, but I use POST for the search parameters...
issue 30.
In the meantime you can use the game-type-link on the lower right side panel (
XTA).
Re: Elo, Glicko and Trueskill ratings on replays.springrts.c
Posted: 23 Nov 2012, 20:36
by Jools
Ok, and just for the record: I don't think any of our admins would do such a thing, but it's good to have it transparent anyway. We have very good admins on the site.
The second issue: so it's a problem with the "[" character. But why are other PRO:s listed?
Re: Elo, Glicko and Trueskill ratings on replays.springrts.c
Posted: 24 Nov 2012, 11:26
by dansan
Jools wrote:so it's a problem with the "[" character. But why are other PRO:s listed?
I see you on page 22 together with the other PROs.
Re: Elo, Glicko and Trueskill ratings on replays.springrts.c
Posted: 24 Nov 2012, 12:06
by dansan
New
All Players page is quick and searchable :)
Re: Elo, Glicko and Trueskill ratings on replays.springrts.c
Posted: 24 Nov 2012, 15:15
by Jazcash
Blocks of text are so much prettier when the lines are justified ^_^
Btw, loving the progress of this. Even though I don't really play anymore, I check the site almost daily just to see what swanky new features and updates there are
Re: Elo, Glicko and Trueskill ratings on replays.springrts.c
Posted: 26 Nov 2012, 13:27
by knorke
just noticed the spoiler "show winner" function is a bit useless when it alreadys shows new and old elo rating
Re: Elo, Glicko and Trueskill ratings on replays.springrts.c
Posted: 27 Nov 2012, 11:56
by dansan
knorke wrote:just noticed the spoiler "show winner" function is a bit useless when it alreadys shows new and old elo rating ;)
Oh right :) ... changed it.
Re: Elo, Glicko and Trueskill ratings on replays.springrts.c
Posted: 27 Nov 2012, 12:58
by Jools
As I wrote in PM, what to do in situations where user forfeits match (e.g. by DGunning opponent Commander) but where Spring still records him as winner?
Can the Engine maybe be improved better determine winner?
NB: I capitalise engine because of it's divine meaning
Re: Elo, Glicko and Trueskill ratings on replays.springrts.c
Posted: 27 Nov 2012, 15:33
by gajop
that's news to me, i thought dguning an opponent commander makes you lose the match, not win it
Re: Elo, Glicko and Trueskill ratings on replays.springrts.c
Posted: 27 Nov 2012, 15:56
by knorke
Re: Elo, Glicko and Trueskill ratings on replays.springrts.c
Posted: 27 Nov 2012, 16:57
by Jazcash
Elo, Glicko and Trueskill should be added to SPADS as optional balancing methods. Should be interesting to see how they fair up to ingame time ranks
Re: Elo, Glicko and Trueskill ratings on replays.springrts.c
Posted: 27 Nov 2012, 17:20
by Silentwings
Yes - all the BA autohost owners are keen to add it.
Re: Elo, Glicko and Trueskill ratings on replays.springrts.c
Posted: 27 Nov 2012, 21:44
by dansan
Jazcash wrote:Elo, Glicko and Trueskill should be added to SPADS as optional balancing methods. Should be interesting to see how they fair up to ingame time ranks :lol:
vbs is working on that.
I haven't done any statistics yet, but if you're interested in a comparison, you can simply browse the games, and see if the team with the higher rating did win or not. When I'm bored I observe the RSS feed :D
My observation of team games: if the difference was >= 10% then >2/3 of the matches were won by the team with the higher rating. At 5% difference the chance is almost 1:1. In such a case it can happen, that if the team with the lower rating won, its sum of the ratings is the higher one afterwards. That means it was almost balanced?