SLDB: Personal TrueSkill history graphs

[bibim]

I added an experimental functionality in SLDB to allow players to generate their TrueSkill history graphs.
You just need to say "!skillGraph <modShortName>" to the bot named SLDB on Spring lobby server.

<modShortName> can take following values:
- BA (Balanced Annihilation)
- EVO (EvolutionRTS)
- KP (Kernel Panic)
- NOTA
- S1944 (Spring: 1944)
- TA (Tech Annihilation)
- XTA
- ZK (Zero-K)

For example "!skillGraph BA" gives this result for me.



The blue line is what is called "estimated skill" in SLDB ("mu" in TrueSkill papers), and the blue area represents the range where the real skill has 99.99% chance to be (mu +- 3*sigma). The bottom limit of this area is what is called "trusted skill" in SLDB ("conservative skill" in TrueSkill papers).

This could be integrated in Spring replay site if needed?

[Jools]

This could be integrated in Spring replay site if needed?

That would be very nice.

represents the range where the real skill has 99.99% chance to be (mu +- 3*sigma).

Except 3 standard deviations account for a certainty of 99.73 %: From http://en.wikipedia.org/wiki/68%E2%80%9 ... 399.7_rule:

[bibim]

Except 3 standard deviations account for a certainty of 99.73 %

True, I mixed up with 4 * sigma which is 99.99%.
It doesn't change anything in the reasoning though, it's still very very likely that the real skill is in this range.

[Jools]

It's a very cool thing, I would see it natural to have those graphs on the personal pages of the replay site, hope it's not that hard to include there.

Are those the exact values or the public ones by the way? I assume they are the exact ones. If they are exact ones, then it overrides the privacy mode, but to be honest, the privacy is not that well guarded anyway unless the community is large, as you can work out a good approximation of skill by some permutations of say 2v2 matches.

[bibim]

These values are the exact ones, as they are supposed to be requested by the player and shown to him only, so for me it doesn't break privacy mode.

[very_bad_soldier]

Yay, very cool, thank you!

[[LOeT]Brenner]

Very nice! Thanks a lot!

[muckl]

thanks!!

[dansan]

Very nice!

I'd love to make those graphs available on the replays site.
For those with privacy_mode=0 (disabled) they could be public and for everybody else private (when logged in, you can see only your own graph).

Bibim: PM me the details about the API please. I'll be traveling for the next two weeks though, so don't expect me to answer ^.^

[Silentwings]

Great work :)

[dansan]

Bibim made the graphs accessible through the SLDB API (actually a while ago... I'm a little slow...). Thank you! The replays site has those on each players page now :)

For example: http://replays.springrts.com/player/72351/

Loading of graphs is done as late as possible (for example no real thumbs) to lower the load on SLDB and loading time of the web pages... could be remedied with some JS-voodoo... Images are cached for 30d on the webserver. Privacy is protected. So you either have to login to see your TS history or disable the privacy to make your graphs public.

... there are "thumbs" for graphs that do not actually exist - only those game/match_type-combinations that are listed in "Win/loss" really have graphs... I think I'll move those buttons to a fourth column in the Rating-table above it...

[Jools]

Very nice, thanks for the work.

[knorke]

Cool. Is there a list of all players who have allowed others to view their stats?
Might be interessting to compare graphs.

So far all graphs I've seen start with "narrowing down" and then remain fairly constant. Maybe because it is all long-time players whose skill does not change anymore.
Would be interessting to see graph of some new player who started at beginning of logging, if there is an upwards trend.

Maybe played games could be shown in skill graph too? (As dots on the timeline?)
Otherwise it is hard to tell if a skill-graph remained constant because skill stayed the same or if simply no games were played in that time.

[bibim]

it is hard to tell if a skill-graph remained constant because skill stayed the same or if simply no games were played in that time.

If no game is played during a month for a given player and a given mod/game, his uncertainty increases and his estimated skill slightly decreases due to inactivity penalties:

the inactivity penalties are applied each month to all high-skill with low uncertainty players who almost didn't play in the month. These penalties are quite light however, because the goal is not to really penalize them but actualy modelize their lost of skill and the increase of their skill uncertainty. Players recovering from inactivity periods should be able to recover their original rating quite fast thanks to this uncertainty increase. This system also prevents old inactive players to stay in the top part of the leaderboards, while not playing anymore.

You can see these inactivity penalties in action at the end of my skill graph for example, in the original post of this thread. So if a skill graph remains constant, it's because the estimated skill stayed the same.

[Silentwings]

So far all graphs I've seen start with "narrowing down" and then remain fairly constant.

Probably what you are seeing here is the initial period where TS quickly learns what an (already developed) players skill should be. If you pick a graph of someone who started playing after TS was introduced, and who then subsequently improved slowly to their "developed" level, you'll likely see a different pattern.

[Jools]

It follows from some basic universal principle that variation decreases when sample amount increases.

[knorke]

Yes, I understood that already. Hence my question was:
"Would be interessting to see graph of some new player who started at beginning of logging, if there is an upwards trend."

I want to see a graph that does not follow this appearently common pattern:
Either byy someone posting his own graph, or through a list of "public" players or maybe anonym example with playername removed.

[Silentwings]

You can find many such examples using replay site; look for new-ish players. E.g. "BA Team" graph for http://replays.springrts.com/player/948465/ (I hope he doesn't mind being an example).

[albator]

Something strange happens.

I used to be 2nd at team ffa hof even i have not played for > 6 months. Then last month I played one game and I went down 8th....

Is it a bug or is it due to the fact that abma erase 50 of my account ? http://springrts.com/phpbb/viewtopic.php?f=11&t=32724

[bibim]

Something strange happens.

I used to be 2nd at team ffa hof even i have not played for > 6 months. Then last month I played one game and I went down 8th....

Here are the details of your TeamFFA ranks for current year, can you be more specific about what you find strange?

Code: Select all

jan: 2nd
feb: 3rd
mar: 3rd
apr: 3rd
may: 4th (6 inactivity penalties)
jun: 4th (+3 inactivity penalties)
jul: 5th (+8 inactivity penalties)
aug: 5th (+8 inactivity penalties)
sep: 6th (+8 inactivity penalties)
oct: 6th (+4 inactivity penalties)
nov: 8th
dec: 8th

Is it a bug or is it due to the fact that abma erase 50 of my account ? http://springrts.com/phpbb/viewtopic.php?f=11&t=32724

SLDB uses its own account database, so uberserver accounts deletion has no effect on SLDB data such as TrueSkill rankings.