SLDB: Personal TrueSkill history graphs

SLDB: Personal TrueSkill history graphs

Discuss development of lobby clients, server, autohosts and auto-download software.

Moderators: Moderators, Lobby Developers

User avatar
bibim
Lobby Developer
Posts: 913
Joined: 06 Dec 2007, 11:12

SLDB: Personal TrueSkill history graphs

Post by bibim »

I added an experimental functionality in SLDB to allow players to generate their TrueSkill history graphs.
You just need to say "!skillGraph <modShortName>" to the bot named SLDB on Spring lobby server.

<modShortName> can take following values:
- BA (Balanced Annihilation)
- EVO (EvolutionRTS)
- KP (Kernel Panic)
- NOTA
- S1944 (Spring: 1944)
- TA (Tech Annihilation)
- XTA
- ZK (Zero-K)

For example "!skillGraph BA" gives this result for me.

Image

The blue line is what is called "estimated skill" in SLDB ("mu" in TrueSkill papers), and the blue area represents the range where the real skill has 99.99% chance to be (mu +- 3*sigma). The bottom limit of this area is what is called "trusted skill" in SLDB ("conservative skill" in TrueSkill papers).

This could be integrated in Spring replay site if needed?
Last edited by bibim on 05 Nov 2014, 18:38, edited 1 time in total.
User avatar
Jools
XTA Developer
Posts: 2812
Joined: 23 Feb 2009, 16:29

Re: SLDB: Personal TrueSkill history graphs

Post by Jools »

bibim wrote: This could be integrated in Spring replay site if needed?
That would be very nice.
bibim wrote: represents the range where the real skill has 99.99% chance to be (mu +- 3*sigma).
Except 3 standard deviations account for a certainty of 99.73 %: From http://en.wikipedia.org/wiki/68%E2%80%9 ... 399.7_rule:

Image
User avatar
bibim
Lobby Developer
Posts: 913
Joined: 06 Dec 2007, 11:12

Re: SLDB: Personal TrueSkill history graphs

Post by bibim »

Jools wrote:Except 3 standard deviations account for a certainty of 99.73 %
True, I mixed up with 4 * sigma which is 99.99%.
It doesn't change anything in the reasoning though, it's still very very likely that the real skill is in this range.
User avatar
Jools
XTA Developer
Posts: 2812
Joined: 23 Feb 2009, 16:29

Re: SLDB: Personal TrueSkill history graphs

Post by Jools »

It's a very cool thing, I would see it natural to have those graphs on the personal pages of the replay site, hope it's not that hard to include there.

Are those the exact values or the public ones by the way? I assume they are the exact ones. If they are exact ones, then it overrides the privacy mode, but to be honest, the privacy is not that well guarded anyway unless the community is large, as you can work out a good approximation of skill by some permutations of say 2v2 matches.
Last edited by Jools on 22 Oct 2014, 17:53, edited 1 time in total.
User avatar
bibim
Lobby Developer
Posts: 913
Joined: 06 Dec 2007, 11:12

Re: SLDB: Personal TrueSkill history graphs

Post by bibim »

These values are the exact ones, as they are supposed to be requested by the player and shown to him only, so for me it doesn't break privacy mode.
User avatar
very_bad_soldier
Posts: 1385
Joined: 20 Feb 2007, 01:10

Re: SLDB: Personal TrueSkill history graphs

Post by very_bad_soldier »

Yay, very cool, thank you!
[LOeT]Brenner
Posts: 25
Joined: 19 Oct 2011, 01:33

Re: SLDB: Personal TrueSkill history graphs

Post by [LOeT]Brenner »

Very nice! Thanks a lot!
muckl
Posts: 151
Joined: 30 Aug 2010, 07:18

Re: SLDB: Personal TrueSkill history graphs

Post by muckl »

thanks!!
dansan
Server Owner & Developer
Posts: 1203
Joined: 29 May 2010, 23:40

Re: SLDB: Personal TrueSkill history graphs

Post by dansan »

Very nice!

I'd love to make those graphs available on the replays site.
For those with privacy_mode=0 (disabled) they could be public and for everybody else private (when logged in, you can see only your own graph).

Bibim: PM me the details about the API please. I'll be traveling for the next two weeks though, so don't expect me to answer ^.^
User avatar
Silentwings
Moderator
Posts: 3666
Joined: 25 Oct 2008, 00:23

Re: SLDB: Personal TrueSkill history graphs

Post by Silentwings »

Great work :)
dansan
Server Owner & Developer
Posts: 1203
Joined: 29 May 2010, 23:40

Re: SLDB: Personal TrueSkill history graphs

Post by dansan »

Bibim made the graphs accessible through the SLDB API (actually a while ago... I'm a little slow...). Thank you! The replays site has those on each players page now :)

For example: http://replays.springrts.com/player/72351/

Loading of graphs is done as late as possible (for example no real thumbs) to lower the load on SLDB and loading time of the web pages... could be remedied with some JS-voodoo... Images are cached for 30d on the webserver. Privacy is protected. So you either have to login to see your TS history or disable the privacy to make your graphs public.

... there are "thumbs" for graphs that do not actually exist - only those game/match_type-combinations that are listed in "Win/loss" really have graphs... I think I'll move those buttons to a fourth column in the Rating-table above it...
User avatar
Jools
XTA Developer
Posts: 2812
Joined: 23 Feb 2009, 16:29

Re: SLDB: Personal TrueSkill history graphs

Post by Jools »

Very nice, thanks for the work.
User avatar
knorke
Posts: 7971
Joined: 22 Feb 2006, 01:02

Re: SLDB: Personal TrueSkill history graphs

Post by knorke »

Cool. Is there a list of all players who have allowed others to view their stats?
Might be interessting to compare graphs.

So far all graphs I've seen start with "narrowing down" and then remain fairly constant. Maybe because it is all long-time players whose skill does not change anymore.
Would be interessting to see graph of some new player who started at beginning of logging, if there is an upwards trend.

Maybe played games could be shown in skill graph too? (As dots on the timeline?)
Otherwise it is hard to tell if a skill-graph remained constant because skill stayed the same or if simply no games were played in that time.
User avatar
bibim
Lobby Developer
Posts: 913
Joined: 06 Dec 2007, 11:12

Re: SLDB: Personal TrueSkill history graphs

Post by bibim »

knorke wrote:it is hard to tell if a skill-graph remained constant because skill stayed the same or if simply no games were played in that time.
If no game is played during a month for a given player and a given mod/game, his uncertainty increases and his estimated skill slightly decreases due to inactivity penalties:
bibim wrote:the inactivity penalties are applied each month to all high-skill with low uncertainty players who almost didn't play in the month. These penalties are quite light however, because the goal is not to really penalize them but actualy modelize their lost of skill and the increase of their skill uncertainty. Players recovering from inactivity periods should be able to recover their original rating quite fast thanks to this uncertainty increase. This system also prevents old inactive players to stay in the top part of the leaderboards, while not playing anymore.
You can see these inactivity penalties in action at the end of my skill graph for example, in the original post of this thread. So if a skill graph remains constant, it's because the estimated skill stayed the same.
User avatar
Silentwings
Moderator
Posts: 3666
Joined: 25 Oct 2008, 00:23

Re: SLDB: Personal TrueSkill history graphs

Post by Silentwings »

So far all graphs I've seen start with "narrowing down" and then remain fairly constant.
Probably what you are seeing here is the initial period where TS quickly learns what an (already developed) players skill should be. If you pick a graph of someone who started playing after TS was introduced, and who then subsequently improved slowly to their "developed" level, you'll likely see a different pattern.
User avatar
Jools
XTA Developer
Posts: 2812
Joined: 23 Feb 2009, 16:29

Re: SLDB: Personal TrueSkill history graphs

Post by Jools »

It follows from some basic universal principle that variation decreases when sample amount increases.
User avatar
knorke
Posts: 7971
Joined: 22 Feb 2006, 01:02

Re: SLDB: Personal TrueSkill history graphs

Post by knorke »

Yes, I understood that already. Hence my question was:
"Would be interessting to see graph of some new player who started at beginning of logging, if there is an upwards trend."

I want to see a graph that does not follow this appearently common pattern:
Either byy someone posting his own graph, or through a list of "public" players or maybe anonym example with playername removed.
User avatar
Silentwings
Moderator
Posts: 3666
Joined: 25 Oct 2008, 00:23

Re: SLDB: Personal TrueSkill history graphs

Post by Silentwings »

You can find many such examples using replay site; look for new-ish players. E.g. "BA Team" graph for http://replays.springrts.com/player/948465/ (I hope he doesn't mind being an example).
User avatar
albator
Posts: 866
Joined: 14 Jan 2009, 14:20

Re: SLDB: Personal TrueSkill history graphs

Post by albator »

Something strange happens.

I used to be 2nd at team ffa hof even i have not played for > 6 months. Then last month I played one game and I went down 8th....

Is it a bug or is it due to the fact that abma erase 50 of my account ? http://springrts.com/phpbb/viewtopic.php?f=11&t=32724
User avatar
bibim
Lobby Developer
Posts: 913
Joined: 06 Dec 2007, 11:12

Re: SLDB: Personal TrueSkill history graphs

Post by bibim »

albator wrote:Something strange happens.

I used to be 2nd at team ffa hof even i have not played for > 6 months. Then last month I played one game and I went down 8th....
Here are the details of your TeamFFA ranks for current year, can you be more specific about what you find strange?

Code: Select all

jan: 2nd
feb: 3rd
mar: 3rd
apr: 3rd
may: 4th (6 inactivity penalties)
jun: 4th (+3 inactivity penalties)
jul: 5th (+8 inactivity penalties)
aug: 5th (+8 inactivity penalties)
sep: 6th (+8 inactivity penalties)
oct: 6th (+4 inactivity penalties)
nov: 8th
dec: 8th
albator wrote:Is it a bug or is it due to the fact that abma erase 50 of my account ? http://springrts.com/phpbb/viewtopic.php?f=11&t=32724
SLDB uses its own account database, so uberserver accounts deletion has no effect on SLDB data such as TrueSkill rankings.
Post Reply

Return to “Lobby Clients & Server”