Proper ranks

Proper ranks

Various things about Spring that do not fit in any of the other forums listed below, including forum rules.

Moderator: Moderators

Fuel
Posts: 114
Joined: 08 Jul 2006, 13:21

Proper ranks

Post by Fuel »

Hello,

My impression is that admins tend to chrank players based on how well or bad player performed in a match.
That's not always correct.

When chranking someone, I think the following matter alot:

#1. Who are the opponents.
If many of them are actually noobs, it doesn't necessarily mean that if the player won the game or did a lot of damage, he is a very good.
It just happens he played against noobs.
Don't rank someone up, he is not as good as the real pro players.

#2. How lucky was he, how exactly did he succeed in what he did. What were the factors?
Even experienced players play bad sometimes. That still doesn't make the player who defeated them or made damage on them a very good player.
I saw many times chranking happening after a game, just because admins weren't happy how things worked for them.

#3. Really, how good is the player overall, not on one map, not in the team he just played in winning the game. Observe the player along time.
Reality is in BA, DeltaSiegeDry is played A LOT. But that doesn't mean you shouldn't think how good that player is in different situations, different maps. Can he do air well? What about water?
Did you see him in FFA?
If he performed well in few scenarios in few DSD games, that doesn't make him a pro player.
If you want to have ranks only for DSD, make a DSD-only host.
Next time map is changed from DSD, that player can under-perform.
User avatar
Jazcash
Posts: 5309
Joined: 08 Dec 2007, 17:39

Re: Proper ranks

Post by Jazcash »

http://replays.springrts.com/hall_of_fame/BA/

Hold on to your horses for a bit, with any luck, these ranking systems should be coming to SPADS soon.
Fuel
Posts: 114
Joined: 08 Jul 2006, 13:21

Re: Proper ranks

Post by Fuel »

I don't understand how some statistical formulas are going to know how a player can actually play in all servers, different maps
gajop
Moderator
Posts: 3051
Joined: 05 Aug 2009, 20:42

Re: Proper ranks

Post by gajop »

Fuel wrote:I don't understand how some statistical formulas are going to know how a player can actually play in all servers, different maps
then go take a couple of machine learning/statistic classes and you'll find out!
you may need to first finish compulsory education if you still haven't
Fuel
Posts: 114
Joined: 08 Jul 2006, 13:21

Re: Proper ranks

Post by Fuel »

gajop wrote:
Fuel wrote:I don't understand how some statistical formulas are going to know how a player can actually play in all servers, different maps
then go take a couple of machine learning/statistic classes and you'll find out!
you may need to first finish compulsory education if you still haven't
not sure if you are able to understand what i am asking.
also based on how you answered, you might go consult a doctor first too

how is a code going to know when a player is bad and lost because he is having a bad day or because he is actually a noob

but looking forward to see it though.
User avatar
Jazcash
Posts: 5309
Joined: 08 Dec 2007, 17:39

Re: Proper ranks

Post by Jazcash »

Fuel wrote:how is a code going to know when a player is bad and lost because he is having a bad day or because he is actually a noob
Because statistics improve with time. If a player has a month of bad days, then is he having a bad month? Chances are, he's actually just bad.
Fuel
Posts: 114
Joined: 08 Jul 2006, 13:21

Re: Proper ranks

Post by Fuel »

Jazcash wrote:
Fuel wrote:how is a code going to know when a player is bad and lost because he is having a bad day or because he is actually a noob
Because statistics improve with time. If a player has a month of bad days, then is he having a bad month? Chances are, he's actually just bad.
what if he plays rarely? unlucky?

anyway. we'll see how it works out.
in my opinion, such a system takes too much time to tweak and it wont work OK. it's not easy.

i don't understand how it would be able to tell how good a player is in a team game.
I mean I can be ranked 1 and win alot of games, just because i am in the good team.
I make shit units and tech too. slowly.
how the is the code going to know that his kind of player is actually a noob?

would be simpler if admins would just rank people as they should and not based whether they lost or won a particular game. easier.
User avatar
smoth
Posts: 22309
Joined: 13 Jan 2005, 00:46

Re: Proper ranks

Post by smoth »

Because numbers eventually level out..



"but what if they only play ocassionaly"
well they will probably have lower than what they should in ELO. because if they played rarely then they will be rusty as well. Playing regularly means you are current with the game and your skills have not languished.

"but what if they always play with other players who are better"
then those other players are part of thier elo. If they played 1000/1100 with really good allies, then yeah, they are part of that allied groups elo and because of that they are part of that team

"admins should just set elo"
Because the opinion of someone about a player is flawed and prone to bias, admins should not
Fuel
Posts: 114
Joined: 08 Jul 2006, 13:21

Re: Proper ranks

Post by Fuel »

smoth wrote:Because numbers eventually level out..



"but what if they only play ocassionaly"
well they will probably have lower than what they should in ELO. because if they played rarely then they will be rusty as well. Playing regularly means you are current with the game and your skills have not languished.

"but what if they always play with other players who are better"
then those other players are part of thier elo. If they played 1000/1100 with really good allies, then yeah, they are part of that allied groups elo and because of that they are part of that team

"admins should just set elo"
Because the opinion of someone about a player is flawed and prone to bias, admins should not

you took some short phrases out of context for each of the ideas I put
doesn't make any sense

if you really want to respond, please do it for the head lines(in bold) #1, #2, #3
User avatar
smoth
Posts: 22309
Joined: 13 Jan 2005, 00:46

Re: Proper ranks

Post by smoth »

I replied to the ideas in your last post.
Fuel
Posts: 114
Joined: 08 Jul 2006, 13:21

Re: Proper ranks

Post by Fuel »

my point was simple.

for those people which do not play often, the rank doesn't work.
and when playing, it matters alot which team you play.
and there are different other smaller factors.

it's really hard for me to see a system which understands all of this.
but as far as I understood, the system actually starts from some manual data, manual ranks and then adjusts from that.
which basically means the foundation is still some manual input.

might be wrong though.
gajop
Moderator
Posts: 3051
Joined: 05 Aug 2009, 20:42

Re: Proper ranks

Post by gajop »

Statistics/ML programs are based on math, not on a belief system. They represent the best prediction tools currently in existence, and they've been used for a while.
However, they are still being improved, and they are sometimes customized for games (f.e games can have multiple elo leagues for different game modes: 1v1, team games, etc.).
The only thing I'd like to see included, if it's not already, is a rating decay (or some other mechanic to take long absence into account). But that's nothing new in particular (some of the other games I played have it)

This disbelief in programs ability to predict events (match outcomes, music preference, etc.) isn't amusing anymore. It may not be perfect or even as good as an average human expert in all fields, but it's usually economically far superior to anything humans can do for the same amount of money.

And:
Fuel wrote: you might go consult a doctor first too
I talk to them every day! My mentor is also a doctor in a subfield of ML, will see what he thinks. Oh, unless you meant those other doctors of course :)
gajop
Moderator
Posts: 3051
Joined: 05 Aug 2009, 20:42

Re: Proper ranks

Post by gajop »

Fuel wrote: but as far as I understood, the system actually starts from some manual data, manual ranks and then adjusts from that.
which basically means the foundation is still some manual input.
Manual data being a single number which every person starts with...
You may at least read the wikipedia articles before posting?
dansan
Server Owner & Developer
Posts: 1203
Joined: 29 May 2010, 23:40

Re: Proper ranks

Post by dansan »

Fuel wrote:for those people which do not play often, the rank doesn't work.
Ofc a players skill cannot be judged well if he plays seldom - but that doesn't have anything to do with an algorithm/auto/manual ranking - that's just life.
Fuel wrote:and when playing, it matters alot which team you play.
If you have played 20 matches, your rating will not be to far off. The skill assessment algo has a deviation factor that tries to match your performance variation. That's true for all players and in all teams, so it should be leveled to a certain extend.
Fuel wrote:and there are different other smaller factors.
1st: be more specific pls,
2nd: a match balance algo must dumb down reality to some numbers. We are not in search for TheTruth (TM) but are trying to find a way to balance matches as good as possible with as little effort (!) as possible.
Fuel wrote:it's really hard for me to see a system which understands all of this.
Reality-check! "Understanding" is not needed, just good enough results.
Fuel wrote:but as far as I understood, the system actually starts from some manual data, manual ranks and then adjusts from that.
which basically means the foundation is still some manual input.
Correct. The rating numbers are meant as relative comparison between players, and don't mean anything on their own. That means staring numbers are basically arbitrary.

On the replay-site the starting Elo (for 1v1 matches) is 1500 with a k-factor of 30 and TrueSkill (for team, ffa and teamffa matches) starts with a mu of 25 and a sigma of 8,3 (see site code). k-factor and sigma are deviation values - means "certainty" on the assessment (the lower, the more exact).
New players do not start with a low rank (as in /ingame rank), but with a medium rank and "high uncertainty".
User avatar
Silentwings
Posts: 3720
Joined: 25 Oct 2008, 00:23

Re: Proper ranks

Post by Silentwings »

Fuel wrote:I don't understand how some statistical formulas are going to know how a player can actually play in all servers, different maps
I don't understand how hair follicles work but I still have hair ... you might like to have a read of http://research.microsoft.com/pubs/7441 ... 7_0931.pdf or http://research.microsoft.com/pubs/6795 ... 6_0688.pdf.

Different ranks are already used for different types of game, on replay site.
User avatar
MidKnight
Posts: 2652
Joined: 10 Sep 2008, 03:11

Re: Proper ranks

Post by MidKnight »

guys, imagine if we set stocks' values based on how much some "admin" person thought they were worth
User avatar
smoth
Posts: 22309
Joined: 13 Jan 2005, 00:46

Re: Proper ranks

Post by smoth »

MidKnight wrote:guys, imagine if we set stocks' values based on how much some "admin" person thought they were worth
that is kinda how stocks work...
User avatar
Silentwings
Posts: 3720
Joined: 25 Oct 2008, 00:23

Re: Proper ranks

Post by Silentwings »

Yeah, that is pretty much how stocks work - you can only sell for what someone agrees to pay...

Luckily it's a lot easier to guess how good someone is at spring.
User avatar
AF
AI Developer
Posts: 20687
Joined: 14 Sep 2004, 11:32

Re: Proper ranks

Post by AF »

*plays 1000 hours of NOTA* *joins BA game* *high rank, obviously a Pro player*
User avatar
Silentwings
Posts: 3720
Joined: 25 Oct 2008, 00:23

Re: Proper ranks

Post by Silentwings »

That's already taken care of - the replay site gives different ratings for differen mods, and even with each mod different ratings for each kind of game (ffa, team ffa, big team, 1v1, ....)
Post Reply

Return to “General Discussion”