chrank and the constant failure of balance

[Johannes]

Now if you always make pure eco for 25 mins at start, you get assigned with people who tend to deal a lot of damage early - I don't think the techer will mind, if anything it's encouragement to deal even less damage to get more aggressive teammates to protect you.

It does not matter if he minds or not. But if he is not able to convert his eco into actual damage then he is a bad player and deserves damage-dealing teammates.

But if you make a decisive nuke or bombing run, for example, your damage dealt will still be very low compared to someone who's been trading stumpies all game long. Dealing 4000 damage to a fusion is more valuable than dealing 4000 damage to 3 stumpies, and the system cannot distinguish between the two. Or if you shared thousands of metal to your allies who then dealt the damage...

There's just too many pitfalls how looking at dmg dealt graph does not give accurate results of how good you actually are. If you've got huge damage dealt, ok probably you're ok, but with low dmg players it' hard to say.

[Silentwings]

Code: Select all

One of the most important parts of a new ranking system is to be able to detect smurfs in a reasonable amount of time. I dont think 100 games is viable in that matter.

Yes, 100 games is far too many to use as smurf detection. I don't think win/loss can be used for that.

I don't konw much about lobby protocol but my feeling is that smurf detection is best done using IP adress, etc.

But I agree that it might be sensible to use say, some statistics on damage dealt, res produced, etc during the first 5 games (but not more imo) to dynamically calculate a players 'initial' ranking. But after those first 5 game are played the rank value should only be changed according to win/loss.

We can never overcome the problem 100% since its possible to evade any lobby based detection mechanism with enough effort and we can't give a new account a high rank for their very first game.

[very_bad_soldier]

@Johannes:
Yeah you are right, I am aware of that.
But: Someone who teched and executed the decisive T2 bombing run is not necessarily a better player than someone who fought the T1 battles the whole game and held the front. Yes, the bombing player actually decided and won that game but it does not mean that he is a very skillfull player. So I think damage dealt is still a viable metric imo, even in that case.

At least I would prefer a hard fighting front guy in my team instead of an eco whoring nuke rusher even if he might decide the game alone sometimes.

[Johannes]

Sure, anyone can rush nukes or bombers, it's no measure of skill.

But now compare to someone who did even less than that, mostly idling and building adv solars. In the balance algorithms eyes these guys are all but equal.

[very_bad_soldier]

Ok that is true, but that is even more true for the win/loss-approach. Basically everyone that happens to be in the winning team is equal in algorithm's eyes.

Another thing that is imo a big problem when using the win/loss-metric:
Lets assume it works well and the win/loss-algorithm calculated proper ranks for each player and is therefore able to balance games really well. (So really good players will play together with really bad players and so on..)
By definition we will end up with games where every player will win about 50% of the games cause it is so well balanced. But this fact will render the balance system itself useless since it relies solely on win/loss-ratio.
So currently I think a system that tries to balance out the win/loss chances for each team (by manipulating it) cant rely on win/loss-statistics. Its a conflict in itself, no?

[BaNa]

Its called a feedback loop and it works pretty well in a lot of cases.

[dansan]

In this discussion I have the feeling, that I read different opinions because of different "goals". The 1v1-players wish to have accurate skill assessment, the team-game players wish to have balance. Those two are as different as the game-types are.

[Johannes]

In this discussion I have the feeling, that I read different opinions because of different "goals". The 1v1-players wish to have accurate skill assessment, the team-game players wish to have balance. Those two are as different as the game-types are.

I don't really see this. Accurate skill assessment -> ability to balance games. The debated thing is if skill is better assessed by something other than results.

[Silentwings]

By definition we will end up with games where every player will win about 50% of the games cause it is so well balanced. But this fact will render the balance system itself useless since it relies solely on win/loss-ratio.

This dosn't quite happen because there is random variation coming in all the time - some people play better one day than the next, new players start, etc.

In practice what happens is that the ranking method would be constantly trying to move towards a situation where (in the long run, at least) everyone wins 50% of the time but the random variations mean it really just oscillates around the 'equilibrium' without getting stuck in it.

I don't see why it would be a problem if what happened was that eventually everyone won very close to 50% of their games - that sounds to me like a successful rank & balance system!

If it really was undesirable then when you autobalance a game you could artifically increase the variation - allow some random influence, pick at random a combination where the chances of winning are somewhere between 60-40 and 40-60 rather than choosing almost exactly 50-50 all the time.

[Rumpelstiltskin]

winning 50% of the time in team games is perfect.
In 1v1s', Where there is no need to balance the 2 opposing sides, all you need is an accurate assessment of skill to assign the proper points to the winner and detract the proper amount of points from the loser.

These two cases, 1v1s and team games, are fundamentally different and thus require a different approach.

[very_bad_soldier]

I don't see why it would be a problem if what happened was that eventually everyone won very close to 50% of their games - that sounds to me like a successful rank & balance system!

Yeah of course it is. I did not mean to say that it is bad. But if the balance system works well and comes near to that 50%-state then it will be hard to use the win/loss-data as the metric cause everyone has the same win/loss-ratio.

[Silentwings]

Dansan, do TERA servers upload their games to your site? Would be a shame not to have stats including those too.

[tzaeru]

One of the most important parts of a new ranking system is to be able to detect smurfs in a reasonable amount of time. I dont think 100 games is viable in that matter.

Even if the system detected a smurf in scope of five games, the smurfer could just roll a new account after those games anyway.

If you can't rely on IP detection and admins to counter smurfs, there aren't so many other methods to rely on either.

I would vote for damage dealt as the used metric. While I agree it would have same flaws it would IMO still be the best metric we can get with a minimal amount of work or overcomplex algorithms.

Elo/Glicko/whatever ranking system based on win/loss stats is even less work.

I find the damage dealt just very problematic, especially when we're talking of small games where most players are quite skilled. Often there might be dedicated air guy to give off transes, and he's prolly doing less damage in the end than others. There might be someone whose sole purpose is to survive for 10 minutes while his ally in the back rolls out amphis, etc.

Of course, all these things can be considered and even tested with feeding replays to the algorithm.

I don't see why it would be a problem if what happened was that eventually everyone won very close to 50% of their games - that sounds to me like a successful rank & balance system!

Yeah of course it is. I did not mean to say that it is bad. But if the balance system works well and comes near to that 50%-state then it will be hard to use the win/loss-data as the metric cause everyone has the same win/loss-ratio.

This was one of the fundamental flaws that I imagined would be encountered. But I think it's actually false to say that such a perfect balance where everyone won 50% of games was ever achieved. In small to medium games, I'd guess that very good players will end up with higher than 50% chance to win, which gives some variety in the win-loss ratios of players.

[Johannes]

Ok that is true, but that is even more true for the win/loss-approach. Basically everyone that happens to be in the winning team is equal in algorithm's eyes.

Another thing that is imo a big problem when using the win/loss-metric:
Lets assume it works well and the win/loss-algorithm calculated proper ranks for each player and is therefore able to balance games really well. (So really good players will play together with really bad players and so on..)
By definition we will end up with games where every player will win about 50% of the games cause it is so well balanced. But this fact will render the balance system itself useless since it relies solely on win/loss-ratio.
So currently I think a system that tries to balance out the win/loss chances for each team (by manipulating it) cant rely on win/loss-statistics. Its a conflict in itself, no?

There's no logical flaw in iterating ranking from win/loss stats. If there was a problem you'd never get to those ~50/50 numbers anyway.


And the big deal about using a metric other than win/loss is not just inaccyracy, but how they can be gamed. If damage dealt is the used metric, one can try to use strategies that minimize damage dealt while still being as useful as possible, and end up getting better teammates and therefore better winrate than you would normally.

winning 50% of the time in team games is perfect.
In 1v1s', Where there is no need to balance the 2 opposing sides, all you need is an accurate assessment of skill to assign the proper points to the winner and detract the proper amount of points from the loser.

These two cases, 1v1s and team games, are fundamentally different and thus require a different approach.

It's not really the difference between team games and 1v1s, but between randomly assigned teams and anything else - not all team games are necessarily pub matches.

[dansan]

Dansan, do TERA servers upload their games to your site? Would be a shame not to have stats including those too.

Unfortunately not. I tried to contact them various times unsucesfully :(
If any of you meet a TERA admin, please be so nice to ask him again, maybe you have more luck. Installation is easy, and I give technical support.

[Silentwings]

I'll see what I can do - it might just be a timezone issue, TERA admins are on pretty regularly.

[Beherith]

God fucking dammit dansan, let me have your babies! Seriously amazing shit!

Your glicko might be off, a guy with 1 loss and 0 wins 1v1 has almost 2k
http://replays.springrts.com/player/231 ... ort=glicko

[muckl]

i added the functionality for replay upload on both TERA hosts

thanks for the information (dansan,bluestone)

http://replays.springrts.com/user/%5BLOeT%5Dupload/

[dansan]

Cool muckl!
Here's your RSS btw: http://replays.springrts.com/feeds/uplo ... %5Dupload/

In case you wonder why some replays show up as "Replay is incomplete, winner and match length are unknown. Please leave a comment." - it's a bug on my side: At some point I forgot that matches can be draws, and considered "no winner" a sign of missing data :D
Will be fixed.

[NeonStorm]

Some players can't manage their metal bar - that is really true!

A team with some players which are skilled in it and some which aren't performs better - that is true independently of the other stats ...

You even have to consider the management from e excess and m excess differently...
Even the ZK's ELO fails to this (although it is easier without makers).

Another thing is, that failing players team disadvantage increase exponentially, because the enemies get reclaim.
Holding an allies lost position's wrecks is more important (especially BA,TA) than pushing over built-up defense lines.

A Team is a Team, NOT a bunch of multiply-able or accumulating ELO values.