statistics service

statistics service

For the discussion of infrastructure improvements and changes.

Moderator: Moderators

dansan
Server Owner & Developer
Posts: 1203
Joined: 29 May 2010, 23:40

statistics service

Post by dansan »

I could create a service that has a HTTP-API that allows to
  • add platform data (CPU, GPU etc), associated with a unique identifier
  • query that data for statistical purposes
The unique identifier would be the springrts accountid. In the service' database it would be stored as a salted hash. Knowing the salt it would still be very fast to brute force it, as the source data pool (accountids) is so small. So this is only a security measure against a programmer mistake, to not publish the IDs unhashed by mistake.

Access could be public or restricted to engine and game devs.

I would create a Python client that consumes a springrts demofile and sends the data to the statistics service. That client could be used by both the replay site and the zero-k server.
gajop
Moderator
Posts: 3051
Joined: 05 Aug 2009, 20:42

Re: statistics service

Post by gajop »

This would be useful to have.
Access can be restricted, but a display like this should be public: http://store.steampowered.com/hwsurvey/
I would create a Python client that consumes a springrts demofile and sends the data to the statistics service. That client could be used by both the replay site and the zero-k server.
I'd prefer an API rather than having to upload demofiles. I understand some people may prefer demofiles though.
dansan
Server Owner & Developer
Posts: 1203
Joined: 29 May 2010, 23:40

Re: statistics service

Post by dansan »

gajop wrote:This would be useful to have.
Access can be restricted, but a display like this should be public: http://store.steampowered.com/hwsurvey/
Sure. But someone else will have to make the gfx, not my thing. I can make tables though :)
gajop wrote:
I would create a Python client that consumes a springrts demofile and sends the data to the statistics service. That client could be used by both the replay site and the zero-k server.
I'd prefer an API rather than having to upload demofiles. I understand some people may prefer demofiles though.
Misunderstanding: there will be an API. And I'll write client code for the replay side. To get ZK into the boat I wanted to offer a script that will allow them to just call "send_stats /../my_replay.sdfz" whenever springie dumps a new replay. Ofc ZK-infra can just do that themselfs, as there won't be much to this API :) Parsing a demofile is not so much fun though.
gajop
Moderator
Posts: 3051
Joined: 05 Aug 2009, 20:42

Re: statistics service

Post by gajop »

API works fine for ZK, as they already do this, through the combination of Chobby, the wrapper program and a 3rd party site for logging user analytics.
As I said, there's the Platform table in Lua now so there's no need to parse infologs.

The demo file is useful for the current messy Spring situation where everyone is using their own lobby, many of which may not be able to extract user statistics (so just sending the replay file after playing MP games is fine, if the user gets that far).
dansan
Server Owner & Developer
Posts: 1203
Joined: 29 May 2010, 23:40

Re: statistics service

Post by dansan »

Ah great.
What data except for that in the table you mentioned in viewtopic.php?t=36288&p=582657#p582657 should be logged?
gajop
Moderator
Posts: 3051
Joined: 05 Aug 2009, 20:42

Re: statistics service

Post by gajop »

Uh, that might be hard to predict. Platform didn't even exist in 103, and I'm sure over time we will have more/different things as well.

I would allow accepting arbitrary key/value tables, but only use a clearly defined subset for parsing.

Additionally I would also log the engine version and lobby.

There's also an Engine table which contains more info:
$ table.echo(Engine)
[f=0000491] return {
buildFlags = "",
version = "103.0.1-1057-g3a29822 develop",
versionFull = "103.0.1-1057-g3a29822 develop",
versionPatchSet = "0",
wordSize = 64,
}
gajop
Moderator
Posts: 3051
Joined: 05 Aug 2009, 20:42

Re: statistics service

Post by gajop »

Just to be clear, you want to create a service that will log user machine stats? You don't want to create something that will monitor what users do?

I forgot, but there's also https://github.com/gajop/analytics with live site if anyone cares for an open source solution. I haven't been working on it though.
dansan
Server Owner & Developer
Posts: 1203
Joined: 29 May 2010, 23:40

Re: statistics service

Post by dansan »

gajop wrote:Just to be clear, you want to create a service that will log user machine stats?
Yes.

I don't have a personal interest in in that data. I understand the engine/game devs have a need there. If I misunderstood, I won't do this. Please clarify.
gajop wrote:You don't want to create something that will monitor what users do?
Yes. (I don't want to monitor what users do.)

I guess it could be used for that too. Just collect some data ingame (how many clicks does a user do? what shortcuts are used, which not? what's the most common zoom-level? how often is the LOS-map shown?) and upload that. But nobody has asked for this, yet.
gajop wrote:I forgot, but there's also https://github.com/gajop/analytics with live site if anyone cares for an open source solution. I haven't been working on it though.
"Home" is empty for me. I think jquery.min.js has to be loaded before bootstrap.min.js.

The widget uses JsonRPC to upload the stats? That's nice! Much better that parsing replays. Prone to abuse though... How can the server know that it's not a script uploading the data?
dansan
Server Owner & Developer
Posts: 1203
Joined: 29 May 2010, 23:40

Re: statistics service

Post by dansan »

The stats collection API went online. Currently it is "feed the beast" only, as there has yet to surface a query. So only POST, no GET.

Source code (AGPL): https://github.com/dansan/springrts-platform-stats
API is online at: http://stats.replays.springrts.com/

I used the domain, because it was easiest and quickest. It is not meant to last like that.

Te service is currently being fed about a year of BA and Evo replays, as those contain platform data. When enough is accumulated, we can begin playing with the results.

I have created dedicated tables for data that seemed usable to cluster and/or I was told about: BA/Evo gadget, Lua platform table, Lua engine table). Everything else goes into a simple key-value table. The database design can be changed. With a complete lack of requirements this is obviously experimental.

To post to the HTTP API, a user account is needed. If you wish to experiment with it, please PM me, and I'll send you a username/password pair. Examples for the command line (curl), Python and JavaScript are on the github page.
dansan
Server Owner & Developer
Posts: 1203
Joined: 29 May 2010, 23:40

Re: statistics service

Post by dansan »

Some stats for current data:
CPU name
  • 1695x Intel ...
  • 412x AMD ...
  • 2x VBS Intel Wide Vigina :shock:
glRenderer
  • 1186x GeForce|Quadro|NVS|GRID
  • 479x AMD|ATI|Radeon|ASUS
  • 436x Intel
  • 5x SVGA3D (LLVM)
RAM
Image

resolution X
Image

resolution Y
Image
dansan
Server Owner & Developer
Posts: 1203
Joined: 29 May 2010, 23:40

Re: statistics service

Post by dansan »

I was thinking if there should be a field (list) for game titles. So if for example the Evo or ZeroK maintainers were interested in stats about only their users, they could filter for that. Stats could still be made over the whole SpringRTS populace, by not filtering.

Would that make sense?
raaar
Metal Factions Developer
Posts: 1094
Joined: 20 Feb 2010, 12:17

Re: statistics service

Post by raaar »

Thanks for the information!

So that means about 25% of the players have AMD gpu and 25% have onboard intel gpu?

The number of entries represent distinct players or player-games?

What does the RAM chart mean?
dansan
Server Owner & Developer
Posts: 1203
Joined: 29 May 2010, 23:40

Re: statistics service

Post by dansan »

raaar wrote:So that means about 25% of the players have AMD gpu and 25% have onboard intel gpu?
Yes.
Although I don't know if the Evo/BA gadget detects GPUs correctly, when the rendering is done by NV/AMD, but the gfx output is channeled through an onboard intel gfx (common in notebooks).
raaar wrote:The number of entries represent distinct players or player-games?
Distinct players - (really accountID+CPU+OS combinations, but usually that's just the same). Which game was played doesn't matter.
The game of the replay that the data was extracted from is currently not saved (see my previous post), so I cannot say if BA or Evo players have the better hardware :)
raaar wrote:What does the RAM chart mean?
X-axis is MB RAM and Y-axis is the number of players with that RAM.
It would probably be more helpful to categorize the results into: <4GB, ~4GB, ~8GB and > 8GB.
... OK, I did:
  • 119: <3400 (<4GB)
  • 469: 3800 - 5000 (~4GB)
  • 1184: 8000 - 8200 (~8GB)
  • 739: 16000 - 16400 (~16GB)
  • 87: > 32000 (~32GB)
Surprising how many people still get cheated into buying <8GB machines...
dansan
Server Owner & Developer
Posts: 1203
Joined: 29 May 2010, 23:40

Re: statistics service

Post by dansan »

dansan wrote:
raaar wrote:So that means about 25% of the players have AMD gpu and 25% have onboard intel gpu?
Yes.
Although I don't know if the Evo/BA gadget detects GPUs correctly, when the rendering is done by NV/AMD, but the gfx output is channeled through an onboard intel gfx (common in notebooks).
We really want the Lua platform table for this.
dansan
Server Owner & Developer
Posts: 1203
Joined: 29 May 2010, 23:40

Re: statistics service

Post by dansan »

I have added a list of all keys that are currently detected when POSTing to the API to https://github.com/dansan/springrts-pla ... to-the-api (All other keys are stored in a generic key/value pair way, but the following ones have their separate database tables):

Get this from OS/spring engine somehow? (See BA/Evo gadget):
cpuName: str
cpuCores: str

resolution_x: int
resolution_y: int
color_depth: int
refresh_rate: int
windowed: bool

Lua Engine table
buildFlags: str
version: str
versionFull: str
versionPatchSet: str
wordSize: str

Lua Platform table
glRenderer: str
glVendor: str
glVersion: str
glVersionShort: str
glewVersion: str
glslVersion: str
glslVersionShort: str
glSupportNonPowerOfTwoTex: bool
glSupportTextureQueryLOD: bool
glSupport24bitDepthBuffer: bool
glSupportRestartPrimitive: bool
glSupportClipSpaceControl: bool
glSupportFragDepthLayout: bool

gpu: str
gpuMemorySize: int
gpuVendor: str

osFamily: str
osName: str

sdlVersionCompiledMajor: int
sdlVersionCompiledMinor: int
sdlVersionCompiledPatch: int
sdlVersionLinkedMajor: int
sdlVersionLinkedMinor: int
sdlVersionLinkedPatch: int
User avatar
Jools
XTA Developer
Posts: 2816
Joined: 23 Feb 2009, 16:29

Re: statistics service

Post by Jools »

dansan wrote:
  • 119: <3400 (<4GB)
  • 469: 3800 - 5000 (~4GB)
  • 1184: 8000 - 8200 (~8GB)
  • 739: 16000 - 16400 (~16GB)
  • 87: > 32000 (~32GB)
Surprising how many people still get cheated into buying <8GB machines...
You should have a < 2 GB category too. A lot of people will disagree, but they are wrong :lol:

This is almost the most important category to monitor.
gajop
Moderator
Posts: 3051
Joined: 05 Aug 2009, 20:42

Re: statistics service

Post by gajop »

Te service is currently being fed about a year of BA and Evo replays, as those contain platform data. When enough is accumulated, we can begin playing with the results.
How did you get the current data? Are those just infologs? In that case, a lot of the information from Lua platform data is contained there.
raaar
Metal Factions Developer
Posts: 1094
Joined: 20 Feb 2010, 12:17

Re: statistics service

Post by raaar »

So players' available RAM is clustered around 4GB, 8GB and 16GB, with spikes on 6GB, 10GB and 16GB.

I never expected so many players with 16GB ram!

The most common resolution also seems quite high : 1920x1096 ?

Now i'm wondering about what kind of first world problems they complain about...
dansan
Server Owner & Developer
Posts: 1203
Joined: 29 May 2010, 23:40

Re: statistics service

Post by dansan »

raaar wrote:The most common resolution also seems quite high : 1920x1096 ?
It's 1920x1080 - the scale is a little off in the graph.
dansan
Server Owner & Developer
Posts: 1203
Joined: 29 May 2010, 23:40

Re: statistics service

Post by dansan »

gajop wrote:
Te service is currently being fed about a year of BA and Evo replays, as those contain platform data. When enough is accumulated, we can begin playing with the results.
How did you get the current data? Are those just infologs? In that case, a lot of the information from Lua platform data is contained there.
The data is extracted from replays which captured what the BA/Evo system_info.lua gadet wrote with a LuaMsg (see viewtopic.php?f=44&t=36301#p582760). That is only a small part of the infolog.
Post Reply

Return to “Infrastructure Development”