infologs.springrts.com - Page 3

infologs.springrts.com

Discuss development of lobby clients, server, autohosts and auto-download software.

Moderators: Moderators, Lobby Developers

User avatar
code_man
Posts: 260
Joined: 19 Jan 2014, 13:10

Re: infologs.springrts.com

Post by code_man »

Sorry if this is a bit offtopic, but i dont think its worth making a new thread.

My suggestion would be to have a feature when spring crashes having a button that uploads infolog to paste.springfiles.com.
Or some other way too when it doesnt crash, but i dont know how that might be called.
gajop
Moderator
Posts: 3051
Joined: 05 Aug 2009, 20:42

Re: infologs.springrts.com

Post by gajop »

dansan, Is there any way you could allow infolog to be uploaded without a user account? And btw, what was the link to the API? I can't seem to find it in the obvious places (repo or site)
User avatar
ThinkSome
Posts: 387
Joined: 14 Jun 2015, 13:36

Re: infologs.springrts.com

Post by ThinkSome »

infolog-only storage is a bad idea. There are several possible problems that can happen to players:
  • Lobby crash
  • Spring crash
  • Errors in infolog due to game-map-AI-engine incompatiblitity
  • ...
I want to make Springlobby auto-upload all three of the above issues. Info that would be included would be a subset of {Springlobby log, infolog, replay, other information (e.g. ls -Ral of ~/.spring)}. For this I would require a remote service that would accept multiple files per issue report and from where we could then reassign issues to responsible persons: game devs, map devs, AI devs, engine devs and (since this could be used by other lobbies as well) lobby devs.

Since issues can occur before the player is registered on the lobby, the service has to allow anonymous uploads.

Feedback welcome.
gajop
Moderator
Posts: 3051
Joined: 05 Aug 2009, 20:42

Re: infologs.springrts.com

Post by gajop »

Well seeing how no one is using even the simple infolog service, I think we shouldn't be asking some convoluted system...

Personally right now, all I want is a microservice that allows me to uploads (info)logs. Although I'd probably use it to upload the entire lobby + infolog.

PS: spring-replays probably already allows you to upload replays, and that's probably all you need to create a bug report (infolog + replay + user entered text)
User avatar
ThinkSome
Posts: 387
Joined: 14 Jun 2015, 13:36

Re: infologs.springrts.com

Post by ThinkSome »

Perhaps Mantis has a suitable API which could be used for this? I'd rather not upload potentially broken replays to spring-replays and all such submitted files should be initially non-public.
dansan
Server Owner & Developer
Posts: 1203
Joined: 29 May 2010, 23:40

Re: infologs.springrts.com

Post by dansan »

gajop wrote: 27 Aug 2018, 10:19 dansan, Is there any way you could allow infolog to be uploaded without a user account? And btw, what was the link to the API? I can't seem to find it in the obvious places (repo or site)
Hi, there is a json_rpc upload method, but it seems broken. I haven't check it in a long time and it seems the latest Django update didn't go well with the jsonrpc lib...
The method is here: https://github.com/dansan/infolog-uploa ... 341ecb5R40
There is also a json browser that doesnt work: https://replays.springrts.com/infolog/json/browse/ ;)
dansan
Server Owner & Developer
Posts: 1203
Joined: 29 May 2010, 23:40

Re: infologs.springrts.com

Post by dansan »

The infolog page has the logic to allow only game+engine devs and the uploader (gamer) to read the contents and add comments.
It also can do post-upload-processing of the data. Currently only extracts the gameID :)
It connects existing replays and infologs using the gameID.

Having a service to upload stuff to is not really a problem. The question is more one of requirements -> concept.
If you create a concept -> concrete requirement a la "lobby clients can upload files x and y with metadata z" and "after the upload a mantis ticket is opened and an email sent to ...", and "only users with lobby accounts from a certain list can access those uploads" and the protocol should be A over transport B, then such a service can be coded.
gajop
Moderator
Posts: 3051
Joined: 05 Aug 2009, 20:42

Re: infologs.springrts.com

Post by gajop »

dansan wrote: 27 Aug 2018, 23:29 Having a service to upload stuff to is not really a problem. The question is more one of requirements -> concept.
I'd start by just having a public service to upload log files (probably intologs but not necessarily). No login required to read nor write. Maybe you could allow us to specify tags so that later we can build on it (user rights, searches, notifications, parsers, etc), but nothing more than a list of strings would be wanted for that, right now.
dansan
Server Owner & Developer
Posts: 1203
Joined: 29 May 2010, 23:40

Re: infologs.springrts.com

Post by dansan »

gajop wrote: 28 Aug 2018, 08:34 I'd start by just having a public service to upload log files (probably intologs but not necessarily). No login required to read nor write. Maybe you could allow us to specify tags so that later we can build on it (user rights, searches, notifications, parsers, etc), but nothing more than a list of strings would be wanted for that, right now.
That's like... pastbin? :D
I'm always afraid this can be used for warez, but I guess measures can be implemented only when it happens, and not before.

A HTTP-ReST-API with unauthenticated GET (list and get) and POST (create) methods OK?
(Plus non-API admin interface to edit/delete for now?)
gajop
Moderator
Posts: 3051
Joined: 05 Aug 2009, 20:42

Re: infologs.springrts.com

Post by gajop »

dansan wrote: 28 Aug 2018, 11:08 That's like... pastbin? :D
I'm always afraid this can be used for warez, but I guess measures can be implemented only when it happens, and not before.
Yep :) It can be as simple and useful as pastebin. I think having a tag based system would allow us to parse logs in a specific way or alert users as needed, but I don't feel like constraining people so let's go with a fairly unstructured format.
If you feel like it might cause issues with warez we could use authentication keys (given to apps), private view, content checks, etc. Probably just one of those measures would be enough, but no need to design for it now (at least imo)
dansan wrote: 28 Aug 2018, 11:08 A HTTP-ReST-API with unauthenticated GET (list and get) and POST (create) methods OK?
(Plus non-API admin interface to edit/delete for now?)
That's probably fine. I'll be using it through my spring-launcher and it shouldn't have any issues with accessing that (whereas using it from Lua might require a tiny bit of extra work)
dansan
Server Owner & Developer
Posts: 1203
Joined: 29 May 2010, 23:40

Re: infologs.springrts.com

Post by dansan »

gajop wrote: 28 Aug 2018, 11:58
dansan wrote: 28 Aug 2018, 11:08 A HTTP-ReST-API with unauthenticated GET (list and get) and POST (create) methods OK?
(Plus non-API admin interface to edit/delete for now?)
That's probably fine. I'll be using it through my spring-launcher and it shouldn't have any issues with accessing that (whereas using it from Lua might require a tiny bit of extra work)
What protocol and/or transport would be better/easier for Lua?
gajop
Moderator
Posts: 3051
Joined: 05 Aug 2009, 20:42

Re: infologs.springrts.com

Post by gajop »

dansan wrote: 28 Aug 2018, 12:22 What protocol and/or transport would be better/easier for Lua?
JSONRPC (over TCP) is probably easiest - but don't worry about Lua, I expect lobby/launchers are far more likely to post logs than ingame Lua widgets - you can't really report crashes reliably through Lua.
dansan
Server Owner & Developer
Posts: 1203
Joined: 29 May 2010, 23:40

Re: infologs.springrts.com

Post by dansan »

Finished coding it: https://github.com/springfiles/springrts_logs

It has a RESTish HTTP interface with Swagger/OpenAPI schema for client-autogeneration, a JSON-RPC and a XML-RPC interface.
It's not yet running anywhere. I'll talk to abma about where to put it.

Documentation is in the README on Github and online API doc will be available, once it's running.
dansan
Server Owner & Developer
Posts: 1203
Joined: 29 May 2010, 23:40

Re: infologs.springrts.com

Post by dansan »

gajop wrote: 28 Aug 2018, 13:58JSONRPC (over TCP)
I realized to late, that most rpc libs nowadays do everything other HTTP.
I have added support for JSON-RPC over TCP.
All interfaces (REST-API, JSON-RPC over HTTP, XML-RPC over HTTP, JSON-RPC over TCP) are now rate limited.
I have started with an arbitrary number of 200 requests per hour. I suggest we'll adapt as we go :)
I have as TODOs still:
* an RSS-feed for my personal convenience (paranoia monitoring;) and maybe yours
* file type checking (this is for text files only, right?)
* upload volume limiting (<5MB per upload?, <1 GB/day?)
But IMHO those are nice-to-have and at this point we can start the service (as soon as abma and I have set it up).
gajop
Moderator
Posts: 3051
Joined: 05 Aug 2009, 20:42

Re: infologs.springrts.com

Post by gajop »

dansan wrote: 08 Sep 2018, 20:07 I realized to late, that most rpc libs nowadays do everything other HTTP.
I have added support for JSON-RPC over TCP.
All interfaces (REST-API, JSON-RPC over HTTP, XML-RPC over HTTP, JSON-RPC over TCP) are now rate limited.
That's a lot of options :) I will probably use the REST-API with my launcher - it's built in electron so that's easy. But nice to see all the options. Ideally I'd stick to just REST-API, but I'd first need to get some decent Lua socket support there.
dansan wrote: 08 Sep 2018, 20:07 I have started with an arbitrary number of 200 requests per hour. I suggest we'll adapt as we go :)
I have as TODOs still:
* an RSS-feed for my personal convenience (paranoia monitoring;) and maybe yours
* file type checking (this is for text files only, right?)
* upload volume limiting (<5MB per upload?, <1 GB/day?)
But IMHO those are nice-to-have and at this point we can start the service (as soon as abma and I have set it up).
This is just text indeed (mostly logs).
5MB upload sounds good. There are cases where logs can exceed that (e.g. when they're spammy), but then they should be truncated at the client side before uploading.
dansan
Server Owner & Developer
Posts: 1203
Joined: 29 May 2010, 23:40

Re: infologs.springrts.com

Post by dansan »

It's up! :)
Getting it running in production took almost longer that coding it ^.^
Documentation: https://github.com/springfiles/springrts_logs
gajop
Moderator
Posts: 3051
Joined: 05 Aug 2009, 20:42

Re: infologs.springrts.com

Post by gajop »

That's awesome, thanks. I'll be sure to use it with my launcher and I encourage other lobby devs to do so too.
dansan
Server Owner & Developer
Posts: 1203
Joined: 29 May 2010, 23:40

Re: infologs.springrts.com

Post by dansan »

Sorry - service was down after the server migration, but is now reachable again.
gajop
Moderator
Posts: 3051
Joined: 05 Aug 2009, 20:42

Re: infologs.springrts.com

Post by gajop »

dansan wrote: 27 Oct 2018, 22:24 It's up! :)
Getting it running in production took almost longer that coding it ^.^
I'd like to report that I've started using it with SpringBoard through spring-launcher, and it's working great, thanks!
That said, would it be possible to provide some kind of website interface for displaying logs? Ideally the logs themselves would be displayed like on pastebin, ideally with some colors based on line type (e.g. warning, error, chat, etc.)
dansan
Server Owner & Developer
Posts: 1203
Joined: 29 May 2010, 23:40

Re: infologs.springrts.com

Post by dansan »

gajop wrote: 15 Dec 2018, 18:09 I'd like to report that I've started using it with SpringBoard through spring-launcher, and it's working great, thanks!
That said, would it be possible to provide some kind of website interface for displaying logs? Ideally the logs themselves would be displayed like on pastebin, ideally with some colors based on line type (e.g. warning, error, chat, etc.)
Glad to hear that.
I created a custom HTML renderer for SpringBoard log entries: http://logs.springrts.com/logfiles/43/

It will be triggered, if the Accept header contains HTML and not JSON and if the list of tags contains SpringBoard.

JSON is still the default, if Content-type and Accept are not set to HTML.
To force JSON (although not necessary), transmit Content-type: application/json and Accept: application/json or append .json to the item number in the URL: http://logs.springrts.com/logfiles/43.json

To force the API doc view, append .api to the URL: http://logs.springrts.com/logfiles/43.api

I'm not big with CSS and stuff. Please feel free to play to PR something better :)
Post Reply

Return to “Lobby Clients & Server”