| View Issue Details [ Jump to Notes ] | [ Issue History ] [ Print ] | ||||||||
| ID | Project | Category | View Status | Date Submitted | Last Update | ||||
|---|---|---|---|---|---|---|---|---|---|
| 0005361 | Buildbot | Buildbot | public | 2016-10-02 02:38 | 2016-10-05 19:40 | ||||
| Reporter | AF | ||||||||
| Assigned To | abma | ||||||||
| Priority | high | Severity | major | Reproducibility | N/A | ||||
| Status | resolved | Resolution | no change required | ||||||
| Summary | 0005361: Notification on Failure | ||||||||
| Description | When buildbot fails there is no notification system. I propose that when a failure occurs, it be emailed to a mailing list. From there individual users can subscribe and filter accordingly | ||||||||
| Steps To Reproduce | N/A | ||||||||
| Additional Information | What's the point of build tests if nobodies told when they fail? Major CI software vendors do this, GH highlights failed builds, Travis sends you notices of failed builds and tests, and tells you when they work again. I myself didn't know Spring builds were being broken, as I don't watch buildbots process, not until I was made aware long after the fact and it had been assumed I was watching when I wasn't. It came as a total surprise for me. As somebody involved in the toolchain I should have been made aware, and as an employee working full time I don't have the luxury of being able to check daily. I have my own projects and clients with their own CI, tickets and build processes to monitor | ||||||||
| Tags | No tags attached. | ||||||||
| Attached Files |
| ||||||||
 Relationships |
|
 Relationships |
| Notes |
|
|
abma (administrator) 2016-10-03 03:13 |
current build system pm's to #sy. #sy keeps history. also http://buildbot.springrts.com/waterfall exists. http://buildbot.springrts.com/rss you wouldn't know with a mailing list as well, i'm pretty sure! there are more than enough existing working solutions, you just need to use them. |
|
AF (reporter) 2016-10-03 03:32 |
Those systems are insufficient. IRC: I can't be expected to watch an active community channel and manually filter out dev discussions so that I can pick up failed build messages and check each one by hand for their cause. If not because it's a large time investment, but because I have a job to do, I'm not always near a computer, and I have work obligations. I also need to sleep. Recently I spent a week in Canada, if Shard had broken the Spring build while I was on a plane how should I know? Waterfall Page: Great once you know the build failed, but do you seriously check this on an hourly or daily basis? If so, that's hideously inefficient. But if it wasn't, then I'm sorry but i have status pages to monitor from work. Fortunately they have modern monitoring systems that actively ping you when things break, even Shard has this now via Travis. RSS: These items lack the information necessary to determine what happened. Given that the validation test that failed due to Shard is still on that feed, this could act as an imprecise mechanism, but with limited information, and delays induced via feed caches A mailing list would allow the full information to be included. It can be filtered and archived via numerous pieces of software. You may not be able to do it but I can. This ticket is about improving build failure notifications. The current setup is insufficient and caters to those who are able to be active in IRC and in specific channels. You may think they are sufficient, I'm telling you, they aren't. |
|
abma (administrator) 2016-10-03 03:35 |
pull requests welcome, no need to keep a ticket. |
|
abma (administrator) 2016-10-03 03:38 |
> The current setup is insufficient and caters to those who are able to be active in IRC and in specific channels. wrong, springs lobby server stores history, no need to be permanently online to read all build messages (which go to #sy). |
|
abma (administrator) 2016-10-03 03:41 |
also: https://github.com/tomjn/Shard/issues/102#issuecomment-250937934 "I personally receive thousands of GitHub emails each day of which my paid work takes precedence, I don't always see Shard related issues immediately. A DM via Twitter/Facebook/Slack/Skype is the best way to get my attention ( in that order )" this won't improve when you get email notification about build/runtime failures of spring, too. stop trolling please. |
|
AF (reporter) 2016-10-03 03:47 |
I'm not reading through the history of #sy everyday to see what happened, not unless I can say "ah sorry I've not gotten to those yet" every time I get complaints I've broken the build I simply don't have the time for a manual fetch style process, it needs to be a push based system, an incoming feed not a pulled down digest. I can work with incoming data but I only have so much time for manual checks and email already takes up a lot of that time Wouldn't it be nice if you didn't have to pester people for breaking the build? If they had already been told by the system itself? Build X broken after a commit by Y, or in test who Y is responsible for Instead of shutting it down, acknowledge it's a problem but that you don't have a solution for it and let others work through it. Right now I know that I could allocate a day of time with full certainty that you'll close the pull request out of spite rather than have a reasonable discussion about it, so why would I waste my time. |
|
abma (administrator) 2016-10-03 03:49 |
> Wouldn't it be nice if you didn't have to pester people for breaking the build? If they had already been told by the system itself? Build X broken after a commit by Y, or in test who Y is responsible for thats how it currently works. but this doesn't and can't cover removal of git repositories. |
|
AF (reporter) 2016-10-03 03:58 |
This is about build failure notifications, which impacts more than Shard. Shard and submodules is a separate issue for which there are unaddressed questions from yourself. Refer to GitHub. There is no fork of Buildbot at https://github.com/spring or build infrastructure to make a pull request against This is a serious ticket, and I don't appreciate the aggressive tone, I don't appreciate that one of my projects has a major hole in alerting and reporting for test failures, and it's a hole I'd like to plug, if not so that people such as yourself don't barge into GitHub tickets mouthing off, but so that I can fix issues as soon as they arise, rather than being unawares |
|
AF (reporter) 2016-10-03 04:00 |
What I'd like to see in this ticket is direction as to where the work needs doing, and that it won't be outright rejected once done |
|
AF (reporter) 2016-10-03 04:05 |
I was pointed to https://github.com/spring/spring/tree/develop/buildbot in #sy Buildbot documentation suggests this has already been implemented and can be configured via https://docs.buildbot.net/current/manual/cfg-statustargets.html#mailnotifier it also suggests that it can email just the developers who made changes in the commits being built |
|
abma (administrator) 2016-10-03 13:39 |
some devs don't use valid mail addresses this is why email notification mostly isn't/wasn't used. i still get mails when travis fails. also travis mails about broken builds but this seems to not cover submodules (which makes sense as they are made by external developers which usually are not to blame when it breaks). i guess to get useful data a build must be made which compiles just shard and runs a test game. else you'll have a lot of false positives which makes a notification about shard related errors useless as its very difficult to find out the cause / the responsible person who broke it. |
|
AF (reporter) 2016-10-03 18:57 |
I'd be happy if I could just filter on the validation test, I don't mind a few false positives here and there, and I can match on things I know come from Shard to reduce that, e.g. the opening mentions of a Shard lua stack trace, or the lack of a "Shard finished init" message By submodules, is that just the submodule that pulls in Shard/KAI/etc/etc or do you mean the submodules for individual overrides currently in Shard? Note that I'm not opposed to their replacement, submodules solve a problem and they're the only solution available at the moment that anybody in the Shard project has the knowledge to implement, but that still leaves notifications if a typo is missed in a pull request |
|
abma (administrator) 2016-10-03 19:59 |
then use sth. like http://buildbot.springrts.com/rss/?builder=validationtests&failures_only=true |
|
AF (reporter) 2016-10-03 20:43 |
The content from each item in that RSS doesn't contain infolog information, can it be made to include that? For example http://buildbot.springrts.com/builders/validationtests/builds/5454 mentions several validation failures including Shard with BA, but that isn't mentioned in the RSS feed content |
|
abma (administrator) 2016-10-04 01:25 |
"test_5" is the Shard. very likely this can be changed by some buildbot config magic, idk how. |
| Notes |
| Issue History |
|||
| Date Modified | Username | Field | Change |
|---|---|---|---|
| 2016-10-02 02:38 | AF | New Issue | |
| 2016-10-03 03:13 | abma | Note Added: 0016723 | |
| 2016-10-03 03:13 | abma | Status | new => closed |
| 2016-10-03 03:13 | abma | Assigned To | => abma |
| 2016-10-03 03:13 | abma | Resolution | open => not fixable |
| 2016-10-03 03:32 | AF | Note Added: 0016724 | |
| 2016-10-03 03:32 | AF | Status | closed => feedback |
| 2016-10-03 03:32 | AF | Resolution | not fixable => reopened |
| 2016-10-03 03:35 | abma | Note Added: 0016725 | |
| 2016-10-03 03:35 | abma | Status | feedback => closed |
| 2016-10-03 03:35 | abma | Resolution | reopened => fixed |
| 2016-10-03 03:38 | abma | Note Added: 0016726 | |
| 2016-10-03 03:41 | abma | Note Added: 0016727 | |
| 2016-10-03 03:47 | AF | Note Added: 0016728 | |
| 2016-10-03 03:47 | AF | Status | closed => feedback |
| 2016-10-03 03:47 | AF | Resolution | fixed => reopened |
| 2016-10-03 03:49 | abma | Note Added: 0016729 | |
| 2016-10-03 03:49 | abma | Status | feedback => closed |
| 2016-10-03 03:49 | abma | Resolution | reopened => no change required |
| 2016-10-03 03:58 | AF | Note Added: 0016730 | |
| 2016-10-03 03:58 | AF | Status | closed => feedback |
| 2016-10-03 03:58 | AF | Resolution | no change required => reopened |
| 2016-10-03 04:00 | AF | Note Added: 0016731 | |
| 2016-10-03 04:00 | AF | Status | feedback => assigned |
| 2016-10-03 04:05 | AF | Note Added: 0016732 | |
| 2016-10-03 13:39 | abma | Note Added: 0016734 | |
| 2016-10-03 18:57 | AF | Note Added: 0016736 | |
| 2016-10-03 19:59 | abma | Note Added: 0016737 | |
| 2016-10-03 20:43 | AF | Note Added: 0016738 | |
| 2016-10-04 01:25 | abma | Note Added: 0016739 | |
| 2016-10-05 19:40 | abma | Status | assigned => resolved |
| 2016-10-05 19:40 | abma | Resolution | reopened => no change required |
| Issue History |