[ARROW-8043] [Developer] Provide better visibility for failed nightly builds

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.16.0
• Component/s: Continuous Integration, Developer Tools
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/24257
• Assignee: Sam Albers
• Reporter: Ben Kietzman
• Created: 09/Mar/20 20:10
• Updated: 11/Jan/23 07:57
• Resolved: 13/Jul/22 15:57

Description

Emails reporting nightly failures are unsatisfactory in two ways: there is a large click/scroll distance between the links presented in that email and the actual error message. Worse, once one is there it's not clear what JIRAs have been made or which of them are in progress.

One solution would be to replace or augment the [NIGHTLY] email with a page (https://ursa-labs.github.org/crossbow would be my favorite) which shows how many nights it has failed, a shortcut to the actual error line in CI's logs, and useful views of JIRA. We could accomplish this with:

• dedicated JIRA tags; one for each nightly job so a JIRA can be easily associated with specific jobs
• A static HTML dashboard with client side JavaScript to
  • scrape JIRA and update the page dynamically as soon as JIRAs are opened
  • show any relationships between failing jobs
  • highlight jobs that have not been addressed, along with a counter of how many nights it has gone unaddressed
• provide automatic and expedited creation of correctly labelled JIRAs, so that viewers can quickly organize/take ownership of a failed nightly job. JIRA supports reading form fields from URL parameters, so this would be fairly straightforward:
  https://issues.apache.org/jira/secure/CreateIssueDetails!init.jspa?pid=12319525&issuetype=1&summary=[NIGHTLY:gandiva-jar-osx,gandiva-jar-trusty]&versions=12340948&fixVersions=12347769&components=12334626

Sub-Tasks

• [Dev] Add a way for people to add information to our saved crossbow data: Open