[ARROW-8043] [Developer] Provide better visibility for failed nightly builds

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.16.0
• Component/s: Continuous Integration, Developer Tools
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/24257
• Assignee: Sam Albers
• Reporter: Ben Kietzman
• Created: 09/Mar/20 20:10
• Updated: 11/Jan/23 07:57
• Resolved: 13/Jul/22 15:57

Description

Emails reporting nightly failures are unsatisfactory in two ways: there is a large click/scroll distance between the links presented in that email and the actual error message. Worse, once one is there it's not clear what JIRAs have been made or which of them are in progress.

One solution would be to replace or augment the [NIGHTLY] email with a page (https://ursa-labs.github.org/crossbow would be my favorite) which shows how many nights it has failed, a shortcut to the actual error line in CI's logs, and useful views of JIRA. We could accomplish this with:

• dedicated JIRA tags; one for each nightly job so a JIRA can be easily associated with specific jobs
• A static HTML dashboard with client side JavaScript to
  • scrape JIRA and update the page dynamically as soon as JIRAs are opened
  • show any relationships between failing jobs
  • highlight jobs that have not been addressed, along with a counter of how many nights it has gone unaddressed
• provide automatic and expedited creation of correctly labelled JIRAs, so that viewers can quickly organize/take ownership of a failed nightly job. JIRA supports reading form fields from URL parameters, so this would be fairly straightforward:
  https://issues.apache.org/jira/secure/CreateIssueDetails!init.jspa?pid=12319525&issuetype=1&summary=[NIGHTLY:gandiva-jar-osx,gandiva-jar-trusty]&versions=12340948&fixVersions=12347769&components=12334626

Sub-Tasks

• [CI][Crossbow] Add a status.json to the gh-pages summary of nightly builds to get around rate limiting: Resolved
• Github data scraping for issue status: Resolved
• Create front end UI to display and summarize information about nightly jobs branches: Resolved
• Download data log result to external repository (firebase / others): Resolved
• Add a column to show us the number of time that this job is failing: Closed
• Deploy web app: Resolved
• [Dev] Add a way for people to add information to our saved crossbow data: Open