[ARROW-13059] [Dev] Adapt the crossbow code to save build status to json

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Duplicate
• Component/s: Developer Tools
• External issue URL: https://github.com/apache/arrow/issues/28768
• Assignee: Mauricio 'Pachá' Vargas Sepúlveda
• Reporter: Jonathan Keane
• Created: 11/Jun/21 17:16
• Updated: 11/Jan/23 08:30
• Resolved: 21/Oct/21 13:05

Description

Add to / adapt the code that archery crossbow already uses to send the email report to also save the status of the builds to a json file and commit that to a new branch in the crossbow (or some other ursacomputing repository)

Crossbow code (hint this is the code that you will want to copy + adapt to do this new task): https://github.com/apache/arrow/blob/master/dev/archery/archery/crossbow/reports.py

This is how the nightly jobs are triggered: https://github.com/ursacomputing/crossbow/blob/master/.github/workflows/nightly_report.yml
(note that it figures out what the job id is and then it runs a command archery crossbow report ...

The archer CLI interface is specifiedin https://github.com/apache/arrow/blob/master/dev/archery/archery/crossbow/cli.py

Ultimately what we want is something like: a new command like crossbow archery save-report-data that uses similar code/approaches to how the report is designed but saves the data to json (or line delimited json) and saves that somewhere reliable (i.e. the ursacomputing/crossbow repository or a new repository under ursacomputing)