[ARROW-16684] [CI][Archery] Add retry mechanism to git fetch

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 9.0.0
• Component/s: Archery, Continuous Integration, Developer Tools
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/32029
• Assignee: Raúl Cumplido
• Reporter: Raúl Cumplido
• Created: 30/May/22 09:00
• Updated: 11/Jan/23 11:45
• Resolved: 30/May/22 14:32

Description

Archery seems to fail sometimes to fetch branches for some repositories. Some of the report packaging jobs (https://github.com/ursacomputing/crossbow/runs/6643769198?check_suite_focus=true) have been failing due to git errors when fetching:

   File "/home/runner/work/crossbow/crossbow/arrow/dev/archery/archery/crossbow/cli.py", line 238, in latest_prefix
    queue.fetch()
  File "/home/runner/work/crossbow/crossbow/arrow/dev/archery/archery/crossbow/core.py", line 271, in fetch
    self.origin.fetch([refspec])
  File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/pygit2/remote.py", line 146, in fetch
    payload.check_error(err)
  File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/pygit2/callbacks.py", line 93, in check_error
    check_error(error_code)
  File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/pygit2/errors.py", line 65, in check_error
    raise GitError(message)
_pygit2.GitError: SSL error: received early EOF
Error: Process completed with exit code 1.

I have seen that retrying the job can make it pass.

We should add a retry mechanism to archery to allow retry on GitErrors when fetching branches.

Issue Links

• links to: GitHub Pull Request #13258