[ARROW-14651] [Release] "archery crossbow download-artifacts" raises read timeout error

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 8.0.0
• Component/s: Developer Tools
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/30194
• Assignee: Kouhei Sutou
• Reporter: Kouhei Sutou
• Created: 10/Nov/21 08:03
• Updated: 11/Jan/23 08:41
• Resolved: 26/Apr/22 10:20

Description

I couldn't all artifacts for 6.0.1 RC1...

Downloading release-6.0.1-rc0-0's artifacts.
Destination directory is /home/kou/work/cpp/arrow.kou/packages/release-6.0.1-rc0-0

[  state] Task / Branch                                               Artifacts
-------------------------------------------------------------------------------
[SUCCESS] debian-bookworm-amd64                                uploaded 70 / 70
 └ https://github.com/ursacomputing/crossbow/runs/4111265571?check_suite_focus=true
                        apache-arrow-apt-source_6.0.1-1.debian.tar.xz [     OK]
                                  apache-arrow-apt-source_6.0.1-1.dsc [     OK]
                              apache-arrow-apt-source_6.0.1-1_all.deb [     OK]
                            apache-arrow-apt-source_6.0.1.orig.tar.gz [     OK]
                                   apache-arrow_6.0.1-1.debian.tar.xz [     OK]
                                             apache-arrow_6.0.1-1.dsc [     OK]
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/urllib3/response.py", line 438, in _error_catcher
    yield
  File "/usr/lib/python3/dist-packages/urllib3/response.py", line 519, in read
    data = self._fp.read(amt) if not fp_closed else b""
  File "/usr/lib/python3.9/http/client.py", line 462, in read
    n = self.readinto(b)
  File "/usr/lib/python3.9/http/client.py", line 506, in readinto
    n = self.fp.readinto(b)
  File "/usr/lib/python3.9/socket.py", line 704, in readinto
    return self._sock.recv_into(b)
  File "/usr/lib/python3.9/ssl.py", line 1241, in recv_into
    return self.read(nbytes, buffer)
  File "/usr/lib/python3.9/ssl.py", line 1099, in read
    return self._sslobj.read(len, buffer)
socket.timeout: The read operation timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/requests/models.py", line 753, in generate
    for chunk in self.raw.stream(chunk_size, decode_content=True):
  File "/usr/lib/python3/dist-packages/urllib3/response.py", line 576, in stream
    data = self.read(amt=amt, decode_content=decode_content)
  File "/usr/lib/python3/dist-packages/urllib3/response.py", line 541, in read
    raise IncompleteRead(self._fp_bytes_read, self.length_remaining)
  File "/usr/lib/python3.9/contextlib.py", line 137, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/usr/lib/python3/dist-packages/urllib3/response.py", line 443, in _error_catcher
    raise ReadTimeoutError(self._pool, None, "Read timed out.")
urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='github-releases.githubusercontent.com', port=443): Read timed out.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/archery", line 33, in <module>
    sys.exit(load_entry_point('archery', 'console_scripts', 'archery')())
  File "/usr/local/lib/python3.9/dist-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.9/dist-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.9/dist-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.9/dist-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.9/dist-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.9/dist-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.9/dist-packages/click/decorators.py", line 33, in new_func
    return f(get_current_context().obj, *args, **kwargs)
  File "/home/kou/work/cpp/arrow.kou/dev/archery/archery/crossbow/cli.py", line 349, in download_artifacts
    report.show(
  File "/home/kou/work/cpp/arrow.kou/dev/archery/archery/crossbow/reports.py", line 132, in show
    asset_callback(task_name, task, asset)
  File "/home/kou/work/cpp/arrow.kou/dev/archery/archery/crossbow/cli.py", line 342, in asset_callback
    asset.download(path)
  File "/usr/local/lib/python3.9/dist-packages/github3/repos/release.py", line 405, in download
    return utils.stream_response_to_file(resp, path)
  File "/usr/local/lib/python3.9/dist-packages/github3/utils.py", line 84, in stream_response_to_file
    for chunk in response.iter_content(chunk_size=512):
  File "/usr/lib/python3/dist-packages/requests/models.py", line 760, in generate
    raise ConnectionError(e)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='github-releases.githubusercontent.com', port=443): Read timed out.

Can we retry on timeout automatically?

Issue Links

• links to: GitHub Pull Request #12996