Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-6027

bootstrap_toolchain can fail to download toolchain components

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: Impala 2.8.0, Impala 2.9.0, Impala 2.10.0, Impala 2.11.0
    • Fix Version/s: None
    • Component/s: Infrastructure
    • Labels:
      None

      Description

      Automated systems that build Impala can fail when downloading toolchain components.

      Downloading avro-1.7.4-p4-gcc-4.9.2-ec2-package-sles-12.tar.gz to toolchain
      Traceback (most recent call last):
        File "bin/bootstrap_toolchain.py", line 357, in <module>
          bootstrap(toolchain_root, packages)
        File "bin/bootstrap_toolchain.py", line 118, in bootstrap
          download_package(toolchain_root, pkg_name, pkg_version, compiler)
        File "bin/bootstrap_toolchain.py", line 100, in download_package
          wget_and_unpack_package(download_path, file_name, destination, True)
        File "bin/bootstrap_toolchain.py", line 82, in wget_and_unpack_package
          sh.wget(download_path, directory_prefix=destination, no_clobber=wget_no_clobber)
        File "infra/python/env/lib/python2.7/site-packages/sh.py", line 1021, in __call__
          return RunningCommand(cmd, call_args, stdin, stdout, stderr)
        File "infra/python/env/lib/python2.7/site-packages/sh.py", line 486, in __init__
          self.wait()
        File "infra/python/env/lib/python2.7/site-packages/sh.py", line 500, in wait
          self.handle_command_exit_code(exit_code)
        File "infra/python/env/lib/python2.7/site-packages/sh.py", line 516, in handle_command_exit_code
          raise exc(self.ran, self.process.stdout, self.process.stderr)
      sh.ErrorReturnCode_8:
      
        RAN: '/usr/bin/wget https://native-toolchain.s3.amazonaws.com/build/474-6c406b4a88/avro/1.7.4-p4-gcc-4.9.2/avro-1.7.4-p4-gcc-4.9.2-ec2-package-sles-12.tar.gz --directory-prefix=toolchain --no-clobber'
      
        STDOUT:
      
      
        STDERR:
      --2017-10-09 05:33:07--  https://native-toolchain.s3.amazonaws.com/build/474-6c406b4a88/avro/1.7.4-p4-gcc-4.9.2/avro-1.7.4-p4-gcc-4.9.2-ec2-package-sles-12.tar.gz
      Resolving native-toolchain.s3.amazonaws.com (native-toolchain.s3.amazonaws.com)... 54.231.237.47
      Connecting to native-toolchain.s3.amazonaws.com (native-toolchain.s3.amazonaws.com)|54.231.237.47|:443... connected.
      HTTP request sent, awaiting response... 500 Internal Server Error
      2017-10-09 05:33:07 ERROR 500: Internal Server Error.
      

      Seems like this 500-series of error could be retried after some delay. A comprehensive solution has to consider all supported platforms. That means any wget solution presented must work in CentOS 5 / SLES 11 in addition to Ubuntu 16. An alternative is to check error code of the wget process and retry in Python.

        Attachments

          Activity

            People

            • Assignee:
              philip Philip Martin
              Reporter:
              mikeb Michael Brown
            • Votes:
              1 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: