Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-11359

bootstrap_toolchain.py failed to download Apache Hive3

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • Infrastructure
    • None
    • ghx-label-13

    Description

      I try building with Apache Hive 3 locally and hit this error:

      $ export USE_APACHE_HIVE=true
      $ ./buildall.sh -noclean -notests -ninja
      IMPALA_VERSION          = 4.2.0-SNAPSHOT
      IMPALA_HOME             = /home/quanlong/workspace/Impala
      HADOOP_HOME             = /home/quanlong/workspace/Impala/toolchain/cdp_components-26065553/hadoop-3.1.1.7.2.16.0-33/
      HADOOP_CONF_DIR         = /home/quanlong/workspace/Impala/fe/src/test/resources
      HADOOP_INCLUDE_DIR      = /home/quanlong/workspace/Impala/toolchain/cdp_components-26065553/hadoop-3.1.1.7.2.16.0-33//include
      HADOOP_LIB_DIR          = /home/quanlong/workspace/Impala/toolchain/cdp_components-26065553/hadoop-3.1.1.7.2.16.0-33//lib
      IMPALA_CLUSTER_NODES_DIR= /home/quanlong/workspace/Impala/testdata/cluster/cdh7
      HIVE_HOME               = /home/quanlong/workspace/Impala/toolchain/apache_components/apache-hive-3.1.2-bin
      HIVE_CONF_DIR           = /home/quanlong/workspace/Impala/fe/./src/test/resources
      HIVE_SRC_DIR            = /home/quanlong/workspace/Impala/toolchain/apache_components/apache-hive-3.1.2-src
      HBASE_HOME              = /home/quanlong/workspace/Impala/toolchain/cdp_components-26065553/hbase-2.4.6.7.2.16.0-33/
      HBASE_CONF_DIR          = /home/quanlong/workspace/Impala/fe/src/test/resources
      RANGER_HOME             = /home/quanlong/workspace/Impala/toolchain/cdp_components-26065553/ranger-2.1.0.7.2.16.0-33-admin
      RANGER_CONF_DIR         = /home/quanlong/workspace/Impala/fe/src/test/resources 
      THRIFT_HOME             = /home/quanlong/workspace/Impala/toolchain/toolchain-packages-gcc7.5.0/thrift-0.11.0-p5
      CLASSPATH               = /home/quanlong/workspace/Impala/fe/src/test/resources:/home/quanlong/workspace/Impala/fe/target/classes:/home/quanlong/workspace/Impala/fe/target/dependency::/home/quanlong/workspace/Impala/fe/src/test/resources:/home/quanlong/workspace/Impala/fe/target/classes:/home/quanlong/workspace/Impala/fe/target/dependency::/home/quanlong/workspace/Impala/fe/src/test/resources:/home/quanlong/workspace/Impala/fe/target/classes:/home/quanlong/workspace/Impala/fe/target/dependency:
      LIBHDFS_OPTS            =  -Djava.library.path=/home/quanlong/workspace/Impala/toolchain/cdp_components-26065553/hadoop-3.1.1.7.2.16.0-33//lib/native/ -XX:ErrorFile=/home/quanlong/workspace/Impala/logs/hs_err_pid%p.log -XX:MaxPermSize=128m -Djava.library.path=/home/quanlong/workspace/Impala/toolchain/cdp_components-26065553/hadoop-3.1.1.7.2.16.0-33//lib/native/ -XX:ErrorFile=/home/quanlong/workspace/Impala/logs/hs_err_pid%p.log -XX:MaxPermSize=128m -Djava.library.path=/home/quanlong/workspace/Impala/toolchain/cdp_components-26065553/hadoop-3.1.1.7.2.16.0-33//lib/native/ -XX:ErrorFile=/home/quanlong/workspace/Impala/logs/hs_err_pid%p.log -XX:MaxPermSize=128m
      JAVA_HOME               = /usr/lib/jvm/java-8-openjdk-amd64
      POSTGRES_JDBC_DRIVER    = /home/quanlong/workspace/Impala/fe/target/dependency/postgresql-42.3.3.jar
      IMPALA_TOOLCHAIN        = /home/quanlong/workspace/Impala/toolchain
      IMPALA_TOOLCHAIN_PACKAGES_HOME = /home/quanlong/workspace/Impala/toolchain/toolchain-packages-gcc7.5.0
      METASTORE_DB            = HMS_home_quanlong_workspace_Impala_cdp
      DOWNLOAD_CDH_COMPONENTS = true
      IMPALA_MAVEN_OPTIONS    = 
      IMPALA_TOOLCHAIN_HOST   = native-toolchain.s3.amazonaws.com
      CDP_BUILD_NUMBER        = 26065553
      CDP_COMPONENTS_HOME     = /home/quanlong/workspace/Impala/toolchain/cdp_components-26065553
      APACHE_MIRROR           = https://www.apache.org/dyn/closer.cgi
      APACHE_COMPONENTS_HOME  = /home/quanlong/workspace/Impala/toolchain/apache_components
      IMPALA_HADOOP_VERSION   = 3.1.1.7.2.16.0-33
      IMPALA_AVRO_JAVA_VERSION= 1.8.2.7.2.16.0-33
      IMPALA_PARQUET_VERSION  = 1.10.99.7.2.16.0-33
      IMPALA_HIVE_VERSION     = 3.1.2
      IMPALA_HBASE_VERSION    = 2.4.6.7.2.16.0-33
      IMPALA_HUDI_VERSION     = 0.5.0-incubating
      IMPALA_KUDU_VERSION     = 09a6c8833e
      IMPALA_RANGER_VERSION   = 2.1.0.7.2.16.0-33
      IMPALA_ICEBERG_VERSION  = 0.13.1.7.2.16.0-33
      IMPALA_GCS_VERSION      = 2.1.2.7.2.16.0-33
      IMPALA_COS_VERSION      = 3.1.0-5.9.3
      SKIP_PYTHON_DOWNLOAD is true, skipping python dependencies download.
      SKIP_TOOLCHAIN_BOOTSTRAP is true, skipping toolchain bootstrap.
      >>> Downloading and extracting cdh components.
      2022-06-16 08:12:30,701 Thread-2 INFO: Downloading https://www.apache.org/dyn/closer.cgi/hive/hive-3.1.2/apache-hive-3.1.2-bin.tar.gz?action=download to /home/quanlong/workspace/Impala/toolchain/apache_components/apache-hive-3.1.2-bin.tar.gz (attempt 1)
      2022-06-16 08:12:30,702 Thread-1 INFO: Downloading https://www.apache.org/dyn/closer.cgi/hive/hive-3.1.2/apache-hive-3.1.2-src.tar.gz?action=download to /home/quanlong/workspace/Impala/toolchain/apache_components/apache-hive-3.1.2-src.tar.gz (attempt 1)
      2022-06-16 08:12:31,534 Thread-2 ERROR: Download failed; retrying after sleep: Command with args '['wget', 'https://www.apache.org/dyn/closer.cgi/hive/hive-3.1.2/apache-hive-3.1.2-bin.tar.gz?action=download', '--output-document=/home/quanlong/workspace/Impala/toolchain/apache_components/apache-hive-3.1.2-bin.tar.gz']' failed with exit code 5:
      --2022-06-16 08:12:30--  https://www.apache.org/dyn/closer.cgi/hive/hive-3.1.2/apache-hive-3.1.2-bin.tar.gz?action=download
      Resolving www.apache.org (www.apache.org)... 151.101.2.132, 2a04:4e42::644
      Connecting to www.apache.org (www.apache.org)|151.101.2.132|:443... connected.
      ERROR: cannot verify www.apache.org's certificate, issued by ‘CN=R3,O=Let's Encrypt,C=US’:
        Issued certificate has expired.
      To connect to www.apache.org insecurely, use `--no-check-certificate'.
      
      2022-06-16 08:12:31,535 Thread-1 ERROR: Download failed; retrying after sleep: Command with args '['wget', 'https://www.apache.org/dyn/closer.cgi/hive/hive-3.1.2/apache-hive-3.1.2-src.tar.gz?action=download', '--output-document=/home/quanlong/workspace/Impala/toolchain/apache_components/apache-hive-3.1.2-src.tar.gz']' failed with exit code 5:
      --2022-06-16 08:12:30--  https://www.apache.org/dyn/closer.cgi/hive/hive-3.1.2/apache-hive-3.1.2-src.tar.gz?action=download
      Resolving www.apache.org (www.apache.org)... 151.101.2.132, 2a04:4e42::644
      Connecting to www.apache.org (www.apache.org)|151.101.2.132|:443... connected.
      ERROR: cannot verify www.apache.org's certificate, issued by ‘CN=R3,O=Let's Encrypt,C=US’:
        Issued certificate has expired.
      To connect to www.apache.org insecurely, use `--no-check-certificate'.
      
      2022-06-16 08:12:43,924 Thread-1 INFO: Downloading https://www.apache.org/dyn/closer.cgi/hive/hive-3.1.2/apache-hive-3.1.2-src.tar.gz?action=download to /home/quanlong/workspace/Impala/toolchain/apache_components/apache-hive-3.1.2-src.tar.gz (attempt 2)
      2022-06-16 08:12:44,749 Thread-1 ERROR: Download failed; retrying after sleep: Command with args '['wget', 'https://www.apache.org/dyn/closer.cgi/hive/hive-3.1.2/apache-hive-3.1.2-src.tar.gz?action=download', '--output-document=/home/quanlong/workspace/Impala/toolchain/apache_components/apache-hive-3.1.2-src.tar.gz']' failed with exit code 5:
      --2022-06-16 08:12:43--  https://www.apache.org/dyn/closer.cgi/hive/hive-3.1.2/apache-hive-3.1.2-src.tar.gz?action=download
      Resolving www.apache.org (www.apache.org)... 151.101.2.132, 2a04:4e42::644
      Connecting to www.apache.org (www.apache.org)|151.101.2.132|:443... connected.
      ERROR: cannot verify www.apache.org's certificate, issued by ‘CN=R3,O=Let's Encrypt,C=US’:
        Issued certificate has expired.
      To connect to www.apache.org insecurely, use `--no-check-certificate'.
      
      2022-06-16 08:12:45,670 Thread-2 INFO: Downloading https://www.apache.org/dyn/closer.cgi/hive/hive-3.1.2/apache-hive-3.1.2-bin.tar.gz?action=download to /home/quanlong/workspace/Impala/toolchain/apache_components/apache-hive-3.1.2-bin.tar.gz (attempt 2)
      2022-06-16 08:12:46,503 Thread-2 ERROR: Download failed; retrying after sleep: Command with args '['wget', 'https://www.apache.org/dyn/closer.cgi/hive/hive-3.1.2/apache-hive-3.1.2-bin.tar.gz?action=download', '--output-document=/home/quanlong/workspace/Impala/toolchain/apache_components/apache-hive-3.1.2-bin.tar.gz']' failed with exit code 5:
      --2022-06-16 08:12:45--  https://www.apache.org/dyn/closer.cgi/hive/hive-3.1.2/apache-hive-3.1.2-bin.tar.gz?action=download
      Resolving www.apache.org (www.apache.org)... 151.101.2.132, 2a04:4e42::644
      Connecting to www.apache.org (www.apache.org)|151.101.2.132|:443... connected.
      ERROR: cannot verify www.apache.org's certificate, issued by ‘CN=R3,O=Let's Encrypt,C=US’:
        Issued certificate has expired.
      To connect to www.apache.org insecurely, use `--no-check-certificate'.
      
      2022-06-16 08:12:55,981 Thread-1 INFO: Downloading https://www.apache.org/dyn/closer.cgi/hive/hive-3.1.2/apache-hive-3.1.2-src.tar.gz?action=download to /home/quanlong/workspace/Impala/toolchain/apache_components/apache-hive-3.1.2-src.tar.gz (attempt 3)
      Traceback (most recent call last):
        File "/home/quanlong/workspace/Impala/bin/bootstrap_toolchain.py", line 568, in <module>
          if __name__ == "__main__": main()
        File "/home/quanlong/workspace/Impala/bin/bootstrap_toolchain.py", line 565, in main
          execute_many(download, components_needing_download)
        File "/home/quanlong/workspace/Impala/bin/bootstrap_toolchain.py", line 441, in execute_many
          return pool.map(f, args, 1)
        File "/usr/lib/python2.7/multiprocessing/pool.py", line 251, in map
          return self.map_async(func, iterable, chunksize).get()
        File "/usr/lib/python2.7/multiprocessing/pool.py", line 567, in get
          raise self._value
      Exception: Command with args '['wget', 'https://www.apache.org/dyn/closer.cgi/hive/hive-3.1.2/apache-hive-3.1.2-src.tar.gz?action=download', '--output-document=/home/quanlong/workspace/Impala/toolchain/apache_components/apache-hive-3.1.2-src.tar.gz']' failed with exit code 5:
      --2022-06-16 08:12:55--  https://www.apache.org/dyn/closer.cgi/hive/hive-3.1.2/apache-hive-3.1.2-src.tar.gz?action=download
      Resolving www.apache.org (www.apache.org)... 151.101.2.132, 2a04:4e42::644
      Connecting to www.apache.org (www.apache.org)|151.101.2.132|:443... connected.
      ERROR: cannot verify www.apache.org's certificate, issued by ‘CN=R3,O=Let's Encrypt,C=US’:
        Issued certificate has expired.
      To connect to www.apache.org insecurely, use `--no-check-certificate'.

      CC chufucun

      Attachments

        Activity

          People

            stigahuang Quanlong Huang
            stigahuang Quanlong Huang
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: