Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-1010

Implement INFORMATION_SCHEMA in Hive

    Details

      Description

      INFORMATION_SCHEMA is part of the SQL92 standard and would be useful to implement using our metastore.

      1. HIVE-1010.10.patch
        180 kB
        Gunther Hagleitner
      2. HIVE-1010.11.patch
        186 kB
        Gunther Hagleitner
      3. HIVE-1010.7.patch
        158 kB
        Gunther Hagleitner
      4. HIVE-1010.8.patch
        158 kB
        Gunther Hagleitner
      5. HIVE-1010.9.patch
        158 kB
        Gunther Hagleitner

        Issue Links

          Activity

          Hide
          cwsteinbach Carl Steinbach added a comment -

          MySQL INFORMATION_SCHEMA views: http://dev.mysql.com/doc/refman/5.1/en/information-schema.html

          There's been some discussion offline about supporting this using the JDO query language, i.e. expressing HiveQL queries against the INFORMATION_SCHEMA as JDOQL queries against the metastore.

          Additionally, supporting an INFORMATION_SCHEMA is probably the biggest missing piece that blocks grafting the PostgreSQL ODBC driver onto Hive (as has been done for LucidDB, H2, etc).

          Show
          cwsteinbach Carl Steinbach added a comment - MySQL INFORMATION_SCHEMA views: http://dev.mysql.com/doc/refman/5.1/en/information-schema.html There's been some discussion offline about supporting this using the JDO query language, i.e. expressing HiveQL queries against the INFORMATION_SCHEMA as JDOQL queries against the metastore. Additionally, supporting an INFORMATION_SCHEMA is probably the biggest missing piece that blocks grafting the PostgreSQL ODBC driver onto Hive (as has been done for LucidDB, H2, etc).
          Hide
          jvs John Sichi added a comment -

          Can we get the native SQL translation from JDO and then wrap this to get the data out of the metastore?

          http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/mapred/lib/db/DBInputFormat.html

          Then use HIve views to map metastore representation to INFORMATION_SCHEMA.

          Show
          jvs John Sichi added a comment - Can we get the native SQL translation from JDO and then wrap this to get the data out of the metastore? http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/mapred/lib/db/DBInputFormat.html Then use HIve views to map metastore representation to INFORMATION_SCHEMA.
          Hide
          jvs John Sichi added a comment -

          See HIVE-1555 for a JDBC storage handler.

          Show
          jvs John Sichi added a comment - See HIVE-1555 for a JDBC storage handler.
          Hide
          hagleitn Gunther Hagleitner added a comment -

          WIP. Based on patch in HIVE-1555. SCHEMATA, TABLES, and COLUMNS are there.

          Show
          hagleitn Gunther Hagleitner added a comment - WIP. Based on patch in HIVE-1555 . SCHEMATA, TABLES, and COLUMNS are there.
          Hide
          hiveqa Hive QA added a comment -

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12853414/HIVE-1010.5.patch

          ERROR: -1 due to build exiting with an error

          Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3648/testReport
          Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3648/console
          Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3648/

          Messages:

          Executing org.apache.hive.ptest.execution.TestCheckPhase
          Executing org.apache.hive.ptest.execution.PrepPhase
          Tests exited with: NonZeroExitCodeException
          Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit status 1 and output '+ date '+%Y-%m-%d %T.%3N'
          2017-02-18 11:12:01.027
          + [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
          + export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
          + JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
          + export PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
          + PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
          + export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
          + ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
          + export 'MAVEN_OPTS=-Xmx1g '
          + MAVEN_OPTS='-Xmx1g '
          + cd /data/hiveptest/working/
          + tee /data/hiveptest/logs/PreCommit-HIVE-Build-3648/source-prep.txt
          + [[ false == \t\r\u\e ]]
          + mkdir -p maven ivy
          + [[ git = \s\v\n ]]
          + [[ git = \g\i\t ]]
          + [[ -z master ]]
          + [[ -d apache-github-source-source ]]
          + [[ ! -d apache-github-source-source/.git ]]
          + [[ ! -d apache-github-source-source ]]
          + date '+%Y-%m-%d %T.%3N'
          2017-02-18 11:12:01.030
          + cd apache-github-source-source
          + git fetch origin
          + git reset --hard HEAD
          HEAD is now at 3f986d7 HIVE-15919: Row count mismatch for count * query (Jason Dere, reviewed by Matt McCline)
          + git clean -f -d
          Removing ql/src/test/queries/clientpositive/llap_reader.q
          Removing ql/src/test/results/clientpositive/llap/llap_reader.q.out
          Removing ql/src/test/results/clientpositive/llap_reader.q.out
          + git checkout master
          Already on 'master'
          Your branch is up-to-date with 'origin/master'.
          + git reset --hard origin/master
          HEAD is now at 3f986d7 HIVE-15919: Row count mismatch for count * query (Jason Dere, reviewed by Matt McCline)
          + git merge --ff-only origin/master
          Already up-to-date.
          + date '+%Y-%m-%d %T.%3N'
          2017-02-18 11:12:02.061
          + patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
          + patchFilePath=/data/hiveptest/working/scratch/build.patch
          + [[ -f /data/hiveptest/working/scratch/build.patch ]]
          + chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
          + /data/hiveptest/working/scratch/smart-apply-patch.sh /data/hiveptest/working/scratch/build.patch
          error: jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/JdbcRecordReader.java: No such file or directory
          error: jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/JdbcSerDe.java: No such file or directory
          error: jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/conf/DatabaseType.java: No such file or directory
          error: jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/conf/JdbcStorageConfigManager.java: No such file or directory
          error: jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/dao/DatabaseAccessorFactory.java: No such file or directory
          error: jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/dao/JdbcRecordIterator.java: No such file or directory
          error: ql/src/test/queries/clientpositive/jdbc_handler.q: No such file or directory
          error: ql/src/test/results/clientpositive/jdbc_handler.q.out: No such file or directory
          error: ql/src/test/results/clientpositive/llap/jdbc_handler.q.out: No such file or directory
          The patch does not appear to apply with p0, p1, or p2
          + exit 1
          '
          

          This message is automatically generated.

          ATTACHMENT ID: 12853414 - PreCommit-HIVE-Build

          Show
          hiveqa Hive QA added a comment - Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12853414/HIVE-1010.5.patch ERROR: -1 due to build exiting with an error Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3648/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3648/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3648/ Messages: Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Tests exited with: NonZeroExitCodeException Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit status 1 and output '+ date '+%Y-%m-%d %T.%3N' 2017-02-18 11:12:01.027 + [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]] + export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 + JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 + export PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games + PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games + export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m ' + ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m ' + export 'MAVEN_OPTS=-Xmx1g ' + MAVEN_OPTS='-Xmx1g ' + cd /data/hiveptest/working/ + tee /data/hiveptest/logs/PreCommit-HIVE-Build-3648/source-prep.txt + [[ false == \t\r\u\e ]] + mkdir -p maven ivy + [[ git = \s\v\n ]] + [[ git = \g\i\t ]] + [[ -z master ]] + [[ -d apache-github-source-source ]] + [[ ! -d apache-github-source-source/.git ]] + [[ ! -d apache-github-source-source ]] + date '+%Y-%m-%d %T.%3N' 2017-02-18 11:12:01.030 + cd apache-github-source-source + git fetch origin + git reset --hard HEAD HEAD is now at 3f986d7 HIVE-15919: Row count mismatch for count * query (Jason Dere, reviewed by Matt McCline) + git clean -f -d Removing ql/src/test/queries/clientpositive/llap_reader.q Removing ql/src/test/results/clientpositive/llap/llap_reader.q.out Removing ql/src/test/results/clientpositive/llap_reader.q.out + git checkout master Already on 'master' Your branch is up-to-date with 'origin/master'. + git reset --hard origin/master HEAD is now at 3f986d7 HIVE-15919: Row count mismatch for count * query (Jason Dere, reviewed by Matt McCline) + git merge --ff-only origin/master Already up-to-date. + date '+%Y-%m-%d %T.%3N' 2017-02-18 11:12:02.061 + patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh + patchFilePath=/data/hiveptest/working/scratch/build.patch + [[ -f /data/hiveptest/working/scratch/build.patch ]] + chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh + /data/hiveptest/working/scratch/smart-apply-patch.sh /data/hiveptest/working/scratch/build.patch error: jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/JdbcRecordReader.java: No such file or directory error: jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/JdbcSerDe.java: No such file or directory error: jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/conf/DatabaseType.java: No such file or directory error: jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/conf/JdbcStorageConfigManager.java: No such file or directory error: jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/dao/DatabaseAccessorFactory.java: No such file or directory error: jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/dao/JdbcRecordIterator.java: No such file or directory error: ql/src/test/queries/clientpositive/jdbc_handler.q: No such file or directory error: ql/src/test/results/clientpositive/jdbc_handler.q.out: No such file or directory error: ql/src/test/results/clientpositive/llap/jdbc_handler.q.out: No such file or directory The patch does not appear to apply with p0, p1, or p2 + exit 1 ' This message is automatically generated. ATTACHMENT ID: 12853414 - PreCommit-HIVE-Build
          Hide
          hagleitn Gunther Hagleitner added a comment -

          (patch requires HIVE-1555) cc Thejas M Nair

          Show
          hagleitn Gunther Hagleitner added a comment - (patch requires HIVE-1555 ) cc Thejas M Nair
          Hide
          hiveqa Hive QA added a comment -

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12855325/HIVE-1010.6.patch

          SUCCESS: +1 due to 3 test(s) being added or modified.

          ERROR: -1 due to 3 failed/errored test(s), 10324 tests executed
          Failed tests:

          org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_text_vec_table] (batchId=147)
          org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_if_expr] (batchId=140)
          org.apache.hive.config.JdbcStorageConfigManagerTest.testWithJdbcUrlMissing (batchId=187)
          

          Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3866/testReport
          Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3866/console
          Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3866/

          Messages:

          Executing org.apache.hive.ptest.execution.TestCheckPhase
          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          Tests exited with: TestsFailedException: 3 tests failed
          

          This message is automatically generated.

          ATTACHMENT ID: 12855325 - PreCommit-HIVE-Build

          Show
          hiveqa Hive QA added a comment - Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12855325/HIVE-1010.6.patch SUCCESS: +1 due to 3 test(s) being added or modified. ERROR: -1 due to 3 failed/errored test(s), 10324 tests executed Failed tests: org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_text_vec_table] (batchId=147) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_if_expr] (batchId=140) org.apache.hive.config.JdbcStorageConfigManagerTest.testWithJdbcUrlMissing (batchId=187) Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3866/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3866/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3866/ Messages: Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 3 tests failed This message is automatically generated. ATTACHMENT ID: 12855325 - PreCommit-HIVE-Build
          Hide
          hagleitn Gunther Hagleitner added a comment -

          Thejas M Nair tests are passing. I still have to make the information schema definitions follow the standard better, but it's a start. I'm looking for feedback on the schema tool etc. The plan create a "hive" metastore folder that contains updates and schema definitions for the metastore db and information schema. Upgrade works by pointing the hive schema tool to a hs2 instance.

          Show
          hagleitn Gunther Hagleitner added a comment - Thejas M Nair tests are passing. I still have to make the information schema definitions follow the standard better, but it's a start. I'm looking for feedback on the schema tool etc. The plan create a "hive" metastore folder that contains updates and schema definitions for the metastore db and information schema. Upgrade works by pointing the hive schema tool to a hs2 instance.
          Hide
          hiveqa Hive QA added a comment -

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12855579/HIVE-1010.7.patch

          ERROR: -1 due to build exiting with an error

          Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3889/testReport
          Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3889/console
          Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3889/

          Messages:

          Executing org.apache.hive.ptest.execution.TestCheckPhase
          Executing org.apache.hive.ptest.execution.PrepPhase
          Tests exited with: NonZeroExitCodeException
          Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit status 1 and output '+ date '+%Y-%m-%d %T.%3N'
          2017-03-02 09:41:30.907
          + [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
          + export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
          + JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
          + export PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
          + PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
          + export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
          + ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
          + export 'MAVEN_OPTS=-Xmx1g '
          + MAVEN_OPTS='-Xmx1g '
          + cd /data/hiveptest/working/
          + tee /data/hiveptest/logs/PreCommit-HIVE-Build-3889/source-prep.txt
          + [[ false == \t\r\u\e ]]
          + mkdir -p maven ivy
          + [[ git = \s\v\n ]]
          + [[ git = \g\i\t ]]
          + [[ -z master ]]
          + [[ -d apache-github-source-source ]]
          + [[ ! -d apache-github-source-source/.git ]]
          + [[ ! -d apache-github-source-source ]]
          + date '+%Y-%m-%d %T.%3N'
          2017-03-02 09:41:30.910
          + cd apache-github-source-source
          + git fetch origin
          + git reset --hard HEAD
          HEAD is now at 4c7f2d9 HIVE-15882: HS2 generating high memory pressure with many partitions and concurrent queries (Misha Dmitriev, reviewed by Sahil, Mohit, Vihang and Rui)
          + git clean -f -d
          + git checkout master
          Already on 'master'
          Your branch is up-to-date with 'origin/master'.
          + git reset --hard origin/master
          HEAD is now at 4c7f2d9 HIVE-15882: HS2 generating high memory pressure with many partitions and concurrent queries (Misha Dmitriev, reviewed by Sahil, Mohit, Vihang and Rui)
          + git merge --ff-only origin/master
          Already up-to-date.
          + date '+%Y-%m-%d %T.%3N'
          2017-03-02 09:41:38.265
          + patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
          + patchFilePath=/data/hiveptest/working/scratch/build.patch
          + [[ -f /data/hiveptest/working/scratch/build.patch ]]
          + chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
          + /data/hiveptest/working/scratch/smart-apply-patch.sh /data/hiveptest/working/scratch/build.patch
          error: patch failed: ql/src/java/org/apache/hadoop/hive/ql/plan/PartitionDesc.java:120
          error: ql/src/java/org/apache/hadoop/hive/ql/plan/PartitionDesc.java: patch does not apply
          The patch does not appear to apply with p0, p1, or p2
          + exit 1
          '
          

          This message is automatically generated.

          ATTACHMENT ID: 12855579 - PreCommit-HIVE-Build

          Show
          hiveqa Hive QA added a comment - Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12855579/HIVE-1010.7.patch ERROR: -1 due to build exiting with an error Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3889/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3889/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3889/ Messages: Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Tests exited with: NonZeroExitCodeException Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit status 1 and output '+ date '+%Y-%m-%d %T.%3N' 2017-03-02 09:41:30.907 + [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]] + export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 + JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 + export PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games + PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games + export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m ' + ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m ' + export 'MAVEN_OPTS=-Xmx1g ' + MAVEN_OPTS='-Xmx1g ' + cd /data/hiveptest/working/ + tee /data/hiveptest/logs/PreCommit-HIVE-Build-3889/source-prep.txt + [[ false == \t\r\u\e ]] + mkdir -p maven ivy + [[ git = \s\v\n ]] + [[ git = \g\i\t ]] + [[ -z master ]] + [[ -d apache-github-source-source ]] + [[ ! -d apache-github-source-source/.git ]] + [[ ! -d apache-github-source-source ]] + date '+%Y-%m-%d %T.%3N' 2017-03-02 09:41:30.910 + cd apache-github-source-source + git fetch origin + git reset --hard HEAD HEAD is now at 4c7f2d9 HIVE-15882: HS2 generating high memory pressure with many partitions and concurrent queries (Misha Dmitriev, reviewed by Sahil, Mohit, Vihang and Rui) + git clean -f -d + git checkout master Already on 'master' Your branch is up-to-date with 'origin/master'. + git reset --hard origin/master HEAD is now at 4c7f2d9 HIVE-15882: HS2 generating high memory pressure with many partitions and concurrent queries (Misha Dmitriev, reviewed by Sahil, Mohit, Vihang and Rui) + git merge --ff-only origin/master Already up-to-date. + date '+%Y-%m-%d %T.%3N' 2017-03-02 09:41:38.265 + patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh + patchFilePath=/data/hiveptest/working/scratch/build.patch + [[ -f /data/hiveptest/working/scratch/build.patch ]] + chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh + /data/hiveptest/working/scratch/smart-apply-patch.sh /data/hiveptest/working/scratch/build.patch error: patch failed: ql/src/java/org/apache/hadoop/hive/ql/plan/PartitionDesc.java:120 error: ql/src/java/org/apache/hadoop/hive/ql/plan/PartitionDesc.java: patch does not apply The patch does not appear to apply with p0, p1, or p2 + exit 1 ' This message is automatically generated. ATTACHMENT ID: 12855579 - PreCommit-HIVE-Build
          Hide
          hagleitn Gunther Hagleitner added a comment -

          .8 is a rebase

          Show
          hagleitn Gunther Hagleitner added a comment - .8 is a rebase
          Hide
          hiveqa Hive QA added a comment -

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12855758/HIVE-1010.8.patch

          ERROR: -1 due to build exiting with an error

          Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3917/testReport
          Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3917/console
          Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3917/

          Messages:

          Executing org.apache.hive.ptest.execution.TestCheckPhase
          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Tests exited with: ExecutionException: java.util.concurrent.ExecutionException: org.apache.hive.ptest.execution.ssh.SSHExecutionException: RSyncResult [localFile=/data/hiveptest/logs/PreCommit-HIVE-Build-3917/failed/272_UTBatch_ql_10_tests, remoteFile=/home/hiveptest/35.184.0.150-hiveptest-1/logs/, getExitCode()=11, getException()=null, getUser()=hiveptest, getHost()=35.184.0.150, getInstance()=1]: 'Warning: Permanently added '35.184.0.150' (ECDSA) to the list of known hosts.
          receiving incremental file list
          ./
          TEST-272_UTBatch_ql_10_tests-TEST-org.apache.hadoop.hive.ql.TestTxnCommands.xml
          
                        0   0%    0.00kB/s    0:00:00  
                    8,230 100%    7.85MB/s    0:00:00 (xfr#1, to-chk=9/11)
          TEST-272_UTBatch_ql_10_tests-TEST-org.apache.hadoop.hive.ql.lockmgr.TestEmbeddedLockManager.xml
          
                        0   0%    0.00kB/s    0:00:00  
                    5,257 100%    5.01MB/s    0:00:00 (xfr#2, to-chk=8/11)
          TEST-272_UTBatch_ql_10_tests-TEST-org.apache.hadoop.hive.ql.lockmgr.TestHiveLockObject.xml
          
                        0   0%    0.00kB/s    0:00:00  
                    5,257 100%    5.01MB/s    0:00:00 (xfr#3, to-chk=7/11)
          TEST-272_UTBatch_ql_10_tests-TEST-org.apache.hadoop.hive.ql.lockmgr.zookeeper.TestZookeeperLockManager.xml
          
                        0   0%    0.00kB/s    0:00:00  
                    5,551 100%    5.29MB/s    0:00:00 (xfr#4, to-chk=6/11)
          maven-test.txt
          
                        0   0%    0.00kB/s    0:00:00  
                    6,475 100%    6.18MB/s    0:00:00 (xfr#5, to-chk=5/11)
          logs/
          logs/derby.log
          
                        0   0%    0.00kB/s    0:00:00  
                      966 100%  943.36kB/s    0:00:00 (xfr#6, to-chk=2/11)
          logs/hive.log
          
                        0   0%    0.00kB/s    0:00:00  
               36,077,568   0%   34.41MB/s    0:14:43  
          rsync: write failed on "/data/hiveptest/logs/PreCommit-HIVE-Build-3917/failed/272_UTBatch_ql_10_tests/logs/hive.log": No space left on device (28)
          rsync error: error in file IO (code 11) at receiver.c(393) [receiver=3.1.1]
          Warning: Permanently added '35.184.0.150' (ECDSA) to the list of known hosts.
          receiving incremental file list
          logs/
          logs/hive.log
          
                        0   0%    0.00kB/s    0:00:00  
          rsync: write failed on "/data/hiveptest/logs/PreCommit-HIVE-Build-3917/failed/272_UTBatch_ql_10_tests/logs/hive.log": No space left on device (28)
          rsync error: error in file IO (code 11) at receiver.c(393) [receiver=3.1.1]
          Warning: Permanently added '35.184.0.150' (ECDSA) to the list of known hosts.
          receiving incremental file list
          logs/
          logs/hive.log
          
                        0   0%    0.00kB/s    0:00:00  
          rsync: write failed on "/data/hiveptest/logs/PreCommit-HIVE-Build-3917/failed/272_UTBatch_ql_10_tests/logs/hive.log": No space left on device (28)
          rsync error: error in file IO (code 11) at receiver.c(393) [receiver=3.1.1]
          Warning: Permanently added '35.184.0.150' (ECDSA) to the list of known hosts.
          receiving incremental file list
          logs/
          logs/hive.log
          
                        0   0%    0.00kB/s    0:00:00  
          rsync: write failed on "/data/hiveptest/logs/PreCommit-HIVE-Build-3917/failed/272_UTBatch_ql_10_tests/logs/hive.log": No space left on device (28)
          rsync error: error in file IO (code 11) at receiver.c(393) [receiver=3.1.1]
          Warning: Permanently added '35.184.0.150' (ECDSA) to the list of known hosts.
          receiving incremental file list
          logs/
          logs/hive.log
          
                        0   0%    0.00kB/s    0:00:00  
          rsync: write failed on "/data/hiveptest/logs/PreCommit-HIVE-Build-3917/failed/272_UTBatch_ql_10_tests/logs/hive.log": No space left on device (28)
          rsync error: error in file IO (code 11) at receiver.c(393) [receiver=3.1.1]
          '
          

          This message is automatically generated.

          ATTACHMENT ID: 12855758 - PreCommit-HIVE-Build

          Show
          hiveqa Hive QA added a comment - Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12855758/HIVE-1010.8.patch ERROR: -1 due to build exiting with an error Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3917/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3917/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3917/ Messages: Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Tests exited with: ExecutionException: java.util.concurrent.ExecutionException: org.apache.hive.ptest.execution.ssh.SSHExecutionException: RSyncResult [localFile=/data/hiveptest/logs/PreCommit-HIVE-Build-3917/failed/272_UTBatch_ql_10_tests, remoteFile=/home/hiveptest/35.184.0.150-hiveptest-1/logs/, getExitCode()=11, getException()=null, getUser()=hiveptest, getHost()=35.184.0.150, getInstance()=1]: 'Warning: Permanently added '35.184.0.150' (ECDSA) to the list of known hosts. receiving incremental file list ./ TEST-272_UTBatch_ql_10_tests-TEST-org.apache.hadoop.hive.ql.TestTxnCommands.xml 0 0% 0.00kB/s 0:00:00 8,230 100% 7.85MB/s 0:00:00 (xfr#1, to-chk=9/11) TEST-272_UTBatch_ql_10_tests-TEST-org.apache.hadoop.hive.ql.lockmgr.TestEmbeddedLockManager.xml 0 0% 0.00kB/s 0:00:00 5,257 100% 5.01MB/s 0:00:00 (xfr#2, to-chk=8/11) TEST-272_UTBatch_ql_10_tests-TEST-org.apache.hadoop.hive.ql.lockmgr.TestHiveLockObject.xml 0 0% 0.00kB/s 0:00:00 5,257 100% 5.01MB/s 0:00:00 (xfr#3, to-chk=7/11) TEST-272_UTBatch_ql_10_tests-TEST-org.apache.hadoop.hive.ql.lockmgr.zookeeper.TestZookeeperLockManager.xml 0 0% 0.00kB/s 0:00:00 5,551 100% 5.29MB/s 0:00:00 (xfr#4, to-chk=6/11) maven-test.txt 0 0% 0.00kB/s 0:00:00 6,475 100% 6.18MB/s 0:00:00 (xfr#5, to-chk=5/11) logs/ logs/derby.log 0 0% 0.00kB/s 0:00:00 966 100% 943.36kB/s 0:00:00 (xfr#6, to-chk=2/11) logs/hive.log 0 0% 0.00kB/s 0:00:00 36,077,568 0% 34.41MB/s 0:14:43 rsync: write failed on "/data/hiveptest/logs/PreCommit-HIVE-Build-3917/failed/272_UTBatch_ql_10_tests/logs/hive.log": No space left on device (28) rsync error: error in file IO (code 11) at receiver.c(393) [receiver=3.1.1] Warning: Permanently added '35.184.0.150' (ECDSA) to the list of known hosts. receiving incremental file list logs/ logs/hive.log 0 0% 0.00kB/s 0:00:00 rsync: write failed on "/data/hiveptest/logs/PreCommit-HIVE-Build-3917/failed/272_UTBatch_ql_10_tests/logs/hive.log": No space left on device (28) rsync error: error in file IO (code 11) at receiver.c(393) [receiver=3.1.1] Warning: Permanently added '35.184.0.150' (ECDSA) to the list of known hosts. receiving incremental file list logs/ logs/hive.log 0 0% 0.00kB/s 0:00:00 rsync: write failed on "/data/hiveptest/logs/PreCommit-HIVE-Build-3917/failed/272_UTBatch_ql_10_tests/logs/hive.log": No space left on device (28) rsync error: error in file IO (code 11) at receiver.c(393) [receiver=3.1.1] Warning: Permanently added '35.184.0.150' (ECDSA) to the list of known hosts. receiving incremental file list logs/ logs/hive.log 0 0% 0.00kB/s 0:00:00 rsync: write failed on "/data/hiveptest/logs/PreCommit-HIVE-Build-3917/failed/272_UTBatch_ql_10_tests/logs/hive.log": No space left on device (28) rsync error: error in file IO (code 11) at receiver.c(393) [receiver=3.1.1] Warning: Permanently added '35.184.0.150' (ECDSA) to the list of known hosts. receiving incremental file list logs/ logs/hive.log 0 0% 0.00kB/s 0:00:00 rsync: write failed on "/data/hiveptest/logs/PreCommit-HIVE-Build-3917/failed/272_UTBatch_ql_10_tests/logs/hive.log": No space left on device (28) rsync error: error in file IO (code 11) at receiver.c(393) [receiver=3.1.1] ' This message is automatically generated. ATTACHMENT ID: 12855758 - PreCommit-HIVE-Build
          Hide
          hiveqa Hive QA added a comment -

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12856039/HIVE-1010.9.patch

          SUCCESS: +1 due to 3 test(s) being added or modified.

          ERROR: -1 due to 15 failed/errored test(s), 10268 tests executed
          Failed tests:

          TestCommandProcessorFactory - did not produce a TEST-*.xml file (likely timed out) (batchId=272)
          TestDbTxnManager - did not produce a TEST-*.xml file (likely timed out) (batchId=272)
          TestDummyTxnManager - did not produce a TEST-*.xml file (likely timed out) (batchId=272)
          TestHiveInputSplitComparator - did not produce a TEST-*.xml file (likely timed out) (batchId=272)
          TestIndexType - did not produce a TEST-*.xml file (likely timed out) (batchId=272)
          TestSplitFilter - did not produce a TEST-*.xml file (likely timed out) (batchId=272)
          org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[escape_comments] (batchId=229)
          org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_text_vec_table] (batchId=147)
          org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb] (batchId=145)
          org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_if_expr] (batchId=140)
          org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14] (batchId=224)
          org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vector_between_in] (batchId=119)
          org.apache.hive.config.JdbcStorageConfigManagerTest.testWithJdbcUrlMissing (batchId=187)
          org.apache.hive.hcatalog.api.TestHCatClient.org.apache.hive.hcatalog.api.TestHCatClient (batchId=170)
          org.apache.hive.service.server.TestHS2HttpServer.testContextRootUrlRewrite (batchId=186)
          

          Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3943/testReport
          Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3943/console
          Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3943/

          Messages:

          Executing org.apache.hive.ptest.execution.TestCheckPhase
          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          Tests exited with: TestsFailedException: 15 tests failed
          

          This message is automatically generated.

          ATTACHMENT ID: 12856039 - PreCommit-HIVE-Build

          Show
          hiveqa Hive QA added a comment - Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12856039/HIVE-1010.9.patch SUCCESS: +1 due to 3 test(s) being added or modified. ERROR: -1 due to 15 failed/errored test(s), 10268 tests executed Failed tests: TestCommandProcessorFactory - did not produce a TEST-*.xml file (likely timed out) (batchId=272) TestDbTxnManager - did not produce a TEST-*.xml file (likely timed out) (batchId=272) TestDummyTxnManager - did not produce a TEST-*.xml file (likely timed out) (batchId=272) TestHiveInputSplitComparator - did not produce a TEST-*.xml file (likely timed out) (batchId=272) TestIndexType - did not produce a TEST-*.xml file (likely timed out) (batchId=272) TestSplitFilter - did not produce a TEST-*.xml file (likely timed out) (batchId=272) org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[escape_comments] (batchId=229) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_text_vec_table] (batchId=147) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb] (batchId=145) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_if_expr] (batchId=140) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14] (batchId=224) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vector_between_in] (batchId=119) org.apache.hive.config.JdbcStorageConfigManagerTest.testWithJdbcUrlMissing (batchId=187) org.apache.hive.hcatalog.api.TestHCatClient.org.apache.hive.hcatalog.api.TestHCatClient (batchId=170) org.apache.hive.service.server.TestHS2HttpServer.testContextRootUrlRewrite (batchId=186) Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3943/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3943/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3943/ Messages: Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 15 tests failed This message is automatically generated. ATTACHMENT ID: 12856039 - PreCommit-HIVE-Build
          Hide
          hiveqa Hive QA added a comment -

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12860896/HIVE-1010.10.patch

          SUCCESS: +1 due to 3 test(s) being added or modified.

          ERROR: -1 due to 6 failed/errored test(s), 10515 tests executed
          Failed tests:

          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[comments] (batchId=35)
          org.apache.hive.config.JdbcStorageConfigManagerTest.testWithJdbcUrlMissing (batchId=189)
          org.apache.hive.hcatalog.api.TestHCatClient.testTransportFailure (batchId=172)
          org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testConnection (batchId=232)
          org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testNegativeTokenAuth (batchId=232)
          org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testProxyAuth (batchId=232)
          

          Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/4416/testReport
          Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/4416/console
          Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-4416/

          Messages:

          Executing org.apache.hive.ptest.execution.TestCheckPhase
          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          Tests exited with: TestsFailedException: 6 tests failed
          

          This message is automatically generated.

          ATTACHMENT ID: 12860896 - PreCommit-HIVE-Build

          Show
          hiveqa Hive QA added a comment - Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12860896/HIVE-1010.10.patch SUCCESS: +1 due to 3 test(s) being added or modified. ERROR: -1 due to 6 failed/errored test(s), 10515 tests executed Failed tests: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[comments] (batchId=35) org.apache.hive.config.JdbcStorageConfigManagerTest.testWithJdbcUrlMissing (batchId=189) org.apache.hive.hcatalog.api.TestHCatClient.testTransportFailure (batchId=172) org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testConnection (batchId=232) org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testNegativeTokenAuth (batchId=232) org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testProxyAuth (batchId=232) Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/4416/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/4416/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-4416/ Messages: Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 6 tests failed This message is automatically generated. ATTACHMENT ID: 12860896 - PreCommit-HIVE-Build
          Hide
          hagleitn Gunther Hagleitner added a comment -

          Checked against the standard. .11 should have the following views compliant with standard: SCHEMATA, TABLES, COLUMNS, TABLE_PRIVILEGES, COLUMN_PRIVILEGES.

          Show
          hagleitn Gunther Hagleitner added a comment - Checked against the standard. .11 should have the following views compliant with standard: SCHEMATA, TABLES, COLUMNS, TABLE_PRIVILEGES, COLUMN_PRIVILEGES.
          Hide
          hiveqa Hive QA added a comment -

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12861003/HIVE-1010.11.patch

          SUCCESS: +1 due to 4 test(s) being added or modified.

          ERROR: -1 due to 3 failed/errored test(s), 10518 tests executed
          Failed tests:

          org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[drop_with_concurrency] (batchId=231)
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[comments] (batchId=35)
          org.apache.hive.hcatalog.api.TestHCatClient.testTransportFailure (batchId=172)
          

          Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/4441/testReport
          Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/4441/console
          Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-4441/

          Messages:

          Executing org.apache.hive.ptest.execution.TestCheckPhase
          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          Tests exited with: TestsFailedException: 3 tests failed
          

          This message is automatically generated.

          ATTACHMENT ID: 12861003 - PreCommit-HIVE-Build

          Show
          hiveqa Hive QA added a comment - Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12861003/HIVE-1010.11.patch SUCCESS: +1 due to 4 test(s) being added or modified. ERROR: -1 due to 3 failed/errored test(s), 10518 tests executed Failed tests: org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[drop_with_concurrency] (batchId=231) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[comments] (batchId=35) org.apache.hive.hcatalog.api.TestHCatClient.testTransportFailure (batchId=172) Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/4441/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/4441/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-4441/ Messages: Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 3 tests failed This message is automatically generated. ATTACHMENT ID: 12861003 - PreCommit-HIVE-Build

            People

            • Assignee:
              hagleitn Gunther Hagleitner
              Reporter:
              hammer Jeff Hammerbacher
            • Votes:
              5 Vote for this issue
              Watchers:
              13 Start watching this issue

              Dates

              • Created:
                Updated:

                Development