Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-1010

Implement INFORMATION_SCHEMA in Hive

        Details

        • Hadoop Flags:
          Incompatible change, Reviewed
        • Release Note:
          Hide
          New method added to StorageHandler to pass credentials around.

          Adds db type to hive schematool. The db type is "hive". When you run the schematool w/ hive it installs the "sys" db and the "information_schema" db. both contain tables and views that expose meta information about tables, columns etc.
          Show
          New method added to StorageHandler to pass credentials around. Adds db type to hive schematool. The db type is "hive". When you run the schematool w/ hive it installs the "sys" db and the "information_schema" db. both contain tables and views that expose meta information about tables, columns etc.

          Description

          INFORMATION_SCHEMA is part of the SQL92 standard and would be useful to implement using our metastore.

          1. Text File
            HIVE-1010.10.patch
            180 kB
            Gunther Hagleitner
          2. Text File
            HIVE-1010.11.patch
            186 kB
            Gunther Hagleitner
          3. Text File
            HIVE-1010.12.patch
            185 kB
            Gunther Hagleitner
          4. Text File
            HIVE-1010.13.patch
            214 kB
            Gunther Hagleitner
          5. Text File
            HIVE-1010.14.patch
            217 kB
            Gunther Hagleitner
          6. Text File
            HIVE-1010.15.patch
            218 kB
            Gunther Hagleitner
          7. Text File
            HIVE-1010.16.patch
            219 kB
            Gunther Hagleitner
          8. Text File
            HIVE-1010.7.patch
            158 kB
            Gunther Hagleitner
          9. Text File
            HIVE-1010.8.patch
            158 kB
            Gunther Hagleitner
          10. Text File
            HIVE-1010.9.patch
            158 kB
            Gunther Hagleitner

            Issue Links

            breaks

            Bug - A problem which impairs or prevents the functions of the product. HIVE-17083 DagUtils overwrites any credentials already added

            • Major - Major loss of function.
            • Resolved
            is blocked by

            New Feature - A new feature of the product, which has yet to be developed. HIVE-675 Add Database/Schema support to Hive QL

            • Major - Major loss of function.
            • Closed

            New Feature - A new feature of the product, which has yet to be developed. HIVE-972 Support views

            • Major - Major loss of function.
            • Closed
            is related to

            Bug - A problem which impairs or prevents the functions of the product. HIVE-16630 Make connection pool pluggable in JdbcStorageHandler

            • Major - Major loss of function.
            • Open

            Bug - A problem which impairs or prevents the functions of the product. HIVE-16657 Replace conf variable with checkDatabaseProduct-like in JDBCStorageHandler

            • Major - Major loss of function.
            • Open
            relates to

            New Feature - A new feature of the product, which has yet to be developed. HIVE-1555 JDBC Storage Handler

            • Major - Major loss of function.
            • Resolved
            links to

            Web Link GitHub Pull Request #177

            Web Link GitHub Pull Request #181

              Activity

              Ascending order - Click to sort in descending order
              Hide
              Permalink
              cwsteinbach Carl Steinbach added a comment -

              MySQL INFORMATION_SCHEMA views: http://dev.mysql.com/doc/refman/5.1/en/information-schema.html

              There's been some discussion offline about supporting this using the JDO query language, i.e. expressing HiveQL queries against the INFORMATION_SCHEMA as JDOQL queries against the metastore.

              Additionally, supporting an INFORMATION_SCHEMA is probably the biggest missing piece that blocks grafting the PostgreSQL ODBC driver onto Hive (as has been done for LucidDB, H2, etc).

              Show
              cwsteinbach Carl Steinbach added a comment - MySQL INFORMATION_SCHEMA views: http://dev.mysql.com/doc/refman/5.1/en/information-schema.html There's been some discussion offline about supporting this using the JDO query language, i.e. expressing HiveQL queries against the INFORMATION_SCHEMA as JDOQL queries against the metastore. Additionally, supporting an INFORMATION_SCHEMA is probably the biggest missing piece that blocks grafting the PostgreSQL ODBC driver onto Hive (as has been done for LucidDB, H2, etc).
              Hide
              Permalink
              jvs John Sichi added a comment -

              Can we get the native SQL translation from JDO and then wrap this to get the data out of the metastore?

              http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/mapred/lib/db/DBInputFormat.html

              Then use HIve views to map metastore representation to INFORMATION_SCHEMA.

              Show
              jvs John Sichi added a comment - Can we get the native SQL translation from JDO and then wrap this to get the data out of the metastore? http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/mapred/lib/db/DBInputFormat.html Then use HIve views to map metastore representation to INFORMATION_SCHEMA.
              Hide
              Permalink
              jvs John Sichi added a comment -

              See HIVE-1555 for a JDBC storage handler.

              Show
              jvs John Sichi added a comment - See HIVE-1555 for a JDBC storage handler.
              Hide
              Permalink
              hagleitn Gunther Hagleitner added a comment -

              WIP. Based on patch in HIVE-1555. SCHEMATA, TABLES, and COLUMNS are there.

              Show
              hagleitn Gunther Hagleitner added a comment - WIP. Based on patch in HIVE-1555 . SCHEMATA, TABLES, and COLUMNS are there.
              Hide
              Permalink
              hiveqa Hive QA added a comment -

              Here are the results of testing the latest attachment:
              https://issues.apache.org/jira/secure/attachment/12853414/HIVE-1010.5.patch

              ERROR: -1 due to build exiting with an error

              Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3648/testReport
              Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3648/console
              Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3648/

              Messages:

              Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2017-02-18 11:12:01.027
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-3648/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2017-02-18 11:12:01.030
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 3f986d7 HIVE-15919: Row count mismatch for count * query (Jason Dere, reviewed by Matt McCline)
+ git clean -f -d
Removing ql/src/test/queries/clientpositive/llap_reader.q
Removing ql/src/test/results/clientpositive/llap/llap_reader.q.out
Removing ql/src/test/results/clientpositive/llap_reader.q.out
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 3f986d7 HIVE-15919: Row count mismatch for count * query (Jason Dere, reviewed by Matt McCline)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2017-02-18 11:12:02.061
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh /data/hiveptest/working/scratch/build.patch
error: jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/JdbcRecordReader.java: No such file or directory
error: jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/JdbcSerDe.java: No such file or directory
error: jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/conf/DatabaseType.java: No such file or directory
error: jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/conf/JdbcStorageConfigManager.java: No such file or directory
error: jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/dao/DatabaseAccessorFactory.java: No such file or directory
error: jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/dao/JdbcRecordIterator.java: No such file or directory
error: ql/src/test/queries/clientpositive/jdbc_handler.q: No such file or directory
error: ql/src/test/results/clientpositive/jdbc_handler.q.out: No such file or directory
error: ql/src/test/results/clientpositive/llap/jdbc_handler.q.out: No such file or directory
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'

              This message is automatically generated.

              ATTACHMENT ID: 12853414 - PreCommit-HIVE-Build

              Show
              hiveqa Hive QA added a comment - Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12853414/HIVE-1010.5.patch ERROR: -1 due to build exiting with an error Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3648/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3648/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3648/ Messages: Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Tests exited with: NonZeroExitCodeException Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit status 1 and output '+ date '+%Y-%m-%d %T.%3N' 2017-02-18 11:12:01.027 + [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]] + export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 + JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 + export PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games + PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games + export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m ' + ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m ' + export 'MAVEN_OPTS=-Xmx1g ' + MAVEN_OPTS='-Xmx1g ' + cd /data/hiveptest/working/ + tee /data/hiveptest/logs/PreCommit-HIVE-Build-3648/source-prep.txt + [[ false == \t\r\u\e ]] + mkdir -p maven ivy + [[ git = \s\v\n ]] + [[ git = \g\i\t ]] + [[ -z master ]] + [[ -d apache-github-source-source ]] + [[ ! -d apache-github-source-source/.git ]] + [[ ! -d apache-github-source-source ]] + date '+%Y-%m-%d %T.%3N' 2017-02-18 11:12:01.030 + cd apache-github-source-source + git fetch origin + git reset --hard HEAD HEAD is now at 3f986d7 HIVE-15919: Row count mismatch for count * query (Jason Dere, reviewed by Matt McCline) + git clean -f -d Removing ql/src/test/queries/clientpositive/llap_reader.q Removing ql/src/test/results/clientpositive/llap/llap_reader.q.out Removing ql/src/test/results/clientpositive/llap_reader.q.out + git checkout master Already on 'master' Your branch is up-to-date with 'origin/master'. + git reset --hard origin/master HEAD is now at 3f986d7 HIVE-15919: Row count mismatch for count * query (Jason Dere, reviewed by Matt McCline) + git merge --ff-only origin/master Already up-to-date. + date '+%Y-%m-%d %T.%3N' 2017-02-18 11:12:02.061 + patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh + patchFilePath=/data/hiveptest/working/scratch/build.patch + [[ -f /data/hiveptest/working/scratch/build.patch ]] + chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh + /data/hiveptest/working/scratch/smart-apply-patch.sh /data/hiveptest/working/scratch/build.patch error: jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/JdbcRecordReader.java: No such file or directory error: jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/JdbcSerDe.java: No such file or directory error: jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/conf/DatabaseType.java: No such file or directory error: jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/conf/JdbcStorageConfigManager.java: No such file or directory error: jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/dao/DatabaseAccessorFactory.java: No such file or directory error: jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/dao/JdbcRecordIterator.java: No such file or directory error: ql/src/test/queries/clientpositive/jdbc_handler.q: No such file or directory error: ql/src/test/results/clientpositive/jdbc_handler.q.out: No such file or directory error: ql/src/test/results/clientpositive/llap/jdbc_handler.q.out: No such file or directory The patch does not appear to apply with p0, p1, or p2 + exit 1 ' This message is automatically generated. ATTACHMENT ID: 12853414 - PreCommit-HIVE-Build
              Hide
              Permalink
              hagleitn Gunther Hagleitner added a comment -

              (patch requires HIVE-1555) cc Thejas M Nair

              Show
              hagleitn Gunther Hagleitner added a comment - (patch requires HIVE-1555 ) cc Thejas M Nair
              Hide
              Permalink
              hiveqa Hive QA added a comment -

              Here are the results of testing the latest attachment:
              https://issues.apache.org/jira/secure/attachment/12855325/HIVE-1010.6.patch

              SUCCESS: +1 due to 3 test(s) being added or modified.

              ERROR: -1 due to 3 failed/errored test(s), 10324 tests executed
              Failed tests:

              org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_text_vec_table] (batchId=147)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_if_expr] (batchId=140)
org.apache.hive.config.JdbcStorageConfigManagerTest.testWithJdbcUrlMissing (batchId=187)

              Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3866/testReport
              Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3866/console
              Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3866/

              Messages:

              Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed

              This message is automatically generated.

              ATTACHMENT ID: 12855325 - PreCommit-HIVE-Build

              Show
              hiveqa Hive QA added a comment - Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12855325/HIVE-1010.6.patch SUCCESS: +1 due to 3 test(s) being added or modified. ERROR: -1 due to 3 failed/errored test(s), 10324 tests executed Failed tests: org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_text_vec_table] (batchId=147) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_if_expr] (batchId=140) org.apache.hive.config.JdbcStorageConfigManagerTest.testWithJdbcUrlMissing (batchId=187) Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3866/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3866/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3866/ Messages: Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 3 tests failed This message is automatically generated. ATTACHMENT ID: 12855325 - PreCommit-HIVE-Build
              Hide
              Permalink
              hagleitn Gunther Hagleitner added a comment -

              Thejas M Nair tests are passing. I still have to make the information schema definitions follow the standard better, but it's a start. I'm looking for feedback on the schema tool etc. The plan create a "hive" metastore folder that contains updates and schema definitions for the metastore db and information schema. Upgrade works by pointing the hive schema tool to a hs2 instance.

              Show
              hagleitn Gunther Hagleitner added a comment - Thejas M Nair tests are passing. I still have to make the information schema definitions follow the standard better, but it's a start. I'm looking for feedback on the schema tool etc. The plan create a "hive" metastore folder that contains updates and schema definitions for the metastore db and information schema. Upgrade works by pointing the hive schema tool to a hs2 instance.
              Hide
              Permalink
              hiveqa Hive QA added a comment -

              Here are the results of testing the latest attachment:
              https://issues.apache.org/jira/secure/attachment/12855579/HIVE-1010.7.patch

              ERROR: -1 due to build exiting with an error

              Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3889/testReport
              Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3889/console
              Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3889/

              Messages:

              Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2017-03-02 09:41:30.907
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-3889/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2017-03-02 09:41:30.910
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 4c7f2d9 HIVE-15882: HS2 generating high memory pressure with many partitions and concurrent queries (Misha Dmitriev, reviewed by Sahil, Mohit, Vihang and Rui)
+ git clean -f -d
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 4c7f2d9 HIVE-15882: HS2 generating high memory pressure with many partitions and concurrent queries (Misha Dmitriev, reviewed by Sahil, Mohit, Vihang and Rui)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2017-03-02 09:41:38.265
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh /data/hiveptest/working/scratch/build.patch
error: patch failed: ql/src/java/org/apache/hadoop/hive/ql/plan/PartitionDesc.java:120
error: ql/src/java/org/apache/hadoop/hive/ql/plan/PartitionDesc.java: patch does not apply
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'

              This message is automatically generated.

              ATTACHMENT ID: 12855579 - PreCommit-HIVE-Build

              Show
              hiveqa Hive QA added a comment - Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12855579/HIVE-1010.7.patch ERROR: -1 due to build exiting with an error Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3889/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3889/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3889/ Messages: Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Tests exited with: NonZeroExitCodeException Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit status 1 and output '+ date '+%Y-%m-%d %T.%3N' 2017-03-02 09:41:30.907 + [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]] + export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 + JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 + export PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games + PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games + export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m ' + ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m ' + export 'MAVEN_OPTS=-Xmx1g ' + MAVEN_OPTS='-Xmx1g ' + cd /data/hiveptest/working/ + tee /data/hiveptest/logs/PreCommit-HIVE-Build-3889/source-prep.txt + [[ false == \t\r\u\e ]] + mkdir -p maven ivy + [[ git = \s\v\n ]] + [[ git = \g\i\t ]] + [[ -z master ]] + [[ -d apache-github-source-source ]] + [[ ! -d apache-github-source-source/.git ]] + [[ ! -d apache-github-source-source ]] + date '+%Y-%m-%d %T.%3N' 2017-03-02 09:41:30.910 + cd apache-github-source-source + git fetch origin + git reset --hard HEAD HEAD is now at 4c7f2d9 HIVE-15882: HS2 generating high memory pressure with many partitions and concurrent queries (Misha Dmitriev, reviewed by Sahil, Mohit, Vihang and Rui) + git clean -f -d + git checkout master Already on 'master' Your branch is up-to-date with 'origin/master'. + git reset --hard origin/master HEAD is now at 4c7f2d9 HIVE-15882: HS2 generating high memory pressure with many partitions and concurrent queries (Misha Dmitriev, reviewed by Sahil, Mohit, Vihang and Rui) + git merge --ff-only origin/master Already up-to-date. + date '+%Y-%m-%d %T.%3N' 2017-03-02 09:41:38.265 + patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh + patchFilePath=/data/hiveptest/working/scratch/build.patch + [[ -f /data/hiveptest/working/scratch/build.patch ]] + chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh + /data/hiveptest/working/scratch/smart-apply-patch.sh /data/hiveptest/working/scratch/build.patch error: patch failed: ql/src/java/org/apache/hadoop/hive/ql/plan/PartitionDesc.java:120 error: ql/src/java/org/apache/hadoop/hive/ql/plan/PartitionDesc.java: patch does not apply The patch does not appear to apply with p0, p1, or p2 + exit 1 ' This message is automatically generated. ATTACHMENT ID: 12855579 - PreCommit-HIVE-Build
              Hide
              Permalink
              hagleitn Gunther Hagleitner added a comment -

              .8 is a rebase

              Show
              hagleitn Gunther Hagleitner added a comment - .8 is a rebase
              Hide
              Permalink
              hiveqa Hive QA added a comment -

              Here are the results of testing the latest attachment:
              https://issues.apache.org/jira/secure/attachment/12855758/HIVE-1010.8.patch

              ERROR: -1 due to build exiting with an error

              Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3917/testReport
              Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3917/console
              Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3917/

              Messages:

              Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Tests exited with: ExecutionException: java.util.concurrent.ExecutionException: org.apache.hive.ptest.execution.ssh.SSHExecutionException: RSyncResult [localFile=/data/hiveptest/logs/PreCommit-HIVE-Build-3917/failed/272_UTBatch_ql_10_tests, remoteFile=/home/hiveptest/35.184.0.150-hiveptest-1/logs/, getExitCode()=11, getException()=null, getUser()=hiveptest, getHost()=35.184.0.150, getInstance()=1]: 'Warning: Permanently added '35.184.0.150' (ECDSA) to the list of known hosts.
receiving incremental file list
./
TEST-272_UTBatch_ql_10_tests-TEST-org.apache.hadoop.hive.ql.TestTxnCommands.xml

              0   0%    0.00kB/s    0:00:00  
          8,230 100%    7.85MB/s    0:00:00 (xfr#1, to-chk=9/11)
TEST-272_UTBatch_ql_10_tests-TEST-org.apache.hadoop.hive.ql.lockmgr.TestEmbeddedLockManager.xml

              0   0%    0.00kB/s    0:00:00  
          5,257 100%    5.01MB/s    0:00:00 (xfr#2, to-chk=8/11)
TEST-272_UTBatch_ql_10_tests-TEST-org.apache.hadoop.hive.ql.lockmgr.TestHiveLockObject.xml

              0   0%    0.00kB/s    0:00:00  
          5,257 100%    5.01MB/s    0:00:00 (xfr#3, to-chk=7/11)
TEST-272_UTBatch_ql_10_tests-TEST-org.apache.hadoop.hive.ql.lockmgr.zookeeper.TestZookeeperLockManager.xml

              0   0%    0.00kB/s    0:00:00  
          5,551 100%    5.29MB/s    0:00:00 (xfr#4, to-chk=6/11)
maven-test.txt

              0   0%    0.00kB/s    0:00:00  
          6,475 100%    6.18MB/s    0:00:00 (xfr#5, to-chk=5/11)
logs/
logs/derby.log

              0   0%    0.00kB/s    0:00:00  
            966 100%  943.36kB/s    0:00:00 (xfr#6, to-chk=2/11)
logs/hive.log

              0   0%    0.00kB/s    0:00:00  
     36,077,568   0%   34.41MB/s    0:14:43  
rsync: write failed on "/data/hiveptest/logs/PreCommit-HIVE-Build-3917/failed/272_UTBatch_ql_10_tests/logs/hive.log": No space left on device (28)
rsync error: error in file IO (code 11) at receiver.c(393) [receiver=3.1.1]
Warning: Permanently added '35.184.0.150' (ECDSA) to the list of known hosts.
receiving incremental file list
logs/
logs/hive.log

              0   0%    0.00kB/s    0:00:00  
rsync: write failed on "/data/hiveptest/logs/PreCommit-HIVE-Build-3917/failed/272_UTBatch_ql_10_tests/logs/hive.log": No space left on device (28)
rsync error: error in file IO (code 11) at receiver.c(393) [receiver=3.1.1]
Warning: Permanently added '35.184.0.150' (ECDSA) to the list of known hosts.
receiving incremental file list
logs/
logs/hive.log

              0   0%    0.00kB/s    0:00:00  
rsync: write failed on "/data/hiveptest/logs/PreCommit-HIVE-Build-3917/failed/272_UTBatch_ql_10_tests/logs/hive.log": No space left on device (28)
rsync error: error in file IO (code 11) at receiver.c(393) [receiver=3.1.1]
Warning: Permanently added '35.184.0.150' (ECDSA) to the list of known hosts.
receiving incremental file list
logs/
logs/hive.log

              0   0%    0.00kB/s    0:00:00  
rsync: write failed on "/data/hiveptest/logs/PreCommit-HIVE-Build-3917/failed/272_UTBatch_ql_10_tests/logs/hive.log": No space left on device (28)
rsync error: error in file IO (code 11) at receiver.c(393) [receiver=3.1.1]
Warning: Permanently added '35.184.0.150' (ECDSA) to the list of known hosts.
receiving incremental file list
logs/
logs/hive.log

              0   0%    0.00kB/s    0:00:00  
rsync: write failed on "/data/hiveptest/logs/PreCommit-HIVE-Build-3917/failed/272_UTBatch_ql_10_tests/logs/hive.log": No space left on device (28)
rsync error: error in file IO (code 11) at receiver.c(393) [receiver=3.1.1]
'

              This message is automatically generated.

              ATTACHMENT ID: 12855758 - PreCommit-HIVE-Build

              Show
              hiveqa Hive QA added a comment - Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12855758/HIVE-1010.8.patch ERROR: -1 due to build exiting with an error Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3917/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3917/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3917/ Messages: Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Tests exited with: ExecutionException: java.util.concurrent.ExecutionException: org.apache.hive.ptest.execution.ssh.SSHExecutionException: RSyncResult [localFile=/data/hiveptest/logs/PreCommit-HIVE-Build-3917/failed/272_UTBatch_ql_10_tests, remoteFile=/home/hiveptest/35.184.0.150-hiveptest-1/logs/, getExitCode()=11, getException()=null, getUser()=hiveptest, getHost()=35.184.0.150, getInstance()=1]: 'Warning: Permanently added '35.184.0.150' (ECDSA) to the list of known hosts. receiving incremental file list ./ TEST-272_UTBatch_ql_10_tests-TEST-org.apache.hadoop.hive.ql.TestTxnCommands.xml 0 0% 0.00kB/s 0:00:00 8,230 100% 7.85MB/s 0:00:00 (xfr#1, to-chk=9/11) TEST-272_UTBatch_ql_10_tests-TEST-org.apache.hadoop.hive.ql.lockmgr.TestEmbeddedLockManager.xml 0 0% 0.00kB/s 0:00:00 5,257 100% 5.01MB/s 0:00:00 (xfr#2, to-chk=8/11) TEST-272_UTBatch_ql_10_tests-TEST-org.apache.hadoop.hive.ql.lockmgr.TestHiveLockObject.xml 0 0% 0.00kB/s 0:00:00 5,257 100% 5.01MB/s 0:00:00 (xfr#3, to-chk=7/11) TEST-272_UTBatch_ql_10_tests-TEST-org.apache.hadoop.hive.ql.lockmgr.zookeeper.TestZookeeperLockManager.xml 0 0% 0.00kB/s 0:00:00 5,551 100% 5.29MB/s 0:00:00 (xfr#4, to-chk=6/11) maven-test.txt 0 0% 0.00kB/s 0:00:00 6,475 100% 6.18MB/s 0:00:00 (xfr#5, to-chk=5/11) logs/ logs/derby.log 0 0% 0.00kB/s 0:00:00 966 100% 943.36kB/s 0:00:00 (xfr#6, to-chk=2/11) logs/hive.log 0 0% 0.00kB/s 0:00:00 36,077,568 0% 34.41MB/s 0:14:43 rsync: write failed on "/data/hiveptest/logs/PreCommit-HIVE-Build-3917/failed/272_UTBatch_ql_10_tests/logs/hive.log": No space left on device (28) rsync error: error in file IO (code 11) at receiver.c(393) [receiver=3.1.1] Warning: Permanently added '35.184.0.150' (ECDSA) to the list of known hosts. receiving incremental file list logs/ logs/hive.log 0 0% 0.00kB/s 0:00:00 rsync: write failed on "/data/hiveptest/logs/PreCommit-HIVE-Build-3917/failed/272_UTBatch_ql_10_tests/logs/hive.log": No space left on device (28) rsync error: error in file IO (code 11) at receiver.c(393) [receiver=3.1.1] Warning: Permanently added '35.184.0.150' (ECDSA) to the list of known hosts. receiving incremental file list logs/ logs/hive.log 0 0% 0.00kB/s 0:00:00 rsync: write failed on "/data/hiveptest/logs/PreCommit-HIVE-Build-3917/failed/272_UTBatch_ql_10_tests/logs/hive.log": No space left on device (28) rsync error: error in file IO (code 11) at receiver.c(393) [receiver=3.1.1] Warning: Permanently added '35.184.0.150' (ECDSA) to the list of known hosts. receiving incremental file list logs/ logs/hive.log 0 0% 0.00kB/s 0:00:00 rsync: write failed on "/data/hiveptest/logs/PreCommit-HIVE-Build-3917/failed/272_UTBatch_ql_10_tests/logs/hive.log": No space left on device (28) rsync error: error in file IO (code 11) at receiver.c(393) [receiver=3.1.1] Warning: Permanently added '35.184.0.150' (ECDSA) to the list of known hosts. receiving incremental file list logs/ logs/hive.log 0 0% 0.00kB/s 0:00:00 rsync: write failed on "/data/hiveptest/logs/PreCommit-HIVE-Build-3917/failed/272_UTBatch_ql_10_tests/logs/hive.log": No space left on device (28) rsync error: error in file IO (code 11) at receiver.c(393) [receiver=3.1.1] ' This message is automatically generated. ATTACHMENT ID: 12855758 - PreCommit-HIVE-Build
              Hide
              Permalink
              hiveqa Hive QA added a comment -

              Here are the results of testing the latest attachment:
              https://issues.apache.org/jira/secure/attachment/12856039/HIVE-1010.9.patch

              SUCCESS: +1 due to 3 test(s) being added or modified.

              ERROR: -1 due to 15 failed/errored test(s), 10268 tests executed
              Failed tests:

              TestCommandProcessorFactory - did not produce a TEST-*.xml file (likely timed out) (batchId=272)
TestDbTxnManager - did not produce a TEST-*.xml file (likely timed out) (batchId=272)
TestDummyTxnManager - did not produce a TEST-*.xml file (likely timed out) (batchId=272)
TestHiveInputSplitComparator - did not produce a TEST-*.xml file (likely timed out) (batchId=272)
TestIndexType - did not produce a TEST-*.xml file (likely timed out) (batchId=272)
TestSplitFilter - did not produce a TEST-*.xml file (likely timed out) (batchId=272)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[escape_comments] (batchId=229)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_text_vec_table] (batchId=147)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb] (batchId=145)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_if_expr] (batchId=140)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14] (batchId=224)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vector_between_in] (batchId=119)
org.apache.hive.config.JdbcStorageConfigManagerTest.testWithJdbcUrlMissing (batchId=187)
org.apache.hive.hcatalog.api.TestHCatClient.org.apache.hive.hcatalog.api.TestHCatClient (batchId=170)
org.apache.hive.service.server.TestHS2HttpServer.testContextRootUrlRewrite (batchId=186)

              Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3943/testReport
              Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3943/console
              Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3943/

              Messages:

              Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 15 tests failed

              This message is automatically generated.

              ATTACHMENT ID: 12856039 - PreCommit-HIVE-Build

              Show
              hiveqa Hive QA added a comment - Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12856039/HIVE-1010.9.patch SUCCESS: +1 due to 3 test(s) being added or modified. ERROR: -1 due to 15 failed/errored test(s), 10268 tests executed Failed tests: TestCommandProcessorFactory - did not produce a TEST-*.xml file (likely timed out) (batchId=272) TestDbTxnManager - did not produce a TEST-*.xml file (likely timed out) (batchId=272) TestDummyTxnManager - did not produce a TEST-*.xml file (likely timed out) (batchId=272) TestHiveInputSplitComparator - did not produce a TEST-*.xml file (likely timed out) (batchId=272) TestIndexType - did not produce a TEST-*.xml file (likely timed out) (batchId=272) TestSplitFilter - did not produce a TEST-*.xml file (likely timed out) (batchId=272) org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[escape_comments] (batchId=229) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_text_vec_table] (batchId=147) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb] (batchId=145) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_if_expr] (batchId=140) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14] (batchId=224) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vector_between_in] (batchId=119) org.apache.hive.config.JdbcStorageConfigManagerTest.testWithJdbcUrlMissing (batchId=187) org.apache.hive.hcatalog.api.TestHCatClient.org.apache.hive.hcatalog.api.TestHCatClient (batchId=170) org.apache.hive.service.server.TestHS2HttpServer.testContextRootUrlRewrite (batchId=186) Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3943/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3943/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3943/ Messages: Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 15 tests failed This message is automatically generated. ATTACHMENT ID: 12856039 - PreCommit-HIVE-Build
              Hide
              Permalink
              hiveqa Hive QA added a comment -

              Here are the results of testing the latest attachment:
              https://issues.apache.org/jira/secure/attachment/12860896/HIVE-1010.10.patch

              SUCCESS: +1 due to 3 test(s) being added or modified.

              ERROR: -1 due to 6 failed/errored test(s), 10515 tests executed
              Failed tests:

              org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[comments] (batchId=35)
org.apache.hive.config.JdbcStorageConfigManagerTest.testWithJdbcUrlMissing (batchId=189)
org.apache.hive.hcatalog.api.TestHCatClient.testTransportFailure (batchId=172)
org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testConnection (batchId=232)
org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testNegativeTokenAuth (batchId=232)
org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testProxyAuth (batchId=232)

              Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/4416/testReport
              Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/4416/console
              Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-4416/

              Messages:

              Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 6 tests failed

              This message is automatically generated.

              ATTACHMENT ID: 12860896 - PreCommit-HIVE-Build

              Show
              hiveqa Hive QA added a comment - Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12860896/HIVE-1010.10.patch SUCCESS: +1 due to 3 test(s) being added or modified. ERROR: -1 due to 6 failed/errored test(s), 10515 tests executed Failed tests: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[comments] (batchId=35) org.apache.hive.config.JdbcStorageConfigManagerTest.testWithJdbcUrlMissing (batchId=189) org.apache.hive.hcatalog.api.TestHCatClient.testTransportFailure (batchId=172) org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testConnection (batchId=232) org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testNegativeTokenAuth (batchId=232) org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testProxyAuth (batchId=232) Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/4416/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/4416/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-4416/ Messages: Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 6 tests failed This message is automatically generated. ATTACHMENT ID: 12860896 - PreCommit-HIVE-Build
              Hide
              Permalink
              hagleitn Gunther Hagleitner added a comment -

              Checked against the standard. .11 should have the following views compliant with standard: SCHEMATA, TABLES, COLUMNS, TABLE_PRIVILEGES, COLUMN_PRIVILEGES.

              Show
              hagleitn Gunther Hagleitner added a comment - Checked against the standard. .11 should have the following views compliant with standard: SCHEMATA, TABLES, COLUMNS, TABLE_PRIVILEGES, COLUMN_PRIVILEGES.
              Hide
              Permalink
              hiveqa Hive QA added a comment -

              Here are the results of testing the latest attachment:
              https://issues.apache.org/jira/secure/attachment/12861003/HIVE-1010.11.patch

              SUCCESS: +1 due to 4 test(s) being added or modified.

              ERROR: -1 due to 3 failed/errored test(s), 10518 tests executed
              Failed tests:

              org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[drop_with_concurrency] (batchId=231)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[comments] (batchId=35)
org.apache.hive.hcatalog.api.TestHCatClient.testTransportFailure (batchId=172)

              Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/4441/testReport
              Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/4441/console
              Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-4441/

              Messages:

              Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed

              This message is automatically generated.

              ATTACHMENT ID: 12861003 - PreCommit-HIVE-Build

              Show
              hiveqa Hive QA added a comment - Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12861003/HIVE-1010.11.patch SUCCESS: +1 due to 4 test(s) being added or modified. ERROR: -1 due to 3 failed/errored test(s), 10518 tests executed Failed tests: org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[drop_with_concurrency] (batchId=231) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[comments] (batchId=35) org.apache.hive.hcatalog.api.TestHCatClient.testTransportFailure (batchId=172) Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/4441/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/4441/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-4441/ Messages: Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 3 tests failed This message is automatically generated. ATTACHMENT ID: 12861003 - PreCommit-HIVE-Build
              Hide
              Permalink
              hagleitn Gunther Hagleitner added a comment -

              .12 is rebased.

              Show
              hagleitn Gunther Hagleitner added a comment - .12 is rebased.
              Hide
              Permalink
              thejas Thejas M Nair added a comment -

              Some questions/comments -

              • Does the create table commands work against other databases ?
              • If you need to customize queries against other db, how would you detect the underlying db type
              • Version table under sys should not be a mirror of underlying rdbms versions table. Since upgrade of 'sys' db would be a separate invocation, it won't work with rdbms schema upgrade. You can create a separate table db_version or so that mirrors underlying version table.
              • upgrade scripts - you can keep that simple by just doing drop and re-create via new 'init' schema file.
              • secure db password being sent to tasks via job.credentials apis
              • Looks like this would go into hive 3.0.0, so schema file name would need to be updated.
              Show
              thejas Thejas M Nair added a comment - Some questions/comments - Does the create table commands work against other databases ? If you need to customize queries against other db, how would you detect the underlying db type Version table under sys should not be a mirror of underlying rdbms versions table. Since upgrade of 'sys' db would be a separate invocation, it won't work with rdbms schema upgrade. You can create a separate table db_version or so that mirrors underlying version table. upgrade scripts - you can keep that simple by just doing drop and re-create via new 'init' schema file. secure db password being sent to tasks via job.credentials apis Looks like this would go into hive 3.0.0, so schema file name would need to be updated.
              Hide
              Permalink
              hiveqa Hive QA added a comment -

              Here are the results of testing the latest attachment:
              https://issues.apache.org/jira/secure/attachment/12865843/HIVE-1010.12.patch

              ERROR: -1 due to build exiting with an error

              Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/4976/testReport
              Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/4976/console
              Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-4976/

              Messages:

              Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2017-05-02 02:05:58.395
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-4976/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2017-05-02 02:05:58.397
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 5ab03cb HIVE-16524: Remove the redundant item type in hiveserver2.jsp and QueryProfileTmpl.jamon (ZhangBing via Xuefu)
+ git clean -f -d
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 5ab03cb HIVE-16524: Remove the redundant item type in hiveserver2.jsp and QueryProfileTmpl.jamon (ZhangBing via Xuefu)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2017-05-02 02:05:58.872
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh /data/hiveptest/working/scratch/build.patch
Going to apply patch with: patch -p0
patching file beeline/src/java/org/apache/hive/beeline/HiveSchemaHelper.java
patching file beeline/src/java/org/apache/hive/beeline/HiveSchemaTool.java
patching file beeline/src/test/org/apache/hive/beeline/TestHiveSchemaTool.java
patching file itests/src/test/resources/testconfiguration.properties
patching file jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/JdbcInputFormat.java
patching file jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/JdbcRecordReader.java
patching file jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/JdbcSerDe.java
patching file jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/JdbcStorageHandler.java
patching file jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/conf/DatabaseType.java
patching file jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/conf/JdbcStorageConfigManager.java
patching file jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/dao/DatabaseAccessorFactory.java
patching file jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/dao/GenericJdbcDatabaseAccessor.java
patching file jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/dao/JdbcRecordIterator.java
patching file jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/dao/MySqlDatabaseAccessor.java
patching file jdbc-handler/src/test/java/org/apache/hive/config/JdbcStorageConfigManagerTest.java
patching file llap-server/src/java/org/apache/hadoop/hive/llap/cli/LlapServiceDriver.java
patching file metastore/scripts/upgrade/hive/hive-schema-2.2.0-hive.sql
patching file ql/src/java/org/apache/hadoop/hive/ql/parse/UnparseTranslator.java
patching file ql/src/java/org/apache/hadoop/hive/ql/plan/PartitionDesc.java
patching file ql/src/test/queries/clientpositive/jdbc_handler.q
patching file ql/src/test/queries/clientpositive/sysdb.q
patching file ql/src/test/results/clientpositive/llap/jdbc_handler.q.out
patching file ql/src/test/results/clientpositive/llap/sysdb.q.out
+ [[ maven == \m\a\v\e\n ]]
+ rm -rf /data/hiveptest/working/maven/org/apache/hive
+ mvn -B clean install -DskipTests -T 4 -q -Dmaven.repo.local=/data/hiveptest/working/maven
[ERROR] COMPILATION ERROR : 
[ERROR] /data/hiveptest/working/apache-github-source-source/common/src/java/org/apache/hadoop/hive/common/jsonexplain/tez/TezJsonParserUtils.java:[27,8] class DagJsonParserUtils is public, should be declared in a file named DagJsonParserUtils.java
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.6.1:compile (default-compile) on project hive-common: Compilation failure
[ERROR] /data/hiveptest/working/apache-github-source-source/common/src/java/org/apache/hadoop/hive/common/jsonexplain/tez/TezJsonParserUtils.java:[27,8] class DagJsonParserUtils is public, should be declared in a file named DagJsonParserUtils.java
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :hive-common
+ exit 1
'

              This message is automatically generated.

              ATTACHMENT ID: 12865843 - PreCommit-HIVE-Build

              Show
              hiveqa Hive QA added a comment - Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12865843/HIVE-1010.12.patch ERROR: -1 due to build exiting with an error Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/4976/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/4976/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-4976/ Messages: Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Tests exited with: NonZeroExitCodeException Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit status 1 and output '+ date '+%Y-%m-%d %T.%3N' 2017-05-02 02:05:58.395 + [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]] + export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 + JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 + export PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games + PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games + export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m ' + ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m ' + export 'MAVEN_OPTS=-Xmx1g ' + MAVEN_OPTS='-Xmx1g ' + cd /data/hiveptest/working/ + tee /data/hiveptest/logs/PreCommit-HIVE-Build-4976/source-prep.txt + [[ false == \t\r\u\e ]] + mkdir -p maven ivy + [[ git = \s\v\n ]] + [[ git = \g\i\t ]] + [[ -z master ]] + [[ -d apache-github-source-source ]] + [[ ! -d apache-github-source-source/.git ]] + [[ ! -d apache-github-source-source ]] + date '+%Y-%m-%d %T.%3N' 2017-05-02 02:05:58.397 + cd apache-github-source-source + git fetch origin + git reset --hard HEAD HEAD is now at 5ab03cb HIVE-16524: Remove the redundant item type in hiveserver2.jsp and QueryProfileTmpl.jamon (ZhangBing via Xuefu) + git clean -f -d + git checkout master Already on 'master' Your branch is up-to-date with 'origin/master'. + git reset --hard origin/master HEAD is now at 5ab03cb HIVE-16524: Remove the redundant item type in hiveserver2.jsp and QueryProfileTmpl.jamon (ZhangBing via Xuefu) + git merge --ff-only origin/master Already up-to-date. + date '+%Y-%m-%d %T.%3N' 2017-05-02 02:05:58.872 + patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh + patchFilePath=/data/hiveptest/working/scratch/build.patch + [[ -f /data/hiveptest/working/scratch/build.patch ]] + chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh + /data/hiveptest/working/scratch/smart-apply-patch.sh /data/hiveptest/working/scratch/build.patch Going to apply patch with: patch -p0 patching file beeline/src/java/org/apache/hive/beeline/HiveSchemaHelper.java patching file beeline/src/java/org/apache/hive/beeline/HiveSchemaTool.java patching file beeline/src/test/org/apache/hive/beeline/TestHiveSchemaTool.java patching file itests/src/test/resources/testconfiguration.properties patching file jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/JdbcInputFormat.java patching file jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/JdbcRecordReader.java patching file jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/JdbcSerDe.java patching file jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/JdbcStorageHandler.java patching file jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/conf/DatabaseType.java patching file jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/conf/JdbcStorageConfigManager.java patching file jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/dao/DatabaseAccessorFactory.java patching file jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/dao/GenericJdbcDatabaseAccessor.java patching file jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/dao/JdbcRecordIterator.java patching file jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/dao/MySqlDatabaseAccessor.java patching file jdbc-handler/src/test/java/org/apache/hive/config/JdbcStorageConfigManagerTest.java patching file llap-server/src/java/org/apache/hadoop/hive/llap/cli/LlapServiceDriver.java patching file metastore/scripts/upgrade/hive/hive-schema-2.2.0-hive.sql patching file ql/src/java/org/apache/hadoop/hive/ql/parse/UnparseTranslator.java patching file ql/src/java/org/apache/hadoop/hive/ql/plan/PartitionDesc.java patching file ql/src/test/queries/clientpositive/jdbc_handler.q patching file ql/src/test/queries/clientpositive/sysdb.q patching file ql/src/test/results/clientpositive/llap/jdbc_handler.q.out patching file ql/src/test/results/clientpositive/llap/sysdb.q.out + [[ maven == \m\a\v\e\n ]] + rm -rf /data/hiveptest/working/maven/org/apache/hive + mvn -B clean install -DskipTests -T 4 -q -Dmaven.repo.local=/data/hiveptest/working/maven [ERROR] COMPILATION ERROR : [ERROR] /data/hiveptest/working/apache-github-source-source/common/src/java/org/apache/hadoop/hive/common/jsonexplain/tez/TezJsonParserUtils.java:[27,8] class DagJsonParserUtils is public, should be declared in a file named DagJsonParserUtils.java [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.6.1:compile (default-compile) on project hive-common: Compilation failure [ERROR] /data/hiveptest/working/apache-github-source-source/common/src/java/org/apache/hadoop/hive/common/jsonexplain/tez/TezJsonParserUtils.java:[27,8] class DagJsonParserUtils is public, should be declared in a file named DagJsonParserUtils.java [ERROR] -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn <goals> -rf :hive-common + exit 1 ' This message is automatically generated. ATTACHMENT ID: 12865843 - PreCommit-HIVE-Build
              Hide
              Permalink
              hagleitn Gunther Hagleitner added a comment -

              Thanks Thejas M Nair. I believe .13 addresses all your feedback. Could you take another look?

              Show
              hagleitn Gunther Hagleitner added a comment - Thanks Thejas M Nair . I believe .13 addresses all your feedback. Could you take another look?
              Hide
              Permalink
              githubbot ASF GitHub Bot added a comment -

              GitHub user hagleitn opened a pull request:

              https://github.com/apache/hive/pull/177

              HIVE-1010: Implement INFORMATION_SCHEMA in Hive (Gunther Hagleitner)

              You can merge this pull request into a Git repository by running:

              $ git pull https://github.com/hagleitn/hive master

              Alternatively you can review and apply these changes as the patch at:

              https://github.com/apache/hive/pull/177.patch

              To close this pull request, make a commit to your master/trunk branch
              with (at least) the following in the commit message:

              This closes #177

              commit a9d3f31b55587717a701fea37c30d185e0c132c8
              Author: Gunther Hagleitner <gunther@apache.org>
              Date: 2017-05-04T06:06:09Z

              HIVE-1010: Implement INFORMATION_SCHEMA in Hive (Gunther Hagleitner)

              Show
              githubbot ASF GitHub Bot added a comment - GitHub user hagleitn opened a pull request: https://github.com/apache/hive/pull/177 HIVE-1010 : Implement INFORMATION_SCHEMA in Hive (Gunther Hagleitner) You can merge this pull request into a Git repository by running: $ git pull https://github.com/hagleitn/hive master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/hive/pull/177.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #177 commit a9d3f31b55587717a701fea37c30d185e0c132c8 Author: Gunther Hagleitner <gunther@apache.org> Date: 2017-05-04T06:06:09Z HIVE-1010 : Implement INFORMATION_SCHEMA in Hive (Gunther Hagleitner)
              Hide
              Permalink
              hiveqa Hive QA added a comment -

              Here are the results of testing the latest attachment:
              https://issues.apache.org/jira/secure/attachment/12866335/HIVE-1010.13.patch

              SUCCESS: +1 due to 6 test(s) being added or modified.

              ERROR: -1 due to 2 failed/errored test(s), 10649 tests executed
              Failed tests:

              org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_if_expr] (batchId=143)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_join30] (batchId=148)

              Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/5039/testReport
              Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/5039/console
              Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-5039/

              Messages:

              Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed

              This message is automatically generated.

              ATTACHMENT ID: 12866335 - PreCommit-HIVE-Build

              Show
              hiveqa Hive QA added a comment - Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12866335/HIVE-1010.13.patch SUCCESS: +1 due to 6 test(s) being added or modified. ERROR: -1 due to 2 failed/errored test(s), 10649 tests executed Failed tests: org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_if_expr] (batchId=143) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_join30] (batchId=148) Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/5039/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/5039/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-5039/ Messages: Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 2 tests failed This message is automatically generated. ATTACHMENT ID: 12866335 - PreCommit-HIVE-Build
              Hide
              Permalink
              hagleitn Gunther Hagleitner added a comment -

              The two failing UT are unrelated.

              Show
              hagleitn Gunther Hagleitner added a comment - The two failing UT are unrelated.
              Hide
              Permalink
              githubbot ASF GitHub Bot added a comment -

              Github user hagleitn closed the pull request at:

              https://github.com/apache/hive/pull/177

              Show
              githubbot ASF GitHub Bot added a comment - Github user hagleitn closed the pull request at: https://github.com/apache/hive/pull/177
              Hide
              Permalink
              githubbot ASF GitHub Bot added a comment -

              GitHub user hagleitn opened a pull request:

              https://github.com/apache/hive/pull/181

              HIVE-1010: Implement INFORMATION_SCHEMA in Hive (Gunther Hagleitner)

              You can merge this pull request into a Git repository by running:

              $ git pull https://github.com/hagleitn/hive master

              Alternatively you can review and apply these changes as the patch at:

              https://github.com/apache/hive/pull/181.patch

              To close this pull request, make a commit to your master/trunk branch
              with (at least) the following in the commit message:

              This closes #181

              commit 3269ca48e753a09fd6c15df826e49aaf952d814d
              Author: Gunther Hagleitner <gunther@apache.org>
              Date: 2017-05-04T06:06:09Z

              HIVE-1010: Implement INFORMATION_SCHEMA in Hive (Gunther Hagleitner)

              Show
              githubbot ASF GitHub Bot added a comment - GitHub user hagleitn opened a pull request: https://github.com/apache/hive/pull/181 HIVE-1010 : Implement INFORMATION_SCHEMA in Hive (Gunther Hagleitner) You can merge this pull request into a Git repository by running: $ git pull https://github.com/hagleitn/hive master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/hive/pull/181.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #181 commit 3269ca48e753a09fd6c15df826e49aaf952d814d Author: Gunther Hagleitner <gunther@apache.org> Date: 2017-05-04T06:06:09Z HIVE-1010 : Implement INFORMATION_SCHEMA in Hive (Gunther Hagleitner)
              Hide
              Permalink
              hiveqa Hive QA added a comment -

              Here are the results of testing the latest attachment:
              https://issues.apache.org/jira/secure/attachment/12866561/HIVE-1010.14.patch

              SUCCESS: +1 due to 6 test(s) being added or modified.

              ERROR: -1 due to 1 failed/errored test(s), 10652 tests executed
              Failed tests:

              org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_if_expr] (batchId=143)

              Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/5065/testReport
              Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/5065/console
              Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-5065/

              Messages:

              Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed

              This message is automatically generated.

              ATTACHMENT ID: 12866561 - PreCommit-HIVE-Build

              Show
              hiveqa Hive QA added a comment - Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12866561/HIVE-1010.14.patch SUCCESS: +1 due to 6 test(s) being added or modified. ERROR: -1 due to 1 failed/errored test(s), 10652 tests executed Failed tests: org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_if_expr] (batchId=143) Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/5065/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/5065/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-5065/ Messages: Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 1 tests failed This message is automatically generated. ATTACHMENT ID: 12866561 - PreCommit-HIVE-Build
              Hide
              Permalink
              prasanth_j Prasanth Jayachandran added a comment -

              left some minor comments in PR. mostly looks good. Except for the password transfer part which Thejas M Nair might be a better person for review.

              Show
              prasanth_j Prasanth Jayachandran added a comment - left some minor comments in PR. mostly looks good. Except for the password transfer part which Thejas M Nair might be a better person for review.
              Hide
              Permalink
              hagleitn Gunther Hagleitner added a comment -

              Thanks Prasanth Jayachandran. Replied on PR. Opened follow up Jira for pluggable pool.

              Show
              hagleitn Gunther Hagleitner added a comment - Thanks Prasanth Jayachandran . Replied on PR. Opened follow up Jira for pluggable pool.
              Hide
              Permalink
              prasanth_j Prasanth Jayachandran added a comment -

              +1. Pending Thejas's review on the credentials.

              Show
              prasanth_j Prasanth Jayachandran added a comment - +1. Pending Thejas's review on the credentials.
              Hide
              Permalink
              thejas Thejas M Nair added a comment -

              Added comments in PR

              Show
              thejas Thejas M Nair added a comment - Added comments in PR
              Hide
              Permalink
              hagleitn Gunther Hagleitner added a comment -

              Thejas M Nair I believe I have addressed all comments. Can you take another look please?

              Show
              hagleitn Gunther Hagleitner added a comment - Thejas M Nair I believe I have addressed all comments. Can you take another look please?
              Hide
              Permalink
              hiveqa Hive QA added a comment -

              Here are the results of testing the latest attachment:
              https://issues.apache.org/jira/secure/attachment/12867674/HIVE-1010.15.patch

              SUCCESS: +1 due to 6 test(s) being added or modified.

              ERROR: -1 due to 11 failed/errored test(s), 10687 tests executed
              Failed tests:

              org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_if_expr] (batchId=144)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_3] (batchId=97)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_3] (batchId=97)
org.apache.hive.config.JdbcStorageConfigManagerTest.testWithDatabaseTypeMissing (batchId=194)
org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testConnection (batchId=237)
org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testIsValid (batchId=237)
org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testIsValidNeg (batchId=237)
org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testNegativeProxyAuth (batchId=237)
org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testNegativeTokenAuth (batchId=237)
org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testProxyAuth (batchId=237)
org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testTokenAuth (batchId=237)

              Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/5218/testReport
              Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/5218/console
              Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-5218/

              Messages:

              Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 11 tests failed

              This message is automatically generated.

              ATTACHMENT ID: 12867674 - PreCommit-HIVE-Build

              Show
              hiveqa Hive QA added a comment - Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12867674/HIVE-1010.15.patch SUCCESS: +1 due to 6 test(s) being added or modified. ERROR: -1 due to 11 failed/errored test(s), 10687 tests executed Failed tests: org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_if_expr] (batchId=144) org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_3] (batchId=97) org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_3] (batchId=97) org.apache.hive.config.JdbcStorageConfigManagerTest.testWithDatabaseTypeMissing (batchId=194) org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testConnection (batchId=237) org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testIsValid (batchId=237) org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testIsValidNeg (batchId=237) org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testNegativeProxyAuth (batchId=237) org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testNegativeTokenAuth (batchId=237) org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testProxyAuth (batchId=237) org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testTokenAuth (batchId=237) Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/5218/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/5218/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-5218/ Messages: Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 11 tests failed This message is automatically generated. ATTACHMENT ID: 12867674 - PreCommit-HIVE-Build
              Hide
              Permalink
              thejas Thejas M Nair added a comment -

              Looks like JdbcStorageConfigManagerTest failure might be related.
              Also added a minor comment to pull request.

              Show
              thejas Thejas M Nair added a comment - Looks like JdbcStorageConfigManagerTest failure might be related. Also added a minor comment to pull request.
              Hide
              Permalink
              thejas Thejas M Nair added a comment -

              One thing to note is that the priv information made available through this patch is the sql standard authorization privileges. Sentry/Ranger stores it in separate dbs. We need some 'RangerStorageHandler' or 'JDBCStorageHandler' that accesses Ranger rdbms for that info.

              Show
              thejas Thejas M Nair added a comment - One thing to note is that the priv information made available through this patch is the sql standard authorization privileges. Sentry/Ranger stores it in separate dbs. We need some 'RangerStorageHandler' or 'JDBCStorageHandler' that accesses Ranger rdbms for that info.
              Hide
              Permalink
              hagleitn Gunther Hagleitner added a comment -

              .16 fixes test case and adds the comment to conf variable Thejas M Nair was asking for.

              Show
              hagleitn Gunther Hagleitner added a comment - .16 fixes test case and adds the comment to conf variable Thejas M Nair was asking for.
              Hide
              Permalink
              hiveqa Hive QA added a comment -

              Here are the results of testing the latest attachment:
              https://issues.apache.org/jira/secure/attachment/12867908/HIVE-1010.16.patch

              SUCCESS: +1 due to 6 test(s) being added or modified.

              ERROR: -1 due to 1 failed/errored test(s), 10698 tests executed
              Failed tests:

              org.apache.hadoop.hive.ql.io.orc.TestNewInputOutputFormat.testNewOutputFormatWithCompression (batchId=258)

              Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/5249/testReport
              Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/5249/console
              Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-5249/

              Messages:

              Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed

              This message is automatically generated.

              ATTACHMENT ID: 12867908 - PreCommit-HIVE-Build

              Show
              hiveqa Hive QA added a comment - Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12867908/HIVE-1010.16.patch SUCCESS: +1 due to 6 test(s) being added or modified. ERROR: -1 due to 1 failed/errored test(s), 10698 tests executed Failed tests: org.apache.hadoop.hive.ql.io.orc.TestNewInputOutputFormat.testNewOutputFormatWithCompression (batchId=258) Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/5249/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/5249/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-5249/ Messages: Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 1 tests failed This message is automatically generated. ATTACHMENT ID: 12867908 - PreCommit-HIVE-Build
              Hide
              Permalink
              hagleitn Gunther Hagleitner added a comment -

              Testfailure is unrelated (passed locally). Thejas M Nair?

              Show
              hagleitn Gunther Hagleitner added a comment - Testfailure is unrelated (passed locally). Thejas M Nair ?
              Hide
              Permalink
              thejas Thejas M Nair added a comment -

              +1

              Show
              thejas Thejas M Nair added a comment - +1
              Hide
              Permalink
              hagleitn Gunther Hagleitner added a comment -

              Committed to master. Thank you Thejas M Nair and Prasanth Jayachandran.

              Show
              hagleitn Gunther Hagleitner added a comment - Committed to master. Thank you Thejas M Nair and Prasanth Jayachandran .
              Hide
              Permalink
              leftylev Lefty Leverenz added a comment - - edited

              Doc note: This adds configuration property hive.metastore.db.type to HiveConf.java and three options to HiveSchemaTool.java (metaDbType, url, and driver), so they need to be documented in the wiki, with version information.

              Added a TODOC3.0 label.

              BTW, three other options also need to be documented for the schema tool: "dbOpts" (HIVE-8374), "validate" (HIVE-15207), and "servers" (HIVE-15805 and HIVE-16647). Might as well take care of them all at once.

              Show
              leftylev Lefty Leverenz added a comment - - edited Doc note: This adds configuration property hive.metastore.db.type to HiveConf.java and three options to HiveSchemaTool.java (metaDbType, url, and driver), so they need to be documented in the wiki, with version information. Configuration Properties – MetaStore Hive Schema Tool – The schematool Command Added a TODOC3.0 label. BTW, three other options also need to be documented for the schema tool: "dbOpts" ( HIVE-8374 ), "validate" ( HIVE-15207 ), and "servers" ( HIVE-15805 and HIVE-16647 ). Might as well take care of them all at once.

                People

                • Assignee:
                  hagleitn Gunther Hagleitner
                  Reporter:
                  hammer Jeff Hammerbacher
                • Votes:
                  5 Vote for this issue
                  Watchers:
                  19 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved:

                    Development