Details
-
Bug
-
Status: Closed
-
Major
-
Resolution: Fixed
-
None
-
None
Description
The TopN key optimizer currently uses a priority queue for keeping track of the largest/smallest rows. Its max size is the same as the user specified limit. This should be replaced a more cache line friendly array with a small (128) maximum size and see how much performance is gained.
Attachments
Attachments
- HIVE-22726.1.patch
- 8 kB
- Attila Magyar
- HIVE-22726.2.patch
- 10 kB
- Attila Magyar
- HIVE-22726.3.patch
- 10 kB
- Attila Magyar
- HIVE-22726.4.patch
- 11 kB
- Attila Magyar
- HIVE-22726.5.patch
- 14 kB
- Attila Magyar
Issue Links
- Dependent
-
HIVE-22666 Introduce TopNKey operator for PTF Reduce Sink
- Closed
- relates to
-
HIVE-17896 TopNKey: Create a standalone vectorizable TopNKey operator
- Closed
- links to
Activity
+1 overall |
Vote | Subsystem | Runtime | Comment |
---|---|---|---|
Prechecks | |||
+1 | @author | 0m 0s | The patch does not contain any @author tags. |
master Compile Tests | |||
0 | mvndep | 2m 10s | Maven dependency ordering for branch |
+1 | mvninstall | 8m 12s | master passed |
+1 | compile | 1m 31s | master passed |
+1 | checkstyle | 0m 58s | master passed |
0 | findbugs | 0m 35s | common in master has 63 extant Findbugs warnings. |
0 | findbugs | 4m 23s | ql in master has 1531 extant Findbugs warnings. |
+1 | javadoc | 1m 21s | master passed |
Patch Compile Tests | |||
0 | mvndep | 0m 29s | Maven dependency ordering for patch |
+1 | mvninstall | 1m 52s | the patch passed |
+1 | compile | 1m 31s | the patch passed |
+1 | javac | 1m 31s | the patch passed |
+1 | checkstyle | 1m 2s | the patch passed |
+1 | whitespace | 0m 0s | The patch has no whitespace issues. |
+1 | findbugs | 5m 14s | the patch passed |
+1 | javadoc | 1m 17s | the patch passed |
Other Tests | |||
+1 | asflicense | 0m 15s | The patch does not generate ASF License warnings. |
31m 32s |
Subsystem | Report/Notes |
---|---|
Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
Build tool | maven |
Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-20185/dev-support/hive-personality.sh |
git revision | master / 72d56f4 |
Default Java | 1.8.0_111 |
findbugs | v3.0.1 |
modules | C: common ql U: . |
Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-20185/yetus.txt |
Powered by | Apache Yetus http://yetus.apache.org |
This message was automatically generated.
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12990883/HIVE-22726.1.patch
ERROR: -1 due to no test(s) being added or modified.
SUCCESS: +1 due to 17876 tests passed
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/20185/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20185/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20185/
Messages:
Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase
This message is automatically generated.
ATTACHMENT ID: 12990883 - PreCommit-HIVE-Build
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12991518/HIVE-22726.2.patch
ERROR: -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/20279/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20279/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20279/
Messages:
Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Tests exited with: NonZeroExitCodeException Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit status 1 and output '+ date '+%Y-%m-%d %T.%3N' 2020-01-22 21:27:37.143 + [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]] + export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 + JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 + export PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games + PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games + export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m ' + ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m ' + export 'MAVEN_OPTS=-Xmx1g ' + MAVEN_OPTS='-Xmx1g ' + cd /data/hiveptest/working/ + tee /data/hiveptest/logs/PreCommit-HIVE-Build-20279/source-prep.txt + [[ false == \t\r\u\e ]] + mkdir -p maven ivy + [[ git = \s\v\n ]] + [[ git = \g\i\t ]] + [[ -z master ]] + [[ -d apache-github-source-source ]] + [[ ! -d apache-github-source-source/.git ]] + [[ ! -d apache-github-source-source ]] + date '+%Y-%m-%d %T.%3N' 2020-01-22 21:27:37.146 + cd apache-github-source-source + git fetch origin + git reset --hard HEAD HEAD is now at 05cabc8 HIVE-22666: Introduce TopNKey operator for PTF Reduce Sink (Krisztian Kasa, reviewed by Jesus Camacho Rodriguez) + git clean -f -d Removing standalone-metastore/metastore-server/src/gen/ + git checkout master Already on 'master' Your branch is up-to-date with 'origin/master'. + git reset --hard origin/master HEAD is now at 05cabc8 HIVE-22666: Introduce TopNKey operator for PTF Reduce Sink (Krisztian Kasa, reviewed by Jesus Camacho Rodriguez) + git merge --ff-only origin/master Already up-to-date. + date '+%Y-%m-%d %T.%3N' 2020-01-22 21:27:37.871 + rm -rf ../yetus_PreCommit-HIVE-Build-20279 + mkdir ../yetus_PreCommit-HIVE-Build-20279 + git gc + cp -R . ../yetus_PreCommit-HIVE-Build-20279 + mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-20279/yetus + patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh + patchFilePath=/data/hiveptest/working/scratch/build.patch + [[ -f /data/hiveptest/working/scratch/build.patch ]] + chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh + /data/hiveptest/working/scratch/smart-apply-patch.sh /data/hiveptest/working/scratch/build.patch Trying to apply the patch with -p0 error: a/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java: does not exist in index error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/TopNKeyFilter.java: does not exist in index error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/TopNKeyOperator.java: does not exist in index error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorTopNKeyOperator.java: does not exist in index error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/vector/wrapper/VectorHashKeyWrapperBatch.java: does not exist in index error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/vector/wrapper/VectorHashKeyWrapperGeneralComparator.java: does not exist in index error: a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/topnkey/TopNKeyProcessor.java: does not exist in index error: a/ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java: does not exist in index Trying to apply the patch with -p1 error: patch failed: ql/src/java/org/apache/hadoop/hive/ql/exec/TopNKeyOperator.java:38 Falling back to three-way merge... Applied patch to 'ql/src/java/org/apache/hadoop/hive/ql/exec/TopNKeyOperator.java' with conflicts. error: patch failed: ql/src/java/org/apache/hadoop/hive/ql/optimizer/topnkey/TopNKeyProcessor.java:58 Falling back to three-way merge... Applied patch to 'ql/src/java/org/apache/hadoop/hive/ql/optimizer/topnkey/TopNKeyProcessor.java' with conflicts. Going to apply patch with: git apply -p1 error: patch failed: ql/src/java/org/apache/hadoop/hive/ql/exec/TopNKeyOperator.java:38 Falling back to three-way merge... Applied patch to 'ql/src/java/org/apache/hadoop/hive/ql/exec/TopNKeyOperator.java' with conflicts. error: patch failed: ql/src/java/org/apache/hadoop/hive/ql/optimizer/topnkey/TopNKeyProcessor.java:58 Falling back to three-way merge... Applied patch to 'ql/src/java/org/apache/hadoop/hive/ql/optimizer/topnkey/TopNKeyProcessor.java' with conflicts. U ql/src/java/org/apache/hadoop/hive/ql/exec/TopNKeyOperator.java U ql/src/java/org/apache/hadoop/hive/ql/optimizer/topnkey/TopNKeyProcessor.java + result=1 + '[' 1 -ne 0 ']' + rm -rf yetus_PreCommit-HIVE-Build-20279 + exit 1 '
This message is automatically generated.
ATTACHMENT ID: 12991518 - PreCommit-HIVE-Build
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12991579/HIVE-22726.3.patch
ERROR: -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/20292/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20292/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20292/
Messages:
Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Tests exited with: NonZeroExitCodeException Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit status 1 and output '+ date '+%Y-%m-%d %T.%3N' 2020-01-23 08:18:56.259 + [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]] + export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 + JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 + export PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games + PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games + export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m ' + ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m ' + export 'MAVEN_OPTS=-Xmx1g ' + MAVEN_OPTS='-Xmx1g ' + cd /data/hiveptest/working/ + tee /data/hiveptest/logs/PreCommit-HIVE-Build-20292/source-prep.txt + [[ false == \t\r\u\e ]] + mkdir -p maven ivy + [[ git = \s\v\n ]] + [[ git = \g\i\t ]] + [[ -z master ]] + [[ -d apache-github-source-source ]] + [[ ! -d apache-github-source-source/.git ]] + [[ ! -d apache-github-source-source ]] + date '+%Y-%m-%d %T.%3N' 2020-01-23 08:18:56.262 + cd apache-github-source-source + git fetch origin + git reset --hard HEAD HEAD is now at 05cabc8 HIVE-22666: Introduce TopNKey operator for PTF Reduce Sink (Krisztian Kasa, reviewed by Jesus Camacho Rodriguez) + git clean -f -d Removing standalone-metastore/metastore-server/src/gen/ + git checkout master Already on 'master' Your branch is up-to-date with 'origin/master'. + git reset --hard origin/master HEAD is now at 05cabc8 HIVE-22666: Introduce TopNKey operator for PTF Reduce Sink (Krisztian Kasa, reviewed by Jesus Camacho Rodriguez) + git merge --ff-only origin/master Already up-to-date. + date '+%Y-%m-%d %T.%3N' 2020-01-23 08:18:57.371 + rm -rf ../yetus_PreCommit-HIVE-Build-20292 + mkdir ../yetus_PreCommit-HIVE-Build-20292 + git gc + cp -R . ../yetus_PreCommit-HIVE-Build-20292 + mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-20292/yetus + patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh + patchFilePath=/data/hiveptest/working/scratch/build.patch + [[ -f /data/hiveptest/working/scratch/build.patch ]] + chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh + /data/hiveptest/working/scratch/smart-apply-patch.sh /data/hiveptest/working/scratch/build.patch Trying to apply the patch with -p0 error: a/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java: does not exist in index error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/TopNKeyFilter.java: does not exist in index error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/TopNKeyOperator.java: does not exist in index error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorTopNKeyOperator.java: does not exist in index error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/vector/wrapper/VectorHashKeyWrapperBatch.java: does not exist in index error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/vector/wrapper/VectorHashKeyWrapperGeneralComparator.java: does not exist in index error: a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/topnkey/TopNKeyProcessor.java: does not exist in index error: a/ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java: does not exist in index Trying to apply the patch with -p1 error: patch failed: ql/src/java/org/apache/hadoop/hive/ql/exec/TopNKeyOperator.java:38 Falling back to three-way merge... Applied patch to 'ql/src/java/org/apache/hadoop/hive/ql/exec/TopNKeyOperator.java' with conflicts. error: patch failed: ql/src/java/org/apache/hadoop/hive/ql/optimizer/topnkey/TopNKeyProcessor.java:58 Falling back to three-way merge... Applied patch to 'ql/src/java/org/apache/hadoop/hive/ql/optimizer/topnkey/TopNKeyProcessor.java' with conflicts. Going to apply patch with: git apply -p1 error: patch failed: ql/src/java/org/apache/hadoop/hive/ql/exec/TopNKeyOperator.java:38 Falling back to three-way merge... Applied patch to 'ql/src/java/org/apache/hadoop/hive/ql/exec/TopNKeyOperator.java' with conflicts. error: patch failed: ql/src/java/org/apache/hadoop/hive/ql/optimizer/topnkey/TopNKeyProcessor.java:58 Falling back to three-way merge... Applied patch to 'ql/src/java/org/apache/hadoop/hive/ql/optimizer/topnkey/TopNKeyProcessor.java' with conflicts. U ql/src/java/org/apache/hadoop/hive/ql/exec/TopNKeyOperator.java U ql/src/java/org/apache/hadoop/hive/ql/optimizer/topnkey/TopNKeyProcessor.java + result=1 + '[' 1 -ne 0 ']' + rm -rf yetus_PreCommit-HIVE-Build-20292 + exit 1 '
This message is automatically generated.
ATTACHMENT ID: 12991579 - PreCommit-HIVE-Build
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12991666/HIVE-22726.3.patch
ERROR: -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/20309/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20309/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20309/
Messages:
Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Tests exited with: NonZeroExitCodeException Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit status 1 and output '+ date '+%Y-%m-%d %T.%3N' 2020-01-24 01:40:18.344 + [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]] + export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 + JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 + export PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games + PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games + export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m ' + ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m ' + export 'MAVEN_OPTS=-Xmx1g ' + MAVEN_OPTS='-Xmx1g ' + cd /data/hiveptest/working/ + tee /data/hiveptest/logs/PreCommit-HIVE-Build-20309/source-prep.txt + [[ false == \t\r\u\e ]] + mkdir -p maven ivy + [[ git = \s\v\n ]] + [[ git = \g\i\t ]] + [[ -z master ]] + [[ -d apache-github-source-source ]] + [[ ! -d apache-github-source-source/.git ]] + [[ ! -d apache-github-source-source ]] + date '+%Y-%m-%d %T.%3N' 2020-01-24 01:40:18.346 + cd apache-github-source-source + git fetch origin + git reset --hard HEAD HEAD is now at 4193b54 HIVE-22751: Move locking in HiveServer2::isDeregisteredWithZooKeeper to ZooKeeperHiveHelper (Rajesh Balamohan, reviewed by Anishek Agarwal) + git clean -f -d Removing ${project.basedir}/ Removing itests/${project.basedir}/ Removing standalone-metastore/metastore-server/src/gen/ + git checkout master Already on 'master' Your branch is up-to-date with 'origin/master'. + git reset --hard origin/master HEAD is now at 4193b54 HIVE-22751: Move locking in HiveServer2::isDeregisteredWithZooKeeper to ZooKeeperHiveHelper (Rajesh Balamohan, reviewed by Anishek Agarwal) + git merge --ff-only origin/master Already up-to-date. + date '+%Y-%m-%d %T.%3N' 2020-01-24 01:40:19.523 + rm -rf ../yetus_PreCommit-HIVE-Build-20309 + mkdir ../yetus_PreCommit-HIVE-Build-20309 + git gc + cp -R . ../yetus_PreCommit-HIVE-Build-20309 + mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-20309/yetus + patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh + patchFilePath=/data/hiveptest/working/scratch/build.patch + [[ -f /data/hiveptest/working/scratch/build.patch ]] + chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh + /data/hiveptest/working/scratch/smart-apply-patch.sh /data/hiveptest/working/scratch/build.patch Trying to apply the patch with -p0 error: a/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java: does not exist in index error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/TopNKeyFilter.java: does not exist in index error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/TopNKeyOperator.java: does not exist in index error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorTopNKeyOperator.java: does not exist in index error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/vector/wrapper/VectorHashKeyWrapperBatch.java: does not exist in index error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/vector/wrapper/VectorHashKeyWrapperGeneralComparator.java: does not exist in index error: a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/topnkey/TopNKeyProcessor.java: does not exist in index error: a/ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java: does not exist in index Trying to apply the patch with -p1 error: patch failed: ql/src/java/org/apache/hadoop/hive/ql/exec/TopNKeyOperator.java:38 Falling back to three-way merge... Applied patch to 'ql/src/java/org/apache/hadoop/hive/ql/exec/TopNKeyOperator.java' with conflicts. error: patch failed: ql/src/java/org/apache/hadoop/hive/ql/optimizer/topnkey/TopNKeyProcessor.java:58 Falling back to three-way merge... Applied patch to 'ql/src/java/org/apache/hadoop/hive/ql/optimizer/topnkey/TopNKeyProcessor.java' with conflicts. Going to apply patch with: git apply -p1 error: patch failed: ql/src/java/org/apache/hadoop/hive/ql/exec/TopNKeyOperator.java:38 Falling back to three-way merge... Applied patch to 'ql/src/java/org/apache/hadoop/hive/ql/exec/TopNKeyOperator.java' with conflicts. error: patch failed: ql/src/java/org/apache/hadoop/hive/ql/optimizer/topnkey/TopNKeyProcessor.java:58 Falling back to three-way merge... Applied patch to 'ql/src/java/org/apache/hadoop/hive/ql/optimizer/topnkey/TopNKeyProcessor.java' with conflicts. U ql/src/java/org/apache/hadoop/hive/ql/exec/TopNKeyOperator.java U ql/src/java/org/apache/hadoop/hive/ql/optimizer/topnkey/TopNKeyProcessor.java + result=1 + '[' 1 -ne 0 ']' + rm -rf yetus_PreCommit-HIVE-Build-20309 + exit 1 '
This message is automatically generated.
ATTACHMENT ID: 12991666 - PreCommit-HIVE-Build
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12991724/HIVE-22726.3.patch
ERROR: -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/20323/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20323/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20323/
Messages:
Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Tests exited with: NonZeroExitCodeException Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit status 1 and output '+ date '+%Y-%m-%d %T.%3N' 2020-01-24 19:05:34.638 + [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]] + export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 + JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 + export PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games + PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games + export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m ' + ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m ' + export 'MAVEN_OPTS=-Xmx1g ' + MAVEN_OPTS='-Xmx1g ' + cd /data/hiveptest/working/ + tee /data/hiveptest/logs/PreCommit-HIVE-Build-20323/source-prep.txt + [[ false == \t\r\u\e ]] + mkdir -p maven ivy + [[ git = \s\v\n ]] + [[ git = \g\i\t ]] + [[ -z master ]] + [[ -d apache-github-source-source ]] + [[ ! -d apache-github-source-source/.git ]] + [[ ! -d apache-github-source-source ]] + date '+%Y-%m-%d %T.%3N' 2020-01-24 19:05:34.659 + cd apache-github-source-source + git fetch origin + git reset --hard HEAD HEAD is now at a344426 HIVE-22712: ReExec Driver execute submit the query in default queue irrespective of user defined queue (Rajkumar Singh via Zoltan Haindrich) + git clean -f -d Removing standalone-metastore/metastore-server/src/gen/ + git checkout master Already on 'master' Your branch is up-to-date with 'origin/master'. + git reset --hard origin/master HEAD is now at a344426 HIVE-22712: ReExec Driver execute submit the query in default queue irrespective of user defined queue (Rajkumar Singh via Zoltan Haindrich) + git merge --ff-only origin/master Already up-to-date. + date '+%Y-%m-%d %T.%3N' 2020-01-24 19:05:41.147 + rm -rf ../yetus_PreCommit-HIVE-Build-20323 + mkdir ../yetus_PreCommit-HIVE-Build-20323 + git gc + cp -R . ../yetus_PreCommit-HIVE-Build-20323 + mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-20323/yetus + patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh + patchFilePath=/data/hiveptest/working/scratch/build.patch + [[ -f /data/hiveptest/working/scratch/build.patch ]] + chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh + /data/hiveptest/working/scratch/smart-apply-patch.sh /data/hiveptest/working/scratch/build.patch Trying to apply the patch with -p0 error: a/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java: does not exist in index error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/TopNKeyFilter.java: does not exist in index error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/TopNKeyOperator.java: does not exist in index error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorTopNKeyOperator.java: does not exist in index error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/vector/wrapper/VectorHashKeyWrapperBatch.java: does not exist in index error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/vector/wrapper/VectorHashKeyWrapperGeneralComparator.java: does not exist in index error: a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/topnkey/TopNKeyProcessor.java: does not exist in index error: a/ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java: does not exist in index Trying to apply the patch with -p1 error: patch failed: ql/src/java/org/apache/hadoop/hive/ql/exec/TopNKeyOperator.java:38 Falling back to three-way merge... Applied patch to 'ql/src/java/org/apache/hadoop/hive/ql/exec/TopNKeyOperator.java' with conflicts. error: patch failed: ql/src/java/org/apache/hadoop/hive/ql/optimizer/topnkey/TopNKeyProcessor.java:58 Falling back to three-way merge... Applied patch to 'ql/src/java/org/apache/hadoop/hive/ql/optimizer/topnkey/TopNKeyProcessor.java' with conflicts. Going to apply patch with: git apply -p1 error: patch failed: ql/src/java/org/apache/hadoop/hive/ql/exec/TopNKeyOperator.java:38 Falling back to three-way merge... Applied patch to 'ql/src/java/org/apache/hadoop/hive/ql/exec/TopNKeyOperator.java' with conflicts. error: patch failed: ql/src/java/org/apache/hadoop/hive/ql/optimizer/topnkey/TopNKeyProcessor.java:58 Falling back to three-way merge... Applied patch to 'ql/src/java/org/apache/hadoop/hive/ql/optimizer/topnkey/TopNKeyProcessor.java' with conflicts. U ql/src/java/org/apache/hadoop/hive/ql/exec/TopNKeyOperator.java U ql/src/java/org/apache/hadoop/hive/ql/optimizer/topnkey/TopNKeyProcessor.java + result=1 + '[' 1 -ne 0 ']' + rm -rf yetus_PreCommit-HIVE-Build-20323 + exit 1 '
This message is automatically generated.
ATTACHMENT ID: 12991724 - PreCommit-HIVE-Build
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12991882/HIVE-22726.4.patch
ERROR: -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/20353/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20353/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20353/
Messages:
Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Tests exited with: NonZeroExitCodeException Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit status 1 and output '+ date '+%Y-%m-%d %T.%3N' 2020-01-27 12:11:09.579 + [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]] + export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 + JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 + export PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games + PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games + export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m ' + ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m ' + export 'MAVEN_OPTS=-Xmx1g ' + MAVEN_OPTS='-Xmx1g ' + cd /data/hiveptest/working/ + tee /data/hiveptest/logs/PreCommit-HIVE-Build-20353/source-prep.txt + [[ false == \t\r\u\e ]] + mkdir -p maven ivy + [[ git = \s\v\n ]] + [[ git = \g\i\t ]] + [[ -z master ]] + [[ -d apache-github-source-source ]] + [[ ! -d apache-github-source-source/.git ]] + [[ ! -d apache-github-source-source ]] + date '+%Y-%m-%d %T.%3N' 2020-01-27 12:11:09.582 + cd apache-github-source-source + git fetch origin From https://github.com/apache/hive 2ea4f44..7bb1d1e master -> origin/master + git reset --hard HEAD HEAD is now at 2ea4f44 HIVE-22772: Log opType and session level information for each operation (Denys Kuzmenko via Peter Vary) + git clean -f -d Removing standalone-metastore/metastore-server/src/gen/ + git checkout master Already on 'master' Your branch is behind 'origin/master' by 1 commit, and can be fast-forwarded. (use "git pull" to update your local branch) + git reset --hard origin/master HEAD is now at 7bb1d1e HIVE-22518: SQLStdHiveAuthorizerFactoryForTest doesn't work correctly for llap tests (Zoltan Haindrich reviewed by Miklos Gergely) + git merge --ff-only origin/master Already up-to-date. + date '+%Y-%m-%d %T.%3N' 2020-01-27 12:11:11.169 + rm -rf ../yetus_PreCommit-HIVE-Build-20353 + mkdir ../yetus_PreCommit-HIVE-Build-20353 + git gc + cp -R . ../yetus_PreCommit-HIVE-Build-20353 + mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-20353/yetus + patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh + patchFilePath=/data/hiveptest/working/scratch/build.patch + [[ -f /data/hiveptest/working/scratch/build.patch ]] + chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh + /data/hiveptest/working/scratch/smart-apply-patch.sh /data/hiveptest/working/scratch/build.patch Trying to apply the patch with -p0 error: a/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java: does not exist in index error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/TopNKeyFilter.java: does not exist in index error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/TopNKeyOperator.java: does not exist in index error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorTopNKeyOperator.java: does not exist in index error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/vector/wrapper/VectorHashKeyWrapperBatch.java: does not exist in index error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/vector/wrapper/VectorHashKeyWrapperGeneralComparator.java: does not exist in index error: a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/topnkey/TopNKeyProcessor.java: does not exist in index error: a/ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java: does not exist in index Trying to apply the patch with -p1 Going to apply patch with: git apply -p1 + [[ maven == \m\a\v\e\n ]] + rm -rf /data/hiveptest/working/maven/org/apache/hive + mvn -B clean install -DskipTests -T 4 -q -Dmaven.repo.local=/data/hiveptest/working/maven protoc-jar: executing: [/tmp/protoc844606007848413894.exe, --version] libprotoc 2.5.0 protoc-jar: executing: [/tmp/protoc844606007848413894.exe, -I/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/src/main/protobuf/org/apache/hadoop/hive/metastore, --java_out=/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/target/generated-sources, /data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/src/main/protobuf/org/apache/hadoop/hive/metastore/metastore.proto] ANTLR Parser Generator Version 3.5.2 protoc-jar: executing: [/tmp/protoc6870449775196821794.exe, --version] libprotoc 2.5.0 ANTLR Parser Generator Version 3.5.2 Output file /data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-server/target/generated-sources/org/apache/hadoop/hive/metastore/parser/FilterParser.java does not exist: must build /data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/parser/Filter.g org/apache/hadoop/hive/metastore/parser/Filter.g log4j:WARN No appenders could be found for logger (DataNucleus.Persistence). log4j:WARN Please initialize the log4j system properly. DataNucleus Enhancer (version 4.1.17) for API "JDO" DataNucleus Enhancer completed with success for 43 classes. Processing annotations Annotations processed Processing annotations No elements to process [ERROR] Failed to execute goal org.apache.maven.plugins:maven-enforcer-plugin:1.3.1:enforce (check-banned-imports) on project hive-exec: Some Enforcer rules have failed. Look above for specific messages explaining why the rule failed. -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn <goals> -rf :hive-exec + result=1 + '[' 1 -ne 0 ']' + rm -rf yetus_PreCommit-HIVE-Build-20353 + exit 1 '
This message is automatically generated.
ATTACHMENT ID: 12991882 - PreCommit-HIVE-Build
-1 overall |
Vote | Subsystem | Runtime | Comment |
---|---|---|---|
Prechecks | |||
+1 | @author | 0m 0s | The patch does not contain any @author tags. |
master Compile Tests | |||
0 | mvndep | 2m 56s | Maven dependency ordering for branch |
+1 | mvninstall | 9m 18s | master passed |
+1 | compile | 1m 35s | master passed |
+1 | checkstyle | 1m 0s | master passed |
0 | findbugs | 0m 39s | common in master has 63 extant Findbugs warnings. |
0 | findbugs | 4m 32s | ql in master has 1538 extant Findbugs warnings. |
+1 | javadoc | 1m 20s | master passed |
Patch Compile Tests | |||
0 | mvndep | 0m 29s | Maven dependency ordering for patch |
+1 | mvninstall | 2m 8s | the patch passed |
+1 | compile | 1m 47s | the patch passed |
+1 | javac | 1m 47s | the patch passed |
-1 | checkstyle | 0m 49s | ql: The patch generated 1 new + 75 unchanged - 0 fixed = 76 total (was 75) |
+1 | whitespace | 0m 0s | The patch has no whitespace issues. |
+1 | findbugs | 5m 48s | the patch passed |
+1 | javadoc | 1m 19s | the patch passed |
Other Tests | |||
+1 | asflicense | 0m 15s | The patch does not generate ASF License warnings. |
35m 7s |
Subsystem | Report/Notes |
---|---|
Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
Build tool | maven |
Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-20380/dev-support/hive-personality.sh |
git revision | master / 22f5ab5 |
Default Java | 1.8.0_111 |
findbugs | v3.0.1 |
checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-20380/yetus/diff-checkstyle-ql.txt |
modules | C: common ql U: . |
Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-20380/yetus.txt |
Powered by | Apache Yetus http://yetus.apache.org |
This message was automatically generated.
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12992005/HIVE-22726.5.patch
SUCCESS: +1 due to 1 test(s) being added or modified.
SUCCESS: +1 due to 17953 tests passed
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/20380/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20380/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20380/
Messages:
Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase
This message is automatically generated.
ATTACHMENT ID: 12992005 - PreCommit-HIVE-Build
cc: rzhappy