Details
-
Type:
Sub-task
-
Status: Resolved
-
Priority:
Minor
-
Resolution: Fixed
-
Affects Version/s: vectorization-branch
-
Fix Version/s: vectorization-branch, 0.13.0
-
Component/s: Query Processor
-
Labels:None
Description
With the dataset attached by Tony, running:
select max(ctinyint), max(csmallint), max(cint), max(cbigint), max(cfloat), max(cdouble) from alltypes_orc;
this is the result in rowmode:
OK
62 16379 1073737829 2146661184 62.0 16376.0
This is the result in vector mode:
OK
1 16379 1071926788 -106115134 62.0 16361.0
Activity
- All
- Comments
- Work Log
- History
- Activity
- Transitions
Overall: -1 no tests executed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12594134/HIVE-4931.1.patch
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/185/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/185/console
Messages:
Executing org.apache.hive.ptest.execution.CleanupPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests failed with: NonZeroExitCodeException: Command 'bash /data/hive-ptest/working/scratch/source-prep.sh' failed with exit status 1 and output '+ [[ -n '' ]]
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128'
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128'
+ cd /data/hive-ptest/working/
+ tee /data/hive-ptest/logs/PreCommit-HIVE-Build-185/source-prep.txt
+ mkdir -p maven ivy
+ [[ svn = \s\v\n ]]
+ [[ -n '' ]]
+ [[ -d apache-svn-trunk-source ]]
+ [[ ! -d apache-svn-trunk-source/.svn ]]
+ [[ ! -d apache-svn-trunk-source ]]
+ cd apache-svn-trunk-source
+ svn revert -R .
Reverted 'ql/src/java/org/apache/hadoop/hive/ql/plan/ListBucketingCtx.java'
Reverted 'ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java'
Reverted 'ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java'
++ egrep -v '^X|^Performing status on external'
++ awk '{print $2}'
++ svn status --no-ignore
+ rm -rf build hcatalog/build hcatalog/core/build hcatalog/storage-handlers/hbase/build hcatalog/server-extensions/build hcatalog/webhcat/svr/build hcatalog/webhcat/java-client/build hcatalog/hcatalog-pig-adapter/build common/src/gen
+ svn update
Fetching external item into 'hcatalog/src/test/e2e/harness'
External at revision 1507115.
At revision 1507115.
+ patchCommandPath=/data/hive-ptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hive-ptest/working/scratch/build.patch
+ [[ -f /data/hive-ptest/working/scratch/build.patch ]]
+ chmod +x /data/hive-ptest/working/scratch/smart-apply-patch.sh
+ /data/hive-ptest/working/scratch/smart-apply-patch.sh /data/hive-ptest/working/scratch/build.patch
The patch does not appear to apply with p0 to p2
+ exit 1
'
This message is automatically generated.
Committed to branch. Thanks, Remus!
-@Override + @Override
And ^ means the last commit on this file went in without check-style.
Hey Edward. I'm working with Remus and others to stabilize vectorization, and this bug was found as a result of that stabilization. I've just uploaded a new patch for HIVE-4794 which contains the tests we've been using to stabilize, and which give us regression coverage for this issue.
Test patch in a follow up. Marking as closed.
The agg templates were incorrectly checking for isNull[0] in addition of hasNulls for repeating values. I also discovered that the MIN/MAX aggregate was using hard-coded comparison instead of templatized <Operand> for this case.
These are the results with the patch applied:
select max(ctinyint), max(csmallint), max(cint), max(cbigint), max(cfloat), max(cdouble) from alltypes_orc;
rowmode
62 16379 1073737829 2146661184 62.0 16376.0
vect:
62 16379 1073737829 2146661184 62.0 16376.0
select min(ctinyint), min(csmallint), min(cint), min(cbigint), min(cfloat), min(cdouble) from alltypes_orc;
rowmode:
-64 -16381 -1073517051 -2147049514 -64.0 -16376.0
vect:
-64 -16381 -1073517051 -2147049514 -64.0 -16376.0
select avg(ctinyint), avg(csmallint), avg(cint), avg(cbigint), avg(cfloat), avg(cdouble) from alltypes_orc;
rowmode:
2.522511719175842 -415.2897318508829 -1.0994357654011336E8 1.9003309384238064E7 -2.2073250490516676 -2877.6021364726403
vect:
2.522511719175842 -415.2897318508829 -1.0994357654011336E8 1.9003309384238064E7 -2.2073250490516676 -2877.6021364726403
select stddev(ctinyint), stddev(csmallint), stddev(cint), stddev(cbigint), stddev(cfloat), stddev(cdouble) from alltypes_orc;
rowmode:
34.52579284952266 8956.673250179136 6.765972128417507E8 1.0284485805202962E9 38.5241575060334 10107.644590369582
vect:
34.52579284952266 8956.673250179136 6.765972128417507E8 1.0284485805202962E9 38.5241575060334 10107.644590369582