[HIVE-4381] Implement vectorized aggregation expressions - ASF JIRA

Jitendra Nath Pandey created issue - 18/Apr/13 19:53

Hide

Permalink

Phabricator added a comment - 22/Apr/13 15:51

rusanu requested code review of "~~HIVE-4381~~ [jira] Implement vectorized aggregation expressions".

Reviewers: JIRA

Vectorized agregate initial implementation. Works for COUNT, SUM, MAX, MIN, AVG, VARIANCE (pop/samp) and STD (pop/samp). GROUP BY does not work, nor COUNT.

Vectorized implementation for sum, min, max, average and count.

TEST PLAN
Added TestVectorGroupByOperator

REVISION DETAIL
https://reviews.facebook.net/D10449

AFFECTED FILES
.arcconfig
.checkstyle
build.xml
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorFileSinkOperator.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorFilterOperator.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorGroupByOperator.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorMapOperator.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorSelectOperator.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorVectorizerOperator.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizationContext.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizedRowBatch.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizedSerde.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/BinaryExpression.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/ColAndCol.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/ColOrCol.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/ColumnExpression.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/FilterExprAndExpr.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/FilterExprOrExpr.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/FilterNotExpr.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/FilterStringColLikeStringScalar.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/IdentityExpression.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/IsNotNull.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/IsNull.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/NotCol.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/SelectColumnIsFalse.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/SelectColumnIsNotNull.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/SelectColumnIsNull.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/SelectColumnIsTrue.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/StringExpr.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/StringLength.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/StringLower.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/StringUnaryUDF.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/StringUpper.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/VectorAggregateExpression.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFAvgDouble.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFAvgLong.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFCountDouble.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFCountLong.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFMaxDouble.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFMaxLong.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFMinDouble.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFMinLong.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFStdPopDouble.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFStdPopLong.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFStdSampDouble.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFStdSampLong.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFSumDouble.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFSumLong.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFVarPopDouble.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFVarPopLong.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFVarSampDouble.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFVarSampLong.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/CodeGen.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/VectorUDAFAvg.txt
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/VectorUDAFCount.txt
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/VectorUDAFMinMax.txt
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/VectorUDAFSum.txt
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/VectorUDAFVar.txt
ql/src/java/org/apache/hadoop/hive/ql/udf/IUDFUnaryString.java
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/TestVectorGroupByOperator.java
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeCaptureOutputDesc.java
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeCaptureOutputOperator.java
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorDataSourceOperator.java
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorDataSourceOperatorDesc.java
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorRowBatchBase.java
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorRowBatchFromConcat.java
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorRowBatchFromIterables.java
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorRowBatchFromRepeats.java

MANAGE HERALD RULES
https://reviews.facebook.net/herald/view/differential/

WHY DID I GET THIS EMAIL?
https://reviews.facebook.net/herald/transcript/24993/

To: JIRA, rusanu

Show

Phabricator added a comment - 22/Apr/13 15:51 rusanu requested code review of " HIVE-4381 [jira] Implement vectorized aggregation expressions". Reviewers: JIRA Vectorized agregate initial implementation. Works for COUNT, SUM, MAX, MIN, AVG, VARIANCE (pop/samp) and STD (pop/samp). GROUP BY does not work, nor COUNT . Vectorized implementation for sum, min, max, average and count. TEST PLAN Added TestVectorGroupByOperator REVISION DETAIL https://reviews.facebook.net/D10449 AFFECTED FILES .arcconfig .checkstyle build.xml ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorFileSinkOperator.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorFilterOperator.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorGroupByOperator.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorMapOperator.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorSelectOperator.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorVectorizerOperator.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizationContext.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizedRowBatch.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizedSerde.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/BinaryExpression.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/ColAndCol.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/ColOrCol.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/ColumnExpression.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/FilterExprAndExpr.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/FilterExprOrExpr.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/FilterNotExpr.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/FilterStringColLikeStringScalar.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/IdentityExpression.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/IsNotNull.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/IsNull.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/NotCol.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/SelectColumnIsFalse.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/SelectColumnIsNotNull.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/SelectColumnIsNull.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/SelectColumnIsTrue.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/StringExpr.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/StringLength.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/StringLower.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/StringUnaryUDF.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/StringUpper.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/VectorAggregateExpression.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFAvgDouble.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFAvgLong.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFCountDouble.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFCountLong.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFMaxDouble.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFMaxLong.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFMinDouble.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFMinLong.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFStdPopDouble.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFStdPopLong.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFStdSampDouble.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFStdSampLong.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFSumDouble.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFSumLong.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFVarPopDouble.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFVarPopLong.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFVarSampDouble.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFVarSampLong.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/CodeGen.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/VectorUDAFAvg.txt ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/VectorUDAFCount.txt ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/VectorUDAFMinMax.txt ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/VectorUDAFSum.txt ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/VectorUDAFVar.txt ql/src/java/org/apache/hadoop/hive/ql/udf/IUDFUnaryString.java ql/src/test/org/apache/hadoop/hive/ql/exec/vector/TestVectorGroupByOperator.java ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeCaptureOutputDesc.java ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeCaptureOutputOperator.java ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorDataSourceOperator.java ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorDataSourceOperatorDesc.java ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorRowBatchBase.java ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorRowBatchFromConcat.java ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorRowBatchFromIterables.java ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorRowBatchFromRepeats.java MANAGE HERALD RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/24993/ To: JIRA, rusanu

Phabricator made changes - 22/Apr/13 15:51

Field	Original Value	New Value
Attachment		HIVE-4381.D10449.1.patch [ 12579841 ]

Hide

Permalink

Remus Rusanu added a comment - 22/Apr/13 15:58

Not sure how arc ended up adding my local changes to .arcconfig, .checkstyle and build.xml. They obviously should not be part of the patch request.

Show

Remus Rusanu added a comment - 22/Apr/13 15:58 Not sure how arc ended up adding my local changes to .arcconfig, .checkstyle and build.xml. They obviously should not be part of the patch request.

Remus Rusanu made changes - 23/Apr/13 07:46

Status

Open [ 1 ]

In Progress [ 3 ]

Hide

Permalink

Phabricator added a comment - 23/Apr/13 08:35

rusanu updated the revision "~~HIVE-4381~~ [jira] Implement vectorized aggregation expressions".

Undo changes to enlistment root files

Reviewers: JIRA

REVISION DETAIL
https://reviews.facebook.net/D10449

CHANGE SINCE LAST DIFF
https://reviews.facebook.net/D10449?vs=32637&id=32721#toc

AFFECTED FILES
.arcconfig
build.xml
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorFileSinkOperator.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorFilterOperator.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorGroupByOperator.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorMapOperator.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorSelectOperator.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorVectorizerOperator.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizationContext.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizedRowBatch.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizedSerde.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/BinaryExpression.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/ColAndCol.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/ColOrCol.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/ColumnExpression.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/FilterExprAndExpr.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/FilterExprOrExpr.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/FilterNotExpr.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/FilterStringColLikeStringScalar.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/IdentityExpression.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/IsNotNull.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/IsNull.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/NotCol.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/SelectColumnIsFalse.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/SelectColumnIsNotNull.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/SelectColumnIsNull.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/SelectColumnIsTrue.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/StringExpr.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/StringLength.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/StringLower.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/StringUnaryUDF.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/StringUpper.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/VectorAggregateExpression.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFAvgDouble.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFAvgLong.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFCountDouble.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFCountLong.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFMaxDouble.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFMaxLong.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFMinDouble.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFMinLong.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFStdPopDouble.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFStdPopLong.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFStdSampDouble.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFStdSampLong.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFSumDouble.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFSumLong.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFVarPopDouble.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFVarPopLong.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFVarSampDouble.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFVarSampLong.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/CodeGen.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/VectorUDAFAvg.txt
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/VectorUDAFCount.txt
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/VectorUDAFMinMax.txt
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/VectorUDAFSum.txt
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/VectorUDAFVar.txt
ql/src/java/org/apache/hadoop/hive/ql/udf/IUDFUnaryString.java
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/TestVectorGroupByOperator.java
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeCaptureOutputDesc.java
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeCaptureOutputOperator.java
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorDataSourceOperator.java
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorDataSourceOperatorDesc.java
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorRowBatchBase.java
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorRowBatchFromConcat.java
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorRowBatchFromIterables.java
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorRowBatchFromRepeats.java

To: JIRA, rusanu

Show

Phabricator added a comment - 23/Apr/13 08:35 rusanu updated the revision " HIVE-4381 [jira] Implement vectorized aggregation expressions". Undo changes to enlistment root files Reviewers: JIRA REVISION DETAIL https://reviews.facebook.net/D10449 CHANGE SINCE LAST DIFF https://reviews.facebook.net/D10449?vs=32637&id=32721#toc AFFECTED FILES .arcconfig build.xml ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorFileSinkOperator.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorFilterOperator.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorGroupByOperator.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorMapOperator.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorSelectOperator.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorVectorizerOperator.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizationContext.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizedRowBatch.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizedSerde.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/BinaryExpression.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/ColAndCol.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/ColOrCol.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/ColumnExpression.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/FilterExprAndExpr.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/FilterExprOrExpr.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/FilterNotExpr.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/FilterStringColLikeStringScalar.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/IdentityExpression.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/IsNotNull.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/IsNull.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/NotCol.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/SelectColumnIsFalse.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/SelectColumnIsNotNull.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/SelectColumnIsNull.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/SelectColumnIsTrue.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/StringExpr.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/StringLength.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/StringLower.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/StringUnaryUDF.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/StringUpper.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/VectorAggregateExpression.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFAvgDouble.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFAvgLong.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFCountDouble.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFCountLong.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFMaxDouble.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFMaxLong.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFMinDouble.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFMinLong.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFStdPopDouble.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFStdPopLong.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFStdSampDouble.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFStdSampLong.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFSumDouble.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFSumLong.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFVarPopDouble.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFVarPopLong.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFVarSampDouble.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFVarSampLong.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/CodeGen.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/VectorUDAFAvg.txt ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/VectorUDAFCount.txt ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/VectorUDAFMinMax.txt ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/VectorUDAFSum.txt ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/VectorUDAFVar.txt ql/src/java/org/apache/hadoop/hive/ql/udf/IUDFUnaryString.java ql/src/test/org/apache/hadoop/hive/ql/exec/vector/TestVectorGroupByOperator.java ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeCaptureOutputDesc.java ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeCaptureOutputOperator.java ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorDataSourceOperator.java ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorDataSourceOperatorDesc.java ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorRowBatchBase.java ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorRowBatchFromConcat.java ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorRowBatchFromIterables.java ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorRowBatchFromRepeats.java To: JIRA, rusanu

Phabricator made changes - 23/Apr/13 08:35

Attachment

HIVE-4381.D10449.2.patch [ 12579991 ]

Hide

Permalink

Phabricator added a comment - 23/Apr/13 09:09

rusanu updated the revision "~~HIVE-4381~~ [jira] Implement vectorized aggregation expressions".

Reactivate commented code in CodeGen.java

Reviewers: JIRA

REVISION DETAIL
https://reviews.facebook.net/D10449

CHANGE SINCE LAST DIFF
https://reviews.facebook.net/D10449?vs=32721&id=32727#toc

AFFECTED FILES
.arcconfig
build.xml
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorFileSinkOperator.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorFilterOperator.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorGroupByOperator.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorMapOperator.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorSelectOperator.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorVectorizerOperator.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizationContext.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizedRowBatch.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizedSerde.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/BinaryExpression.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/ColAndCol.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/ColOrCol.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/ColumnExpression.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/FilterExprAndExpr.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/FilterExprOrExpr.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/FilterNotExpr.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/FilterStringColLikeStringScalar.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/IdentityExpression.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/IsNotNull.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/IsNull.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/NotCol.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/SelectColumnIsFalse.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/SelectColumnIsNotNull.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/SelectColumnIsNull.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/SelectColumnIsTrue.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/StringExpr.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/StringLength.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/StringLower.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/StringUnaryUDF.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/StringUpper.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/VectorAggregateExpression.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFAvgDouble.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFAvgLong.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFCountDouble.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFCountLong.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFMaxDouble.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFMaxLong.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFMinDouble.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFMinLong.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFStdPopDouble.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFStdPopLong.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFStdSampDouble.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFStdSampLong.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFSumDouble.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFSumLong.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFVarPopDouble.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFVarPopLong.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFVarSampDouble.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFVarSampLong.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/CodeGen.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/VectorUDAFAvg.txt
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/VectorUDAFCount.txt
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/VectorUDAFMinMax.txt
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/VectorUDAFSum.txt
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/VectorUDAFVar.txt
ql/src/java/org/apache/hadoop/hive/ql/udf/IUDFUnaryString.java
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/TestVectorGroupByOperator.java
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeCaptureOutputDesc.java
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeCaptureOutputOperator.java
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorDataSourceOperator.java
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorDataSourceOperatorDesc.java
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorRowBatchBase.java
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorRowBatchFromConcat.java
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorRowBatchFromIterables.java
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorRowBatchFromRepeats.java

To: JIRA, rusanu

Show

Phabricator added a comment - 23/Apr/13 09:09 rusanu updated the revision " HIVE-4381 [jira] Implement vectorized aggregation expressions". Reactivate commented code in CodeGen.java Reviewers: JIRA REVISION DETAIL https://reviews.facebook.net/D10449 CHANGE SINCE LAST DIFF https://reviews.facebook.net/D10449?vs=32721&id=32727#toc AFFECTED FILES .arcconfig build.xml ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorFileSinkOperator.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorFilterOperator.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorGroupByOperator.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorMapOperator.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorSelectOperator.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorVectorizerOperator.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizationContext.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizedRowBatch.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizedSerde.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/BinaryExpression.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/ColAndCol.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/ColOrCol.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/ColumnExpression.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/FilterExprAndExpr.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/FilterExprOrExpr.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/FilterNotExpr.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/FilterStringColLikeStringScalar.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/IdentityExpression.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/IsNotNull.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/IsNull.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/NotCol.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/SelectColumnIsFalse.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/SelectColumnIsNotNull.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/SelectColumnIsNull.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/SelectColumnIsTrue.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/StringExpr.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/StringLength.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/StringLower.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/StringUnaryUDF.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/StringUpper.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/VectorAggregateExpression.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFAvgDouble.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFAvgLong.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFCountDouble.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFCountLong.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFMaxDouble.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFMaxLong.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFMinDouble.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFMinLong.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFStdPopDouble.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFStdPopLong.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFStdSampDouble.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFStdSampLong.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFSumDouble.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFSumLong.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFVarPopDouble.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFVarPopLong.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFVarSampDouble.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFVarSampLong.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/CodeGen.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/VectorUDAFAvg.txt ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/VectorUDAFCount.txt ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/VectorUDAFMinMax.txt ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/VectorUDAFSum.txt ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/VectorUDAFVar.txt ql/src/java/org/apache/hadoop/hive/ql/udf/IUDFUnaryString.java ql/src/test/org/apache/hadoop/hive/ql/exec/vector/TestVectorGroupByOperator.java ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeCaptureOutputDesc.java ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeCaptureOutputOperator.java ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorDataSourceOperator.java ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorDataSourceOperatorDesc.java ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorRowBatchBase.java ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorRowBatchFromConcat.java ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorRowBatchFromIterables.java ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorRowBatchFromRepeats.java To: JIRA, rusanu

Phabricator made changes - 23/Apr/13 09:09

Attachment

HIVE-4381.D10449.3.patch [ 12579995 ]

Hide

Permalink

Remus Rusanu added a comment - 23/Apr/13 18:01

Path submitted through Phabricator

Show

Remus Rusanu added a comment - 23/Apr/13 18:01 Path submitted through Phabricator

Remus Rusanu made changes - 23/Apr/13 18:01

Status	In Progress [ 3 ]	Patch Available [ 10002 ]
Affects Version/s		vectorization-branch [ 12324334 ]
Labels		patch
Fix Version/s		vectorization-branch [ 12324334 ]

Remus Rusanu made changes - 23/Apr/13 18:06

Component/s

Query Processor [ 12312586 ]

Hide

Permalink

Phabricator added a comment - 24/Apr/13 11:09

rusanu updated the revision "~~HIVE-4381~~ [jira] Implement vectorized aggregation expressions".

Merge latest path submited mto vectorization, ammend the diff from head

Reviewers: JIRA

REVISION DETAIL
https://reviews.facebook.net/D10449

CHANGE SINCE LAST DIFF
https://reviews.facebook.net/D10449?vs=32727&id=32781#toc

AFFECTED FILES
.arcconfig
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorFileSinkOperator.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorFilterOperator.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorGroupByOperator.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorMapOperator.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorSelectOperator.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorVectorizerOperator.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizationContext.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizedRowBatch.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizedSerde.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/BinaryExpression.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/ColAndCol.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/ColOrCol.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/ColumnExpression.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/FilterExprAndExpr.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/FilterExprOrExpr.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/FilterNotExpr.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/FilterStringColLikeStringScalar.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/IdentityExpression.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/IsNotNull.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/IsNull.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/NotCol.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/SelectColumnIsFalse.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/SelectColumnIsNotNull.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/SelectColumnIsNull.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/SelectColumnIsTrue.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/StringExpr.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/StringLength.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/StringLower.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/StringUnaryUDF.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/StringUpper.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/VectorAggregateExpression.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFAvgDouble.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFAvgLong.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFCountDouble.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFCountLong.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFMaxDouble.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFMaxLong.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFMinDouble.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFMinLong.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFStdPopDouble.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFStdPopLong.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFStdSampDouble.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFStdSampLong.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFSumDouble.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFSumLong.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFVarPopDouble.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFVarPopLong.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFVarSampDouble.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFVarSampLong.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/DoubleColAddDoubleColumn.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/DoubleColAddLongColumn.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/DoubleColDivideDoubleColumn.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/DoubleColDivideLongColumn.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/DoubleColMultiplyDoubleColumn.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/DoubleColMultiplyLongColumn.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/DoubleColSubtractDoubleColumn.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/DoubleColSubtractLongColumn.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/DoubleScalarAddDoubleColumn.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/DoubleScalarAddLongColumn.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/DoubleScalarDivideDoubleColumn.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/DoubleScalarDivideLongColumn.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/DoubleScalarModuloDoubleColumn.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/DoubleScalarModuloLongColumn.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/DoubleScalarMultiplyDoubleColumn.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/DoubleScalarMultiplyLongColumn.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/DoubleScalarSubtractDoubleColumn.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/DoubleScalarSubtractLongColumn.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/FilterDoubleColEqualDoubleColumn.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/FilterDoubleColEqualLongColumn.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/FilterDoubleColGreaterDoubleColumn.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/FilterDoubleColGreaterEqualDoubleColumn.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/FilterDoubleColGreaterEqualLongColumn.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/FilterDoubleColGreaterLongColumn.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/FilterDoubleColLessDoubleColumn.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/FilterDoubleColLessEqualDoubleColumn.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/FilterDoubleColLessEqualLongColumn.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/FilterDoubleColLessLongColumn.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/FilterDoubleColNotEqualDoubleColumn.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/FilterDoubleColNotEqualLongColumn.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/FilterLongColEqualDoubleColumn.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/FilterLongColEqualLongColumn.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/FilterLongColGreaterDoubleColumn.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/FilterLongColGreaterEqualDoubleColumn.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/FilterLongColGreaterEqualLongColumn.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/FilterLongColGreaterLongColumn.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/FilterLongColLessDoubleColumn.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/FilterLongColLessEqualDoubleColumn.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/FilterLongColLessEqualLongColumn.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/FilterLongColLessLongColumn.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/FilterLongColNotEqualDoubleColumn.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/FilterLongColNotEqualLongColumn.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/LongColAddDoubleColumn.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/LongColAddLongColumn.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/LongColDivideDoubleColumn.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/LongColDivideLongColumn.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/LongColModuloLongColumn.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/LongColMultiplyDoubleColumn.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/LongColMultiplyLongColumn.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/LongColSubtractDoubleColumn.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/LongColSubtractLongColumn.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/LongScalarAddDoubleColumn.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/LongScalarAddLongColumn.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/LongScalarDivideDoubleColumn.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/LongScalarDivideLongColumn.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/LongScalarModuloDoubleColumn.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/LongScalarModuloLongColumn.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/LongScalarMultiplyDoubleColumn.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/LongScalarMultiplyLongColumn.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/LongScalarSubtractDoubleColumn.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/LongScalarSubtractLongColumn.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/CodeGen.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/ColumnArithmeticColumn.txt
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/FilterColumnCompareColumn.txt
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/ScalarArithmeticColumn.txt
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/VectorUDAFAvg.txt
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/VectorUDAFCount.txt
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/VectorUDAFMinMax.txt
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/VectorUDAFSum.txt
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/VectorUDAFVar.txt
ql/src/java/org/apache/hadoop/hive/ql/udf/IUDFUnaryString.java
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/TestVectorGroupByOperator.java
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/expressions/TestVectorArithmeticExpressions.java
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/expressions/TestVectorFilterExpressions.java
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/expressions/TestVectorScalarColArithmetic.java
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/expressions/VectorizedRowGroupGenUtil.java
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeCaptureOutputDesc.java
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeCaptureOutputOperator.java
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorDataSourceOperator.java
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorDataSourceOperatorDesc.java
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorRowBatchBase.java
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorRowBatchFromConcat.java
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorRowBatchFromIterables.java
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorRowBatchFromRepeats.java

To: JIRA, rusanu

Show

Phabricator added a comment - 24/Apr/13 11:09 rusanu updated the revision " HIVE-4381 [jira] Implement vectorized aggregation expressions". Merge latest path submited mto vectorization, ammend the diff from head Reviewers: JIRA REVISION DETAIL https://reviews.facebook.net/D10449 CHANGE SINCE LAST DIFF https://reviews.facebook.net/D10449?vs=32727&id=32781#toc AFFECTED FILES .arcconfig ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorFileSinkOperator.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorFilterOperator.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorGroupByOperator.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorMapOperator.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorSelectOperator.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorVectorizerOperator.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizationContext.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizedRowBatch.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizedSerde.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/BinaryExpression.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/ColAndCol.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/ColOrCol.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/ColumnExpression.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/FilterExprAndExpr.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/FilterExprOrExpr.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/FilterNotExpr.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/FilterStringColLikeStringScalar.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/IdentityExpression.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/IsNotNull.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/IsNull.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/NotCol.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/SelectColumnIsFalse.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/SelectColumnIsNotNull.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/SelectColumnIsNull.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/SelectColumnIsTrue.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/StringExpr.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/StringLength.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/StringLower.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/StringUnaryUDF.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/StringUpper.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/VectorAggregateExpression.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFAvgDouble.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFAvgLong.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFCountDouble.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFCountLong.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFMaxDouble.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFMaxLong.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFMinDouble.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFMinLong.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFStdPopDouble.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFStdPopLong.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFStdSampDouble.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFStdSampLong.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFSumDouble.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFSumLong.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFVarPopDouble.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFVarPopLong.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFVarSampDouble.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFVarSampLong.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/DoubleColAddDoubleColumn.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/DoubleColAddLongColumn.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/DoubleColDivideDoubleColumn.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/DoubleColDivideLongColumn.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/DoubleColMultiplyDoubleColumn.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/DoubleColMultiplyLongColumn.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/DoubleColSubtractDoubleColumn.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/DoubleColSubtractLongColumn.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/DoubleScalarAddDoubleColumn.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/DoubleScalarAddLongColumn.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/DoubleScalarDivideDoubleColumn.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/DoubleScalarDivideLongColumn.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/DoubleScalarModuloDoubleColumn.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/DoubleScalarModuloLongColumn.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/DoubleScalarMultiplyDoubleColumn.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/DoubleScalarMultiplyLongColumn.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/DoubleScalarSubtractDoubleColumn.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/DoubleScalarSubtractLongColumn.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/FilterDoubleColEqualDoubleColumn.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/FilterDoubleColEqualLongColumn.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/FilterDoubleColGreaterDoubleColumn.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/FilterDoubleColGreaterEqualDoubleColumn.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/FilterDoubleColGreaterEqualLongColumn.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/FilterDoubleColGreaterLongColumn.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/FilterDoubleColLessDoubleColumn.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/FilterDoubleColLessEqualDoubleColumn.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/FilterDoubleColLessEqualLongColumn.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/FilterDoubleColLessLongColumn.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/FilterDoubleColNotEqualDoubleColumn.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/FilterDoubleColNotEqualLongColumn.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/FilterLongColEqualDoubleColumn.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/FilterLongColEqualLongColumn.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/FilterLongColGreaterDoubleColumn.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/FilterLongColGreaterEqualDoubleColumn.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/FilterLongColGreaterEqualLongColumn.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/FilterLongColGreaterLongColumn.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/FilterLongColLessDoubleColumn.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/FilterLongColLessEqualDoubleColumn.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/FilterLongColLessEqualLongColumn.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/FilterLongColLessLongColumn.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/FilterLongColNotEqualDoubleColumn.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/FilterLongColNotEqualLongColumn.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/LongColAddDoubleColumn.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/LongColAddLongColumn.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/LongColDivideDoubleColumn.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/LongColDivideLongColumn.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/LongColModuloLongColumn.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/LongColMultiplyDoubleColumn.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/LongColMultiplyLongColumn.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/LongColSubtractDoubleColumn.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/LongColSubtractLongColumn.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/LongScalarAddDoubleColumn.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/LongScalarAddLongColumn.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/LongScalarDivideDoubleColumn.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/LongScalarDivideLongColumn.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/LongScalarModuloDoubleColumn.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/LongScalarModuloLongColumn.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/LongScalarMultiplyDoubleColumn.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/LongScalarMultiplyLongColumn.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/LongScalarSubtractDoubleColumn.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/LongScalarSubtractLongColumn.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/CodeGen.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/ColumnArithmeticColumn.txt ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/FilterColumnCompareColumn.txt ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/ScalarArithmeticColumn.txt ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/VectorUDAFAvg.txt ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/VectorUDAFCount.txt ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/VectorUDAFMinMax.txt ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/VectorUDAFSum.txt ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/VectorUDAFVar.txt ql/src/java/org/apache/hadoop/hive/ql/udf/IUDFUnaryString.java ql/src/test/org/apache/hadoop/hive/ql/exec/vector/TestVectorGroupByOperator.java ql/src/test/org/apache/hadoop/hive/ql/exec/vector/expressions/TestVectorArithmeticExpressions.java ql/src/test/org/apache/hadoop/hive/ql/exec/vector/expressions/TestVectorFilterExpressions.java ql/src/test/org/apache/hadoop/hive/ql/exec/vector/expressions/TestVectorScalarColArithmetic.java ql/src/test/org/apache/hadoop/hive/ql/exec/vector/expressions/VectorizedRowGroupGenUtil.java ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeCaptureOutputDesc.java ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeCaptureOutputOperator.java ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorDataSourceOperator.java ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorDataSourceOperatorDesc.java ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorRowBatchBase.java ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorRowBatchFromConcat.java ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorRowBatchFromIterables.java ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorRowBatchFromRepeats.java To: JIRA, rusanu

Phabricator made changes - 24/Apr/13 11:09

Attachment

HIVE-4381.D10449.4.patch [ 12580266 ]

Hide

Permalink

Phabricator added a comment - 25/Apr/13 11:14

rusanu requested code review of "~~HIVE-4381~~ [jira] Implement vectorized aggregation expressions".

Reviewers: JIRA

Vector aggregate operators. Vectorized implementation for sum, min, max, average, count and variance (std, var, samp, pop).
Introduce a skeleton VectorGroupByOperator. This does not yet address actual group by (work onyl for full table).
Vectorized aggregate expressions are generated using the CodeGen templates ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/

TEST PLAN
TestVectorGroupByOperator added for JUnit

REVISION DETAIL
https://reviews.facebook.net/D10551

AFFECTED FILES
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorGroupByOperator.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizationContext.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/ColumnExpression.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFAvgDouble.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFAvgLong.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFCountDouble.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFCountLong.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFMaxDouble.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFMaxLong.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFMinDouble.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFMinLong.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFStdPopDouble.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFStdPopLong.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFStdSampDouble.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFStdSampLong.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFSumDouble.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFSumLong.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFVarPopDouble.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFVarPopLong.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFVarSampDouble.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFVarSampLong.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/CodeGen.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/VectorUDAFAvg.txt
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/VectorUDAFCount.txt
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/VectorUDAFMinMax.txt
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/VectorUDAFSum.txt
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/VectorUDAFVar.txt
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/TestVectorGroupByOperator.java
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeCaptureOutputDesc.java
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeCaptureOutputOperator.java
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorDataSourceOperator.java
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorDataSourceOperatorDesc.java
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorRowBatchBase.java
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorRowBatchFromConcat.java
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorRowBatchFromIterables.java
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorRowBatchFromRepeats.java

MANAGE HERALD RULES
https://reviews.facebook.net/herald/view/differential/

WHY DID I GET THIS EMAIL?
https://reviews.facebook.net/herald/transcript/25233/

To: JIRA, rusanu

Show

Phabricator added a comment - 25/Apr/13 11:14 rusanu requested code review of " HIVE-4381 [jira] Implement vectorized aggregation expressions". Reviewers: JIRA Vector aggregate operators. Vectorized implementation for sum, min, max, average, count and variance (std, var, samp, pop). Introduce a skeleton VectorGroupByOperator. This does not yet address actual group by (work onyl for full table). Vectorized aggregate expressions are generated using the CodeGen templates ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/ TEST PLAN TestVectorGroupByOperator added for JUnit REVISION DETAIL https://reviews.facebook.net/D10551 AFFECTED FILES ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorGroupByOperator.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizationContext.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/ColumnExpression.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFAvgDouble.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFAvgLong.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFCountDouble.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFCountLong.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFMaxDouble.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFMaxLong.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFMinDouble.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFMinLong.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFStdPopDouble.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFStdPopLong.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFStdSampDouble.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFStdSampLong.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFSumDouble.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFSumLong.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFVarPopDouble.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFVarPopLong.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFVarSampDouble.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFVarSampLong.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/CodeGen.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/VectorUDAFAvg.txt ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/VectorUDAFCount.txt ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/VectorUDAFMinMax.txt ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/VectorUDAFSum.txt ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/VectorUDAFVar.txt ql/src/test/org/apache/hadoop/hive/ql/exec/vector/TestVectorGroupByOperator.java ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeCaptureOutputDesc.java ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeCaptureOutputOperator.java ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorDataSourceOperator.java ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorDataSourceOperatorDesc.java ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorRowBatchBase.java ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorRowBatchFromConcat.java ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorRowBatchFromIterables.java ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorRowBatchFromRepeats.java MANAGE HERALD RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/25233/ To: JIRA, rusanu

Phabricator made changes - 25/Apr/13 11:14

Attachment

HIVE-4381.D10551.1.patch [ 12580513 ]

Hide

Permalink

Remus Rusanu added a comment - 25/Apr/13 11:17

I have resubmitted with a much smaller patch that only covers the actual vectorized aggregate operators. This does not work end-to-end in a query yet, but the code compiles and JUnit tests pass. End-to-end functionality requires some more patches from Jitendra to go in first (the VectorMapExec).

Show

Remus Rusanu added a comment - 25/Apr/13 11:17 I have resubmitted with a much smaller patch that only covers the actual vectorized aggregate operators. This does not work end-to-end in a query yet, but the code compiles and JUnit tests pass. End-to-end functionality requires some more patches from Jitendra to go in first (the VectorMapExec).

Hide

Permalink

Phabricator added a comment - 29/Apr/13 15:12

rusanu updated the revision "~~HIVE-4381~~ [jira] Implement vectorized aggregation expressions".

update patch after 4f7470d

Reviewers: JIRA

REVISION DETAIL
https://reviews.facebook.net/D10551

CHANGE SINCE LAST DIFF
https://reviews.facebook.net/D10551?vs=32901&id=33147#toc

AFFECTED FILES
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorGroupByOperator.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizationContext.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/ColumnExpression.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/VectorAggregateExpression.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFAvgDouble.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFAvgLong.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFCountDouble.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFCountLong.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFMaxDouble.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFMaxLong.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFMinDouble.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFMinLong.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFStdPopDouble.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFStdPopLong.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFStdSampDouble.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFStdSampLong.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFSumDouble.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFSumLong.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFVarPopDouble.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFVarPopLong.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFVarSampDouble.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFVarSampLong.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/CodeGen.java
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/VectorUDAFAvg.txt
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/VectorUDAFCount.txt
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/VectorUDAFMinMax.txt
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/VectorUDAFSum.txt
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/VectorUDAFVar.txt
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/TestVectorGroupByOperator.java
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeCaptureOutputDesc.java
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeCaptureOutputOperator.java
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorDataSourceOperator.java
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorDataSourceOperatorDesc.java
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorRowBatchBase.java
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorRowBatchFromConcat.java
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorRowBatchFromIterables.java
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorRowBatchFromRepeats.java

To: JIRA, rusanu

Show

Phabricator added a comment - 29/Apr/13 15:12 rusanu updated the revision " HIVE-4381 [jira] Implement vectorized aggregation expressions". update patch after 4f7470d Reviewers: JIRA REVISION DETAIL https://reviews.facebook.net/D10551 CHANGE SINCE LAST DIFF https://reviews.facebook.net/D10551?vs=32901&id=33147#toc AFFECTED FILES ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorGroupByOperator.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizationContext.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/ColumnExpression.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/VectorAggregateExpression.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFAvgDouble.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFAvgLong.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFCountDouble.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFCountLong.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFMaxDouble.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFMaxLong.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFMinDouble.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFMinLong.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFStdPopDouble.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFStdPopLong.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFStdSampDouble.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFStdSampLong.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFSumDouble.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFSumLong.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFVarPopDouble.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFVarPopLong.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFVarSampDouble.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFVarSampLong.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/CodeGen.java ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/VectorUDAFAvg.txt ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/VectorUDAFCount.txt ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/VectorUDAFMinMax.txt ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/VectorUDAFSum.txt ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/VectorUDAFVar.txt ql/src/test/org/apache/hadoop/hive/ql/exec/vector/TestVectorGroupByOperator.java ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeCaptureOutputDesc.java ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeCaptureOutputOperator.java ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorDataSourceOperator.java ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorDataSourceOperatorDesc.java ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorRowBatchBase.java ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorRowBatchFromConcat.java ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorRowBatchFromIterables.java ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorRowBatchFromRepeats.java To: JIRA, rusanu

Phabricator made changes - 29/Apr/13 15:12

Attachment

HIVE-4381.D10551.2.patch [ 12580969 ]

Hide

Permalink

Phabricator added a comment - 29/Apr/13 16:06

ashutoshc has accepted the revision "~~HIVE-4381~~ [jira] Implement vectorized aggregation expressions".

+1

REVISION DETAIL
https://reviews.facebook.net/D10551

BRANCH
vectorization

ARCANIST PROJECT
hive

To: JIRA, ashutoshc, rusanu

Show

Phabricator added a comment - 29/Apr/13 16:06 ashutoshc has accepted the revision " HIVE-4381 [jira] Implement vectorized aggregation expressions". +1 REVISION DETAIL https://reviews.facebook.net/D10551 BRANCH vectorization ARCANIST PROJECT hive To: JIRA, ashutoshc, rusanu

Hide

Permalink

Ashutosh Chauhan added a comment - 29/Apr/13 16:13

Committed to branch. Thanks, Remus!

Show

Ashutosh Chauhan added a comment - 29/Apr/13 16:13 Committed to branch. Thanks, Remus!

Ashutosh Chauhan made changes - 29/Apr/13 16:13

Status	Patch Available [ 10002 ]	Resolved [ 5 ]
Resolution		Fixed [ 1 ]

Thejas M Nair made changes - 23/Oct/13 22:59

Fix Version/s

0.13.0 [ 12324986 ]

Implement vectorized aggregation expressions

Details

Description

Attachments

Activity

People

Dates

Development

Agile