Hive
  1. Hive
  2. HIVE-5283

Merge vectorization branch to trunk

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.13.0
    • Component/s: None
    • Labels:
      None

      Description

      The purpose of this jira is to upload vectorization patch, run tests etc. The actual work will continue under HIVE-4160 umbrella jira.

      1. HIVE-5283.4.patch
        2.61 MB
        Jitendra Nath Pandey
      2. HIVE-5283.3.patch
        2.55 MB
        Jitendra Nath Pandey
      3. HIVE-5283.2.patch
        4.55 MB
        Jitendra Nath Pandey
      4. HIVE-5283.1.patch
        4.55 MB
        Jitendra Nath Pandey
      5. alltypesorc
        368 kB
        Jitendra Nath Pandey

        Issue Links

          Activity

          Hide
          Thejas M Nair added a comment -

          Added fix version of 0.13 in addition to vectorization-branch for these 106 jiras (fixed+ fix-version=vectorization-branch ).

          Show
          Thejas M Nair added a comment - Added fix version of 0.13 in addition to vectorization-branch for these 106 jiras (fixed+ fix-version=vectorization-branch ).
          Hide
          Thejas M Nair added a comment -

          Now that vectorization changes are merged into trunk, I think we should add fix-version of 0.13 to the jiras. That way the release notes for 0.13 will include these changes as well. I will go ahead and make that change to the jiras.

          Show
          Thejas M Nair added a comment - Now that vectorization changes are merged into trunk, I think we should add fix-version of 0.13 to the jiras. That way the release notes for 0.13 will include these changes as well. I will go ahead and make that change to the jiras.
          Hide
          Hudson added a comment -

          SUCCESS: Integrated in Hive-trunk-hadoop1-ptest #187 (See https://builds.apache.org/job/Hive-trunk-hadoop1-ptest/187/)
          HIVE-5283 : Merge vectorization branch to trunk (Jitendra Nath Pandey via Ashutosh Chauhan) (hashutosh: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1527858)

          • /hive/trunk
          • /hive/trunk/ant/src/org/apache/hadoop/hive/ant/GenVectorCode.java
          • /hive/trunk/ant/src/org/apache/hadoop/hive/ant/GenVectorTestCode.java
          • /hive/trunk/build-common.xml
          • /hive/trunk/build.xml
          • /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
          • /hive/trunk/conf/hive-default.xml.template
          • /hive/trunk/data/files/alltypesorc
          • /hive/trunk/ql/build.xml
          • /hive/trunk/ql/src/gen/vectorization
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FetchOperator.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FilterOperator.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/GroupByOperator.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/KeyWrapper.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Operator.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/OperatorFactory.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ReduceSinkOperator.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/mr/ExecDriver.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/vector
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/CommonRCFileInputFormat.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/VectorizedRCFileInputFormat.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/VectorizedRCFileRecordReader.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/BitFieldReader.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/DynamicByteArray.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/IntegerReader.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcSerde.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/RecordReader.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/RecordReaderImpl.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/RunLengthByteReader.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/RunLengthIntegerReader.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/RunLengthIntegerReaderV2.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/VectorizedOrcInputFormat.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/VectorizedOrcSerde.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/PhysicalOptimizer.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/AbstractOperatorDesc.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/ExprNodeGenericFuncDesc.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/MapWork.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFHex.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/util/JavaDataModel.java
          • /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/QTestUtil.java
          • /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/exec/vector
          • /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestVectorizedORCReader.java
          • /hive/trunk/ql/src/test/queries/clientpositive/vectorization_short_regress.q
          • /hive/trunk/ql/src/test/queries/clientpositive/vectorized_rcfile_columnar.q
          • /hive/trunk/ql/src/test/results/clientpositive/add_part_exist.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/alter1.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/alter2.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/alter3.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/alter4.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/alter5.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/alter_index.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/alter_rename_partition.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/describe_table_json.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/index_creation.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/input2.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/input3.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/input4.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/plan_json.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/rename_column.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/show_tables.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/vectorization_short_regress.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/vectorized_rcfile_columnar.q.out
          • /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/columnar/ColumnarSerDe.java
          • /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyUtils.java
          Show
          Hudson added a comment - SUCCESS: Integrated in Hive-trunk-hadoop1-ptest #187 (See https://builds.apache.org/job/Hive-trunk-hadoop1-ptest/187/ ) HIVE-5283 : Merge vectorization branch to trunk (Jitendra Nath Pandey via Ashutosh Chauhan) (hashutosh: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1527858 ) /hive/trunk /hive/trunk/ant/src/org/apache/hadoop/hive/ant/GenVectorCode.java /hive/trunk/ant/src/org/apache/hadoop/hive/ant/GenVectorTestCode.java /hive/trunk/build-common.xml /hive/trunk/build.xml /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java /hive/trunk/conf/hive-default.xml.template /hive/trunk/data/files/alltypesorc /hive/trunk/ql/build.xml /hive/trunk/ql/src/gen/vectorization /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FetchOperator.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FilterOperator.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/GroupByOperator.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/KeyWrapper.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Operator.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/OperatorFactory.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ReduceSinkOperator.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/mr/ExecDriver.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/vector /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/CommonRCFileInputFormat.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/VectorizedRCFileInputFormat.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/VectorizedRCFileRecordReader.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/BitFieldReader.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/DynamicByteArray.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/IntegerReader.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcSerde.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/RecordReader.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/RecordReaderImpl.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/RunLengthByteReader.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/RunLengthIntegerReader.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/RunLengthIntegerReaderV2.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/VectorizedOrcInputFormat.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/VectorizedOrcSerde.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/PhysicalOptimizer.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/AbstractOperatorDesc.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/ExprNodeGenericFuncDesc.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/MapWork.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFHex.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/util/JavaDataModel.java /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/QTestUtil.java /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/exec/vector /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestVectorizedORCReader.java /hive/trunk/ql/src/test/queries/clientpositive/vectorization_short_regress.q /hive/trunk/ql/src/test/queries/clientpositive/vectorized_rcfile_columnar.q /hive/trunk/ql/src/test/results/clientpositive/add_part_exist.q.out /hive/trunk/ql/src/test/results/clientpositive/alter1.q.out /hive/trunk/ql/src/test/results/clientpositive/alter2.q.out /hive/trunk/ql/src/test/results/clientpositive/alter3.q.out /hive/trunk/ql/src/test/results/clientpositive/alter4.q.out /hive/trunk/ql/src/test/results/clientpositive/alter5.q.out /hive/trunk/ql/src/test/results/clientpositive/alter_index.q.out /hive/trunk/ql/src/test/results/clientpositive/alter_rename_partition.q.out /hive/trunk/ql/src/test/results/clientpositive/describe_table_json.q.out /hive/trunk/ql/src/test/results/clientpositive/index_creation.q.out /hive/trunk/ql/src/test/results/clientpositive/input2.q.out /hive/trunk/ql/src/test/results/clientpositive/input3.q.out /hive/trunk/ql/src/test/results/clientpositive/input4.q.out /hive/trunk/ql/src/test/results/clientpositive/plan_json.q.out /hive/trunk/ql/src/test/results/clientpositive/rename_column.q.out /hive/trunk/ql/src/test/results/clientpositive/show_tables.q.out /hive/trunk/ql/src/test/results/clientpositive/vectorization_short_regress.q.out /hive/trunk/ql/src/test/results/clientpositive/vectorized_rcfile_columnar.q.out /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/columnar/ColumnarSerDe.java /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyUtils.java
          Hide
          Hudson added a comment -

          FAILURE: Integrated in Hive-trunk-hadoop2-ptest #121 (See https://builds.apache.org/job/Hive-trunk-hadoop2-ptest/121/)
          HIVE-5283 : Merge vectorization branch to trunk (Jitendra Nath Pandey via Ashutosh Chauhan) (hashutosh: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1527858)

          • /hive/trunk
          • /hive/trunk/ant/src/org/apache/hadoop/hive/ant/GenVectorCode.java
          • /hive/trunk/ant/src/org/apache/hadoop/hive/ant/GenVectorTestCode.java
          • /hive/trunk/build-common.xml
          • /hive/trunk/build.xml
          • /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
          • /hive/trunk/conf/hive-default.xml.template
          • /hive/trunk/data/files/alltypesorc
          • /hive/trunk/ql/build.xml
          • /hive/trunk/ql/src/gen/vectorization
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FetchOperator.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FilterOperator.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/GroupByOperator.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/KeyWrapper.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Operator.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/OperatorFactory.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ReduceSinkOperator.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/mr/ExecDriver.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/vector
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/CommonRCFileInputFormat.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/VectorizedRCFileInputFormat.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/VectorizedRCFileRecordReader.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/BitFieldReader.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/DynamicByteArray.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/IntegerReader.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcSerde.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/RecordReader.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/RecordReaderImpl.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/RunLengthByteReader.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/RunLengthIntegerReader.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/RunLengthIntegerReaderV2.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/VectorizedOrcInputFormat.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/VectorizedOrcSerde.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/PhysicalOptimizer.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/AbstractOperatorDesc.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/ExprNodeGenericFuncDesc.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/MapWork.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFHex.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/util/JavaDataModel.java
          • /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/QTestUtil.java
          • /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/exec/vector
          • /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestVectorizedORCReader.java
          • /hive/trunk/ql/src/test/queries/clientpositive/vectorization_short_regress.q
          • /hive/trunk/ql/src/test/queries/clientpositive/vectorized_rcfile_columnar.q
          • /hive/trunk/ql/src/test/results/clientpositive/add_part_exist.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/alter1.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/alter2.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/alter3.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/alter4.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/alter5.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/alter_index.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/alter_rename_partition.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/describe_table_json.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/index_creation.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/input2.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/input3.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/input4.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/plan_json.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/rename_column.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/show_tables.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/vectorization_short_regress.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/vectorized_rcfile_columnar.q.out
          • /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/columnar/ColumnarSerDe.java
          • /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyUtils.java
          Show
          Hudson added a comment - FAILURE: Integrated in Hive-trunk-hadoop2-ptest #121 (See https://builds.apache.org/job/Hive-trunk-hadoop2-ptest/121/ ) HIVE-5283 : Merge vectorization branch to trunk (Jitendra Nath Pandey via Ashutosh Chauhan) (hashutosh: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1527858 ) /hive/trunk /hive/trunk/ant/src/org/apache/hadoop/hive/ant/GenVectorCode.java /hive/trunk/ant/src/org/apache/hadoop/hive/ant/GenVectorTestCode.java /hive/trunk/build-common.xml /hive/trunk/build.xml /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java /hive/trunk/conf/hive-default.xml.template /hive/trunk/data/files/alltypesorc /hive/trunk/ql/build.xml /hive/trunk/ql/src/gen/vectorization /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FetchOperator.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FilterOperator.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/GroupByOperator.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/KeyWrapper.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Operator.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/OperatorFactory.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ReduceSinkOperator.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/mr/ExecDriver.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/vector /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/CommonRCFileInputFormat.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/VectorizedRCFileInputFormat.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/VectorizedRCFileRecordReader.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/BitFieldReader.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/DynamicByteArray.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/IntegerReader.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcSerde.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/RecordReader.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/RecordReaderImpl.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/RunLengthByteReader.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/RunLengthIntegerReader.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/RunLengthIntegerReaderV2.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/VectorizedOrcInputFormat.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/VectorizedOrcSerde.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/PhysicalOptimizer.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/AbstractOperatorDesc.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/ExprNodeGenericFuncDesc.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/MapWork.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFHex.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/util/JavaDataModel.java /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/QTestUtil.java /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/exec/vector /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestVectorizedORCReader.java /hive/trunk/ql/src/test/queries/clientpositive/vectorization_short_regress.q /hive/trunk/ql/src/test/queries/clientpositive/vectorized_rcfile_columnar.q /hive/trunk/ql/src/test/results/clientpositive/add_part_exist.q.out /hive/trunk/ql/src/test/results/clientpositive/alter1.q.out /hive/trunk/ql/src/test/results/clientpositive/alter2.q.out /hive/trunk/ql/src/test/results/clientpositive/alter3.q.out /hive/trunk/ql/src/test/results/clientpositive/alter4.q.out /hive/trunk/ql/src/test/results/clientpositive/alter5.q.out /hive/trunk/ql/src/test/results/clientpositive/alter_index.q.out /hive/trunk/ql/src/test/results/clientpositive/alter_rename_partition.q.out /hive/trunk/ql/src/test/results/clientpositive/describe_table_json.q.out /hive/trunk/ql/src/test/results/clientpositive/index_creation.q.out /hive/trunk/ql/src/test/results/clientpositive/input2.q.out /hive/trunk/ql/src/test/results/clientpositive/input3.q.out /hive/trunk/ql/src/test/results/clientpositive/input4.q.out /hive/trunk/ql/src/test/results/clientpositive/plan_json.q.out /hive/trunk/ql/src/test/results/clientpositive/rename_column.q.out /hive/trunk/ql/src/test/results/clientpositive/show_tables.q.out /hive/trunk/ql/src/test/results/clientpositive/vectorization_short_regress.q.out /hive/trunk/ql/src/test/results/clientpositive/vectorized_rcfile_columnar.q.out /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/columnar/ColumnarSerDe.java /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyUtils.java
          Hide
          Hudson added a comment -

          FAILURE: Integrated in Hive-trunk-hadoop2 #467 (See https://builds.apache.org/job/Hive-trunk-hadoop2/467/)
          HIVE-5283 : Merge vectorization branch to trunk (Jitendra Nath Pandey via Ashutosh Chauhan) (hashutosh: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1527858)

          • /hive/trunk
          • /hive/trunk/ant/src/org/apache/hadoop/hive/ant/GenVectorCode.java
          • /hive/trunk/ant/src/org/apache/hadoop/hive/ant/GenVectorTestCode.java
          • /hive/trunk/build-common.xml
          • /hive/trunk/build.xml
          • /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
          • /hive/trunk/conf/hive-default.xml.template
          • /hive/trunk/data/files/alltypesorc
          • /hive/trunk/ql/build.xml
          • /hive/trunk/ql/src/gen/vectorization
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FetchOperator.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FilterOperator.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/GroupByOperator.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/KeyWrapper.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Operator.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/OperatorFactory.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ReduceSinkOperator.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/mr/ExecDriver.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/vector
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/CommonRCFileInputFormat.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/VectorizedRCFileInputFormat.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/VectorizedRCFileRecordReader.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/BitFieldReader.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/DynamicByteArray.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/IntegerReader.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcSerde.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/RecordReader.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/RecordReaderImpl.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/RunLengthByteReader.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/RunLengthIntegerReader.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/RunLengthIntegerReaderV2.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/VectorizedOrcInputFormat.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/VectorizedOrcSerde.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/PhysicalOptimizer.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/AbstractOperatorDesc.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/ExprNodeGenericFuncDesc.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/MapWork.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFHex.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/util/JavaDataModel.java
          • /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/QTestUtil.java
          • /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/exec/vector
          • /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestVectorizedORCReader.java
          • /hive/trunk/ql/src/test/queries/clientpositive/vectorization_short_regress.q
          • /hive/trunk/ql/src/test/queries/clientpositive/vectorized_rcfile_columnar.q
          • /hive/trunk/ql/src/test/results/clientpositive/add_part_exist.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/alter1.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/alter2.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/alter3.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/alter4.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/alter5.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/alter_index.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/alter_rename_partition.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/describe_table_json.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/index_creation.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/input2.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/input3.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/input4.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/plan_json.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/rename_column.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/show_tables.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/vectorization_short_regress.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/vectorized_rcfile_columnar.q.out
          • /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/columnar/ColumnarSerDe.java
          • /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyUtils.java
          Show
          Hudson added a comment - FAILURE: Integrated in Hive-trunk-hadoop2 #467 (See https://builds.apache.org/job/Hive-trunk-hadoop2/467/ ) HIVE-5283 : Merge vectorization branch to trunk (Jitendra Nath Pandey via Ashutosh Chauhan) (hashutosh: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1527858 ) /hive/trunk /hive/trunk/ant/src/org/apache/hadoop/hive/ant/GenVectorCode.java /hive/trunk/ant/src/org/apache/hadoop/hive/ant/GenVectorTestCode.java /hive/trunk/build-common.xml /hive/trunk/build.xml /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java /hive/trunk/conf/hive-default.xml.template /hive/trunk/data/files/alltypesorc /hive/trunk/ql/build.xml /hive/trunk/ql/src/gen/vectorization /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FetchOperator.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FilterOperator.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/GroupByOperator.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/KeyWrapper.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Operator.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/OperatorFactory.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ReduceSinkOperator.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/mr/ExecDriver.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/vector /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/CommonRCFileInputFormat.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/VectorizedRCFileInputFormat.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/VectorizedRCFileRecordReader.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/BitFieldReader.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/DynamicByteArray.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/IntegerReader.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcSerde.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/RecordReader.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/RecordReaderImpl.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/RunLengthByteReader.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/RunLengthIntegerReader.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/RunLengthIntegerReaderV2.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/VectorizedOrcInputFormat.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/VectorizedOrcSerde.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/PhysicalOptimizer.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/AbstractOperatorDesc.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/ExprNodeGenericFuncDesc.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/MapWork.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFHex.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/util/JavaDataModel.java /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/QTestUtil.java /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/exec/vector /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestVectorizedORCReader.java /hive/trunk/ql/src/test/queries/clientpositive/vectorization_short_regress.q /hive/trunk/ql/src/test/queries/clientpositive/vectorized_rcfile_columnar.q /hive/trunk/ql/src/test/results/clientpositive/add_part_exist.q.out /hive/trunk/ql/src/test/results/clientpositive/alter1.q.out /hive/trunk/ql/src/test/results/clientpositive/alter2.q.out /hive/trunk/ql/src/test/results/clientpositive/alter3.q.out /hive/trunk/ql/src/test/results/clientpositive/alter4.q.out /hive/trunk/ql/src/test/results/clientpositive/alter5.q.out /hive/trunk/ql/src/test/results/clientpositive/alter_index.q.out /hive/trunk/ql/src/test/results/clientpositive/alter_rename_partition.q.out /hive/trunk/ql/src/test/results/clientpositive/describe_table_json.q.out /hive/trunk/ql/src/test/results/clientpositive/index_creation.q.out /hive/trunk/ql/src/test/results/clientpositive/input2.q.out /hive/trunk/ql/src/test/results/clientpositive/input3.q.out /hive/trunk/ql/src/test/results/clientpositive/input4.q.out /hive/trunk/ql/src/test/results/clientpositive/plan_json.q.out /hive/trunk/ql/src/test/results/clientpositive/rename_column.q.out /hive/trunk/ql/src/test/results/clientpositive/show_tables.q.out /hive/trunk/ql/src/test/results/clientpositive/vectorization_short_regress.q.out /hive/trunk/ql/src/test/results/clientpositive/vectorized_rcfile_columnar.q.out /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/columnar/ColumnarSerDe.java /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyUtils.java
          Hide
          Hudson added a comment -

          FAILURE: Integrated in Hive-trunk-h0.21 #2369 (See https://builds.apache.org/job/Hive-trunk-h0.21/2369/)
          HIVE-5283 : Merge vectorization branch to trunk (Jitendra Nath Pandey via Ashutosh Chauhan) (hashutosh: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1527858)

          • /hive/trunk
          • /hive/trunk/ant/src/org/apache/hadoop/hive/ant/GenVectorCode.java
          • /hive/trunk/ant/src/org/apache/hadoop/hive/ant/GenVectorTestCode.java
          • /hive/trunk/build-common.xml
          • /hive/trunk/build.xml
          • /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
          • /hive/trunk/conf/hive-default.xml.template
          • /hive/trunk/data/files/alltypesorc
          • /hive/trunk/ql/build.xml
          • /hive/trunk/ql/src/gen/vectorization
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FetchOperator.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FilterOperator.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/GroupByOperator.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/KeyWrapper.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Operator.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/OperatorFactory.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ReduceSinkOperator.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/mr/ExecDriver.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/vector
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/CommonRCFileInputFormat.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/VectorizedRCFileInputFormat.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/VectorizedRCFileRecordReader.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/BitFieldReader.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/DynamicByteArray.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/IntegerReader.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcSerde.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/RecordReader.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/RecordReaderImpl.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/RunLengthByteReader.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/RunLengthIntegerReader.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/RunLengthIntegerReaderV2.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/VectorizedOrcInputFormat.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/VectorizedOrcSerde.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/PhysicalOptimizer.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/AbstractOperatorDesc.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/ExprNodeGenericFuncDesc.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/MapWork.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFHex.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/util/JavaDataModel.java
          • /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/QTestUtil.java
          • /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/exec/vector
          • /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestVectorizedORCReader.java
          • /hive/trunk/ql/src/test/queries/clientpositive/vectorization_short_regress.q
          • /hive/trunk/ql/src/test/queries/clientpositive/vectorized_rcfile_columnar.q
          • /hive/trunk/ql/src/test/results/clientpositive/add_part_exist.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/alter1.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/alter2.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/alter3.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/alter4.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/alter5.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/alter_index.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/alter_rename_partition.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/describe_table_json.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/index_creation.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/input2.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/input3.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/input4.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/plan_json.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/rename_column.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/show_tables.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/vectorization_short_regress.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/vectorized_rcfile_columnar.q.out
          • /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/columnar/ColumnarSerDe.java
          • /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyUtils.java
          Show
          Hudson added a comment - FAILURE: Integrated in Hive-trunk-h0.21 #2369 (See https://builds.apache.org/job/Hive-trunk-h0.21/2369/ ) HIVE-5283 : Merge vectorization branch to trunk (Jitendra Nath Pandey via Ashutosh Chauhan) (hashutosh: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1527858 ) /hive/trunk /hive/trunk/ant/src/org/apache/hadoop/hive/ant/GenVectorCode.java /hive/trunk/ant/src/org/apache/hadoop/hive/ant/GenVectorTestCode.java /hive/trunk/build-common.xml /hive/trunk/build.xml /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java /hive/trunk/conf/hive-default.xml.template /hive/trunk/data/files/alltypesorc /hive/trunk/ql/build.xml /hive/trunk/ql/src/gen/vectorization /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FetchOperator.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FilterOperator.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/GroupByOperator.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/KeyWrapper.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Operator.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/OperatorFactory.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ReduceSinkOperator.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/mr/ExecDriver.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/vector /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/CommonRCFileInputFormat.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/VectorizedRCFileInputFormat.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/VectorizedRCFileRecordReader.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/BitFieldReader.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/DynamicByteArray.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/IntegerReader.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcSerde.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/RecordReader.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/RecordReaderImpl.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/RunLengthByteReader.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/RunLengthIntegerReader.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/RunLengthIntegerReaderV2.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/VectorizedOrcInputFormat.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/VectorizedOrcSerde.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/PhysicalOptimizer.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/AbstractOperatorDesc.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/ExprNodeGenericFuncDesc.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/MapWork.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFHex.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/util/JavaDataModel.java /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/QTestUtil.java /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/exec/vector /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestVectorizedORCReader.java /hive/trunk/ql/src/test/queries/clientpositive/vectorization_short_regress.q /hive/trunk/ql/src/test/queries/clientpositive/vectorized_rcfile_columnar.q /hive/trunk/ql/src/test/results/clientpositive/add_part_exist.q.out /hive/trunk/ql/src/test/results/clientpositive/alter1.q.out /hive/trunk/ql/src/test/results/clientpositive/alter2.q.out /hive/trunk/ql/src/test/results/clientpositive/alter3.q.out /hive/trunk/ql/src/test/results/clientpositive/alter4.q.out /hive/trunk/ql/src/test/results/clientpositive/alter5.q.out /hive/trunk/ql/src/test/results/clientpositive/alter_index.q.out /hive/trunk/ql/src/test/results/clientpositive/alter_rename_partition.q.out /hive/trunk/ql/src/test/results/clientpositive/describe_table_json.q.out /hive/trunk/ql/src/test/results/clientpositive/index_creation.q.out /hive/trunk/ql/src/test/results/clientpositive/input2.q.out /hive/trunk/ql/src/test/results/clientpositive/input3.q.out /hive/trunk/ql/src/test/results/clientpositive/input4.q.out /hive/trunk/ql/src/test/results/clientpositive/plan_json.q.out /hive/trunk/ql/src/test/results/clientpositive/rename_column.q.out /hive/trunk/ql/src/test/results/clientpositive/show_tables.q.out /hive/trunk/ql/src/test/results/clientpositive/vectorization_short_regress.q.out /hive/trunk/ql/src/test/results/clientpositive/vectorized_rcfile_columnar.q.out /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/columnar/ColumnarSerDe.java /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyUtils.java
          Ashutosh Chauhan made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Fix Version/s 0.13.0 [ 12324986 ]
          Resolution Fixed [ 1 ]
          Hide
          Ashutosh Chauhan added a comment -

          Committed to trunk. Thanks, Jitendra!

          Show
          Ashutosh Chauhan added a comment - Committed to trunk. Thanks, Jitendra!
          Hide
          Jitendra Nath Pandey added a comment -

          I have filed following 3 jiras for the follow up work:
          HIVE-5397, HIVE-5398 and HIVE-5399.

          HIVE-5397 is aimed for the discussion/work related to public members in VectorizedRowBatch.

          Show
          Jitendra Nath Pandey added a comment - I have filed following 3 jiras for the follow up work: HIVE-5397 , HIVE-5398 and HIVE-5399 . HIVE-5397 is aimed for the discussion/work related to public members in VectorizedRowBatch.
          Hide
          Brock Noland added a comment -

          Inlining would really depend on JIT implementation and

          Is there a JVM we are targeting that doesn't have an advanced JIT implementation?

          in most cases only for final classes and methods.

          Where are you getting this? I am guessing the state of of the art JIT has improved dramatically since Brian Goetz wrote this article in 2002:

          http://www.ibm.com/developerworks/java/library/j-jtp1029/index.html

          Where he says:

          "The common perception is that declaring classes or methods final makes it easier for the compiler to inline method calls, but this perception is incorrect (or at the very least, greatly overstated)."

          "So the reality is that while final might be a useful hint to a dumb run-time optimizer that doesn't perform any global dependency analysis, its use doesn't actually enable very many compile-time optimizations, and is not needed by a smart JIT to perform run-time optimizations."

          Show
          Brock Noland added a comment - Inlining would really depend on JIT implementation and Is there a JVM we are targeting that doesn't have an advanced JIT implementation? in most cases only for final classes and methods. Where are you getting this? I am guessing the state of of the art JIT has improved dramatically since Brian Goetz wrote this article in 2002: http://www.ibm.com/developerworks/java/library/j-jtp1029/index.html Where he says: "The common perception is that declaring classes or methods final makes it easier for the compiler to inline method calls, but this perception is incorrect (or at the very least, greatly overstated)." "So the reality is that while final might be a useful hint to a dumb run-time optimizer that doesn't perform any global dependency analysis, its use doesn't actually enable very many compile-time optimizations, and is not needed by a smart JIT to perform run-time optimizations."
          Hide
          Jitendra Nath Pandey added a comment -

          Inlining would really depend on JIT implementation and in most cases only for final classes and methods.
          Although, VectorizedRowBatch is not polymorphic for now, but it may evolve in future, and at that point changing the vectorized expressions could become a big deal.
          We could add comments or annotations in the class to indicate that these variables should be accessed only in restricted packages. Would that be reasonable?

          Show
          Jitendra Nath Pandey added a comment - Inlining would really depend on JIT implementation and in most cases only for final classes and methods. Although, VectorizedRowBatch is not polymorphic for now, but it may evolve in future, and at that point changing the vectorized expressions could become a big deal. We could add comments or annotations in the class to indicate that these variables should be accessed only in restricted packages. Would that be reasonable?
          Hide
          Brock Noland added a comment -

          Hey thanks for your feedback. Is there evidence showing that the method calls are not inlined by JIT?

          Show
          Brock Noland added a comment - Hey thanks for your feedback. Is there evidence showing that the method calls are not inlined by JIT?
          Hide
          Jitendra Nath Pandey added a comment -

          Thanks for the review Brock Noland.
          VectorizedRowBatch is accessed and mutated by vectorized expressions in the inner loops. The members are exposed as public to avoid method call overheads.

          I will file jiras to address rest of the comments.

          Show
          Jitendra Nath Pandey added a comment - Thanks for the review Brock Noland . VectorizedRowBatch is accessed and mutated by vectorized expressions in the inner loops. The members are exposed as public to avoid method call overheads. I will file jiras to address rest of the comments.
          Hide
          Brock Noland added a comment -

          Great work guys!

          Can you address these three issues in follow on jiras?

          We should never be blindly converting throwable to an exception.

          +    } catch (Throwable e) {
          +      throw new HiveException(e);
          +    }
          

          This is non-standard:

          +      transient private long value;
          

          Unless we have a very good reason we shouldn't be exposing the members of VectorizedRowBatch publicly.

          Show
          Brock Noland added a comment - Great work guys! Can you address these three issues in follow on jiras? We should never be blindly converting throwable to an exception. + } catch (Throwable e) { + throw new HiveException(e); + } This is non-standard: + transient private long value; Unless we have a very good reason we shouldn't be exposing the members of VectorizedRowBatch publicly.
          Hide
          Carl Steinbach added a comment -

          +1 the latest patch looks good to me.

          Show
          Carl Steinbach added a comment - +1 the latest patch looks good to me.
          Hide
          Ashutosh Chauhan added a comment -

          +1 I have overseen much of this work so I am familiar with this work. But it won't harm to get extra pair of eyes on it. So, it will be good if other committers also chime in.

          Show
          Ashutosh Chauhan added a comment - +1 I have overseen much of this work so I am familiar with this work. But it won't harm to get extra pair of eyes on it. So, it will be good if other committers also chime in.
          Hide
          Jitendra Nath Pandey added a comment -

          I ran tests in my setup. All tests pass.

          Carl Steinbach Please take a look, the patch addresses your feedback comments.
          Other committers are also requested to take a look.

          Show
          Jitendra Nath Pandey added a comment - I ran tests in my setup. All tests pass. Carl Steinbach Please take a look, the patch addresses your feedback comments. Other committers are also requested to take a look.
          Jitendra Nath Pandey made changes -
          Attachment alltypesorc [ 12605157 ]
          Hide
          Jitendra Nath Pandey added a comment -

          The tests are failing because the patch contains a binary file that patch doesn't handle correctly.
          Uploaded the binary file. I will run the tests manually and provide the results.

          Show
          Jitendra Nath Pandey added a comment - The tests are failing because the patch contains a binary file that patch doesn't handle correctly. Uploaded the binary file. I will run the tests manually and provide the results.
          Hide
          Hive QA added a comment -

          Overall: -1 at least one tests failed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12605095/HIVE-5283.4.patch

          ERROR: -1 due to 51 failed/errored test(s), 2128 tests executed
          Failed tests:

          org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_external_table_ppd
          org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_binary_external_table_queries
          org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_binary_map_queries
          org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_binary_map_queries_prefix
          org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_binary_storage_queries
          org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_joins
          org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_ppd_key_range
          org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_pushdown
          org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_queries
          org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_scan_params
          org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_single_sourced_multi_insert
          org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_stats
          org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_stats2
          org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_stats3
          org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_stats_empty_partition
          org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_ppd_key_ranges
          org.apache.hadoop.hive.cli.TestHBaseMinimrCliDriver.testCliDriver_hbase_bulk
          org.apache.hadoop.hive.cli.TestHBaseNegativeCliDriver.testCliDriver_cascade_dbdrop_hadoop20
          org.apache.hadoop.hive.ql.TestLocationQueries.testAlterTablePartitionLocation_alter5
          org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1
          org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_ambiguous_join_col
          org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_duplicate_alias
          org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_garbage
          org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_insert_wrong_number_columns
          org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_invalid_create_table
          org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_invalid_dot
          org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_invalid_function_param2
          org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_invalid_index
          org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_invalid_list_index
          org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_invalid_list_index2
          org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_invalid_map_index
          org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_invalid_map_index2
          org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_invalid_select
          org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_macro_reserved_word
          org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_missing_overwrite
          org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_nonkey_groupby
          org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_quoted_string
          org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_unknown_column1
          org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_unknown_column2
          org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_unknown_column3
          org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_unknown_column4
          org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_unknown_column5
          org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_unknown_column6
          org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_unknown_function1
          org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_unknown_function2
          org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_unknown_function3
          org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_unknown_function4
          org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_unknown_table1
          org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_unknown_table2
          org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_wrong_distinct1
          org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_wrong_distinct2
          

          Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/902/testReport
          Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/902/console

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          Tests failed with: TestsFailedException: 51 tests failed
          

          This message is automatically generated.

          Show
          Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12605095/HIVE-5283.4.patch ERROR: -1 due to 51 failed/errored test(s), 2128 tests executed Failed tests: org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_external_table_ppd org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_binary_external_table_queries org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_binary_map_queries org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_binary_map_queries_prefix org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_binary_storage_queries org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_joins org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_ppd_key_range org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_pushdown org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_queries org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_scan_params org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_single_sourced_multi_insert org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_stats org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_stats2 org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_stats3 org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_stats_empty_partition org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_ppd_key_ranges org.apache.hadoop.hive.cli.TestHBaseMinimrCliDriver.testCliDriver_hbase_bulk org.apache.hadoop.hive.cli.TestHBaseNegativeCliDriver.testCliDriver_cascade_dbdrop_hadoop20 org.apache.hadoop.hive.ql.TestLocationQueries.testAlterTablePartitionLocation_alter5 org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1 org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_ambiguous_join_col org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_duplicate_alias org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_garbage org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_insert_wrong_number_columns org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_invalid_create_table org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_invalid_dot org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_invalid_function_param2 org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_invalid_index org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_invalid_list_index org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_invalid_list_index2 org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_invalid_map_index org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_invalid_map_index2 org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_invalid_select org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_macro_reserved_word org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_missing_overwrite org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_nonkey_groupby org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_quoted_string org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_unknown_column1 org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_unknown_column2 org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_unknown_column3 org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_unknown_column4 org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_unknown_column5 org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_unknown_column6 org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_unknown_function1 org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_unknown_function2 org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_unknown_function3 org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_unknown_function4 org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_unknown_table1 org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_unknown_table2 org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_wrong_distinct1 org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_wrong_distinct2 Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/902/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/902/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests failed with: TestsFailedException: 51 tests failed This message is automatically generated.
          Hide
          Jitendra Nath Pandey added a comment -

          Are we really going to apply a patch, or are we going to SVN merge? I would rather see a merge so we do not lose the commit history on the branch.

          I am fine with either approach to get this work into trunk.

          Show
          Jitendra Nath Pandey added a comment - Are we really going to apply a patch, or are we going to SVN merge? I would rather see a merge so we do not lose the commit history on the branch. I am fine with either approach to get this work into trunk.
          Jitendra Nath Pandey made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Jitendra Nath Pandey made changes -
          Attachment HIVE-5283.4.patch [ 12605095 ]
          Jitendra Nath Pandey made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Hide
          Hive QA added a comment -

          Overall: -1 no tests executed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12604927/HIVE-5283.3.patch

          Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/884/testReport
          Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/884/console

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Tests failed with: NonZeroExitCodeException: Command 'bash /data/hive-ptest/working/scratch/source-prep.sh' failed with exit status 1 and output '+ [[ -n '' ]]
          + export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128'
          + ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128'
          + cd /data/hive-ptest/working/
          + tee /data/hive-ptest/logs/PreCommit-HIVE-Build-884/source-prep.txt
          + mkdir -p maven ivy
          + [[ svn = \s\v\n ]]
          + [[ -n '' ]]
          + [[ -d apache-svn-trunk-source ]]
          + [[ ! -d apache-svn-trunk-source/.svn ]]
          + [[ ! -d apache-svn-trunk-source ]]
          + cd apache-svn-trunk-source
          + svn revert -R .
          Reverted 'jdbc/src/java/org/apache/hive/jdbc/HiveStatement.java'
          Reverted 'service/src/java/org/apache/hive/service/cli/session/HiveSessionImpl.java'
          ++ awk '{print $2}'
          ++ egrep -v '^X|^Performing status on external'
          ++ svn status --no-ignore
          + rm -rf build hcatalog/build hcatalog/core/build hcatalog/storage-handlers/hbase/build hcatalog/server-extensions/build hcatalog/webhcat/svr/build hcatalog/webhcat/java-client/build hcatalog/hcatalog-pig-adapter/build common/src/gen
          + svn update
          A    beeline/src/test/org/apache/hive/beeline/src/test/TestSchemaTool.java
          U    beeline/src/java/org/apache/hive/beeline/BeeLineOpts.java
          A    beeline/src/java/org/apache/hive/beeline/HiveSchemaTool.java
          A    beeline/src/java/org/apache/hive/beeline/HiveSchemaHelper.java
          U    beeline/src/java/org/apache/hive/beeline/Commands.java
          U    beeline/src/java/org/apache/hive/beeline/BeeLine.java
          U    build.xml
          U    metastore/scripts/upgrade/derby/014-HIVE-3764.derby.sql
          U    metastore/scripts/upgrade/mysql/014-HIVE-3764.mysql.sql
          U    metastore/scripts/upgrade/oracle/014-HIVE-3764.oracle.sql
          U    metastore/scripts/upgrade/postgres/014-HIVE-3764.postgres.sql
          A    bin/schematool
          A    bin/ext/schemaTool.sh
          
          Fetching external item into 'hcatalog/src/test/e2e/harness'
          Updated external to revision 1526125.
          
          Updated to revision 1526125.
          + patchCommandPath=/data/hive-ptest/working/scratch/smart-apply-patch.sh
          + patchFilePath=/data/hive-ptest/working/scratch/build.patch
          + [[ -f /data/hive-ptest/working/scratch/build.patch ]]
          + chmod +x /data/hive-ptest/working/scratch/smart-apply-patch.sh
          + /data/hive-ptest/working/scratch/smart-apply-patch.sh /data/hive-ptest/working/scratch/build.patch
          The patch does not appear to apply with p0 to p2
          + exit 1
          '
          

          This message is automatically generated.

          Show
          Hive QA added a comment - Overall : -1 no tests executed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12604927/HIVE-5283.3.patch Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/884/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/884/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Tests failed with: NonZeroExitCodeException: Command 'bash /data/hive-ptest/working/scratch/source-prep.sh' failed with exit status 1 and output '+ [[ -n '' ]] + export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128' + ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128' + cd /data/hive-ptest/working/ + tee /data/hive-ptest/logs/PreCommit-HIVE-Build-884/source-prep.txt + mkdir -p maven ivy + [[ svn = \s\v\n ]] + [[ -n '' ]] + [[ -d apache-svn-trunk-source ]] + [[ ! -d apache-svn-trunk-source/.svn ]] + [[ ! -d apache-svn-trunk-source ]] + cd apache-svn-trunk-source + svn revert -R . Reverted 'jdbc/src/java/org/apache/hive/jdbc/HiveStatement.java' Reverted 'service/src/java/org/apache/hive/service/cli/session/HiveSessionImpl.java' ++ awk '{print $2}' ++ egrep -v '^X|^Performing status on external' ++ svn status --no-ignore + rm -rf build hcatalog/build hcatalog/core/build hcatalog/storage-handlers/hbase/build hcatalog/server-extensions/build hcatalog/webhcat/svr/build hcatalog/webhcat/java-client/build hcatalog/hcatalog-pig-adapter/build common/src/gen + svn update A beeline/src/test/org/apache/hive/beeline/src/test/TestSchemaTool.java U beeline/src/java/org/apache/hive/beeline/BeeLineOpts.java A beeline/src/java/org/apache/hive/beeline/HiveSchemaTool.java A beeline/src/java/org/apache/hive/beeline/HiveSchemaHelper.java U beeline/src/java/org/apache/hive/beeline/Commands.java U beeline/src/java/org/apache/hive/beeline/BeeLine.java U build.xml U metastore/scripts/upgrade/derby/014-HIVE-3764.derby.sql U metastore/scripts/upgrade/mysql/014-HIVE-3764.mysql.sql U metastore/scripts/upgrade/oracle/014-HIVE-3764.oracle.sql U metastore/scripts/upgrade/postgres/014-HIVE-3764.postgres.sql A bin/schematool A bin/ext/schemaTool.sh Fetching external item into 'hcatalog/src/test/e2e/harness' Updated external to revision 1526125. Updated to revision 1526125. + patchCommandPath=/data/hive-ptest/working/scratch/smart-apply-patch.sh + patchFilePath=/data/hive-ptest/working/scratch/build.patch + [[ -f /data/hive-ptest/working/scratch/build.patch ]] + chmod +x /data/hive-ptest/working/scratch/smart-apply-patch.sh + /data/hive-ptest/working/scratch/smart-apply-patch.sh /data/hive-ptest/working/scratch/build.patch The patch does not appear to apply with p0 to p2 + exit 1 ' This message is automatically generated.
          Hide
          Edward Capriolo added a comment -

          Are we really going to apply a patch, or are we going to SVN merge? I would rather see a merge so we do not lose the commit history on the branch.

          Show
          Edward Capriolo added a comment - Are we really going to apply a patch, or are we going to SVN merge? I would rather see a merge so we do not lose the commit history on the branch.
          Jitendra Nath Pandey made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Jitendra Nath Pandey made changes -
          Attachment HIVE-5283.3.patch [ 12604927 ]
          Hide
          Jitendra Nath Pandey added a comment -

          Updated patch, addressing the review comments. Updated on review board as well.

          Show
          Jitendra Nath Pandey added a comment - Updated patch, addressing the review comments. Updated on review board as well.
          Jitendra Nath Pandey made changes -
          Link This issue is blocked by HIVE-5309 [ HIVE-5309 ]
          Jitendra Nath Pandey made changes -
          Link This issue is blocked by HIVE-5314 [ HIVE-5314 ]
          Jitendra Nath Pandey made changes -
          Link This issue is blocked by HIVE-5308 [ HIVE-5308 ]
          Hide
          Carl Steinbach added a comment -

          Tony Murphy Thanks for the very detailed explanation. Clearly a lot of thought has gone into writing these tests. Now we just need to find a way of conveying this information to people down the road.

          I recommend doing the following: concatenate all of these tests into a single qfile named vectorization_short_regress.q and include the information from above in a comment at the top of the file. It would also be great if you could include a short comment per query so folks have an easy way of telling them apart.

          Show
          Carl Steinbach added a comment - Tony Murphy Thanks for the very detailed explanation. Clearly a lot of thought has gone into writing these tests. Now we just need to find a way of conveying this information to people down the road. I recommend doing the following: concatenate all of these tests into a single qfile named vectorization_short_regress.q and include the information from above in a comment at the top of the file. It would also be great if you could include a short comment per query so folks have an easy way of telling them apart.
          Hide
          Tony Murphy added a comment -

          Carl SteinbachThanks for the questions and I would definitely appreciate some feedback on how to appropriately document the test strategy I used here.

          In regards to you question about magic numbers in the queries, the values of effectively random, but they are important. If you look at ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/OrcFileGenerator.java which is the data generation class you'll see that those values are specified in the initializeFixedPointValues for each data type. When I created the queries I used those values where I needed scalar values to ensure that when the queries executed their predicates would be filtering on values that are guaranteed to exist.

          Beyond those values, all the other data in the alltypesorc file is random, but there is a specific pattern to the data that is important for coverage. In orc and subsequently vectorization there are a number of optimizations for certain data patterns: AllValues, NoNulls, RepeatingValue, RepeatingNull. The data in alltypesorc is generated such that each column has exactly 3 batches of each data pattern. This gives us coverage for the vector expression optimizations and ensure the metadata in appropriately set on the row batch object which are reused across batches.

          For the queries themselves in order to efficiently cover as much of the new vectorization functionality as I could I used a number of different techniques to create the vectorization_*.q test suites, primarily equivalence classes, and pairwise combinations.

          First I divided the search space into a number of dimensions such as type, aggregate function, filter operation, arithmetic operation, etc. The types were explored as equivalence classes of long, double, time, string, and bool. Also, rather than creating a very large number of small queries the resulting vectors were grouped by compatible dimensions to reduce the number of queries.

          It wouldn't be to much work to add comments into the .q files that summarize the coverage they provide based on the vectors used to create each scenario.

          Show
          Tony Murphy added a comment - Carl Steinbach Thanks for the questions and I would definitely appreciate some feedback on how to appropriately document the test strategy I used here. In regards to you question about magic numbers in the queries, the values of effectively random, but they are important. If you look at ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/OrcFileGenerator.java which is the data generation class you'll see that those values are specified in the initializeFixedPointValues for each data type. When I created the queries I used those values where I needed scalar values to ensure that when the queries executed their predicates would be filtering on values that are guaranteed to exist. Beyond those values, all the other data in the alltypesorc file is random, but there is a specific pattern to the data that is important for coverage. In orc and subsequently vectorization there are a number of optimizations for certain data patterns: AllValues, NoNulls, RepeatingValue, RepeatingNull. The data in alltypesorc is generated such that each column has exactly 3 batches of each data pattern. This gives us coverage for the vector expression optimizations and ensure the metadata in appropriately set on the row batch object which are reused across batches. For the queries themselves in order to efficiently cover as much of the new vectorization functionality as I could I used a number of different techniques to create the vectorization_*.q test suites, primarily equivalence classes, and pairwise combinations. First I divided the search space into a number of dimensions such as type, aggregate function, filter operation, arithmetic operation, etc. The types were explored as equivalence classes of long, double, time, string, and bool. Also, rather than creating a very large number of small queries the resulting vectors were grouped by compatible dimensions to reduce the number of queries. It wouldn't be to much work to add comments into the .q files that summarize the coverage they provide based on the vectors used to create each scenario.
          Hide
          Carl Steinbach added a comment -

          I added some more comments on RB. I wanted to note here since the site seems overwhelmed by the size of this patch and I have my doubts that they're actually going to get reposted here.

          Show
          Carl Steinbach added a comment - I added some more comments on RB. I wanted to note here since the site seems overwhelmed by the size of this patch and I have my doubts that they're actually going to get reposted here.
          Carl Steinbach made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Hide
          Carl Steinbach added a comment -

          Jitendra Nath Pandey Thanks for posting the patch on reviewboard. Here's some high-level feedback:

          • I think we should avoid committing generated code if at all possible. In addition, the build should be modified to run vector codegen as part of the ql compile phase.
          • This patch adds sixteen new vectorization_*.q tests. It's not clear to me what the difference is between vectorization_3.q and vectorization_8.q, or between vectorization_10.q and vectorization_7.q, etc, etc. I think it would take some effort for a maintainer to determine what the testing strategy is, or if there even is one. I'm also worried that in the future people will add redundant tests because they don't want to spend the time figuring out what is already covered. In order to avoid this I think it would be good to augment the filenames with quick descriptions, and to include comments in each qfile quickly explaining what is being tested.
          Show
          Carl Steinbach added a comment - Jitendra Nath Pandey Thanks for posting the patch on reviewboard. Here's some high-level feedback: I think we should avoid committing generated code if at all possible. In addition, the build should be modified to run vector codegen as part of the ql compile phase. This patch adds sixteen new vectorization_*.q tests. It's not clear to me what the difference is between vectorization_3.q and vectorization_8.q, or between vectorization_10.q and vectorization_7.q, etc, etc. I think it would take some effort for a maintainer to determine what the testing strategy is, or if there even is one. I'm also worried that in the future people will add redundant tests because they don't want to spend the time figuring out what is already covered. In order to avoid this I think it would be good to augment the filenames with quick descriptions, and to include comments in each qfile quickly explaining what is being tested.
          Hide
          Hive QA added a comment -

          Overall: -1 at least one tests failed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12602928/HIVE-5283.2.patch

          ERROR: -1 due to 2 failed/errored test(s), 3965 tests executed
          Failed tests:

          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_input4
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_plan_json
          

          Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/719/testReport
          Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/719/console

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          Tests failed with: TestsFailedException: 2 tests failed
          

          This message is automatically generated.

          Show
          Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12602928/HIVE-5283.2.patch ERROR: -1 due to 2 failed/errored test(s), 3965 tests executed Failed tests: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_input4 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_plan_json Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/719/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/719/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests failed with: TestsFailedException: 2 tests failed This message is automatically generated.
          Hide
          Brock Noland added a comment -

          Not sure what happened. I would re-run. But there is quite a queue at present so you might want to run locally.

          Show
          Brock Noland added a comment - Not sure what happened. I would re-run. But there is quite a queue at present so you might want to run locally.
          Hide
          Jitendra Nath Pandey added a comment -

          Brock Noland The tests are not getting executed for this patch and the console output shows following exception in hudson. Could you please take a look? I verified that the patch applies cleanly to trunk.

          ERROR: Failed to archive artifacts: hive/build/test-results/TEST-*.xml
          hudson.util.IOException2: hudson.util.IOException2: Failed to extract /home/jenkins/jenkins-slave/workspace/PreCommit-HIVE-Build/hive/build/test-results/TEST-*.xml
          	at hudson.FilePath.readFromTar(FilePath.java:2050)
          	at hudson.FilePath.copyRecursiveTo(FilePath.java:1962)
          	at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:137)
          	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
          	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:782)
          	at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:754)
          	at hudson.model.Build$BuildExecution.post2(Build.java:183)
          	at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:707)
          	at hudson.model.Run.execute(Run.java:1628)
          	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
          	at hudson.model.ResourceController.execute(ResourceController.java:88)
          	at hudson.model.Executor.run(Executor.java:246)
          Caused by: java.io.IOException
          	at hudson.remoting.FastPipedInputStream.read(FastPipedInputStream.java:175)
          	at hudson.util.HeadBufferingStream.read(HeadBufferingStream.java:61)
          	at com.jcraft.jzlib.InflaterInputStream.fill(InflaterInputStream.java:175)
          	at com.jcraft.jzlib.InflaterInputStream.read(InflaterInputStream.java:106)
          	at org.apache.tools.tar.TarBuffer.readBlock(TarBuffer.java:257)
          	at org.apache.tools.tar.TarBuffer.readRecord(TarBuffer.java:223)
          	at hudson.org.apache.tools.tar.TarInputStream.read(TarInputStream.java:345)
          	at java.io.FilterInputStream.read(FilterInputStream.java:107)
          	at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1025)
          	at org.apache.commons.io.IOUtils.copy(IOUtils.java:999)
          	at hudson.util.IOUtils.copy(IOUtils.java:37)
          	at hudson.FilePath.readFromTar(FilePath.java:2040)
          	... 11 more
          
          	at hudson.FilePath.copyRecursiveTo(FilePath.java:1969)
          	at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:137)
          	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
          	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:782)
          	at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:754)
          	at hudson.model.Build$BuildExecution.post2(Build.java:183)
          	at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:707)
          	at hudson.model.Run.execute(Run.java:1628)
          	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
          	at hudson.model.ResourceController.execute(ResourceController.java:88)
          	at hudson.model.Executor.run(Executor.java:246)
          Caused by: java.util.concurrent.ExecutionException: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
          	at hudson.remoting.Request$1.get(Request.java:278)
          	at hudson.remoting.Request$1.get(Request.java:210)
          	at hudson.remoting.FutureAdapter.get(FutureAdapter.java:59)
          	at hudson.FilePath.copyRecursiveTo(FilePath.java:1965)
          	... 10 more
          Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
          	at hudson.remoting.Request.abort(Request.java:299)
          	at hudson.remoting.Channel.terminate(Channel.java:774)
          	at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:69)
          Caused by: java.io.IOException: Unexpected termination of the channel
          	at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:50)
          Caused by: java.io.EOFException
          	at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2596)
          	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1316)
          	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
          	at hudson.remoting.Command.readFrom(Command.java:92)
          	at hudson.remoting.ClassicCommandTransport.read(ClassicCommandTransport.java:71)
          	at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:48)
          
          
          Show
          Jitendra Nath Pandey added a comment - Brock Noland The tests are not getting executed for this patch and the console output shows following exception in hudson. Could you please take a look? I verified that the patch applies cleanly to trunk. ERROR: Failed to archive artifacts: hive/build/test-results/TEST-*.xml hudson.util.IOException2: hudson.util.IOException2: Failed to extract /home/jenkins/jenkins-slave/workspace/PreCommit-HIVE-Build/hive/build/test-results/TEST-*.xml at hudson.FilePath.readFromTar(FilePath.java:2050) at hudson.FilePath.copyRecursiveTo(FilePath.java:1962) at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:137) at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20) at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:782) at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:754) at hudson.model.Build$BuildExecution.post2(Build.java:183) at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:707) at hudson.model.Run.execute(Run.java:1628) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:246) Caused by: java.io.IOException at hudson.remoting.FastPipedInputStream.read(FastPipedInputStream.java:175) at hudson.util.HeadBufferingStream.read(HeadBufferingStream.java:61) at com.jcraft.jzlib.InflaterInputStream.fill(InflaterInputStream.java:175) at com.jcraft.jzlib.InflaterInputStream.read(InflaterInputStream.java:106) at org.apache.tools.tar.TarBuffer.readBlock(TarBuffer.java:257) at org.apache.tools.tar.TarBuffer.readRecord(TarBuffer.java:223) at hudson.org.apache.tools.tar.TarInputStream.read(TarInputStream.java:345) at java.io.FilterInputStream.read(FilterInputStream.java:107) at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1025) at org.apache.commons.io.IOUtils.copy(IOUtils.java:999) at hudson.util.IOUtils.copy(IOUtils.java:37) at hudson.FilePath.readFromTar(FilePath.java:2040) ... 11 more at hudson.FilePath.copyRecursiveTo(FilePath.java:1969) at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:137) at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20) at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:782) at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:754) at hudson.model.Build$BuildExecution.post2(Build.java:183) at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:707) at hudson.model.Run.execute(Run.java:1628) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:246) Caused by: java.util.concurrent.ExecutionException: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel at hudson.remoting.Request$1.get(Request.java:278) at hudson.remoting.Request$1.get(Request.java:210) at hudson.remoting.FutureAdapter.get(FutureAdapter.java:59) at hudson.FilePath.copyRecursiveTo(FilePath.java:1965) ... 10 more Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel at hudson.remoting.Request.abort(Request.java:299) at hudson.remoting.Channel.terminate(Channel.java:774) at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:69) Caused by: java.io.IOException: Unexpected termination of the channel at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:50) Caused by: java.io.EOFException at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2596) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1316) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370) at hudson.remoting.Command.readFrom(Command.java:92) at hudson.remoting.ClassicCommandTransport.read(ClassicCommandTransport.java:71) at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:48)
          Hide
          Jitendra Nath Pandey added a comment -

          Added review request on the reviewboard: https://reviews.apache.org/r/14130/

          is the plan to commit this patch or to merge the vectorization branch into trunk?

          I am fine with either approach. I have uploaded this patch for jenkins runs and to have all the changes in a single place. But, if change in by-laws takes too long, it might make sense to just get it committed as a patch.

          Show
          Jitendra Nath Pandey added a comment - Added review request on the reviewboard: https://reviews.apache.org/r/14130/ is the plan to commit this patch or to merge the vectorization branch into trunk? I am fine with either approach. I have uploaded this patch for jenkins runs and to have all the changes in a single place. But, if change in by-laws takes too long, it might make sense to just get it committed as a patch.
          Hide
          Carl Steinbach added a comment -

          Jitendra Nath Pandey Can you please post a review request on phabricator or reviewboard?

          Also, is the plan to commit this patch or to merge the vectorization branch into trunk?

          Show
          Carl Steinbach added a comment - Jitendra Nath Pandey Can you please post a review request on phabricator or reviewboard? Also, is the plan to commit this patch or to merge the vectorization branch into trunk?
          Jitendra Nath Pandey made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Jitendra Nath Pandey made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Jitendra Nath Pandey made changes -
          Attachment HIVE-5283.2.patch [ 12602928 ]
          Hide
          Jitendra Nath Pandey added a comment -

          The previous patch didn't apply cleanly, here is the updated patch.

          Show
          Jitendra Nath Pandey added a comment - The previous patch didn't apply cleanly, here is the updated patch.
          Hide
          Hive QA added a comment -

          Overall: -1 no tests executed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12602831/HIVE-5283.1.patch

          Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/713/testReport
          Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/713/console

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Tests failed with: NonZeroExitCodeException: Command 'bash /data/hive-ptest/working/scratch/source-prep.sh' failed with exit status 1 and output '+ [[ -n '' ]]
          + export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128'
          + ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128'
          + cd /data/hive-ptest/working/
          + tee /data/hive-ptest/logs/PreCommit-HIVE-Build-713/source-prep.txt
          + mkdir -p maven ivy
          + [[ svn = \s\v\n ]]
          + [[ -n '' ]]
          + [[ -d apache-svn-trunk-source ]]
          + [[ ! -d apache-svn-trunk-source/.svn ]]
          + [[ ! -d apache-svn-trunk-source ]]
          + cd apache-svn-trunk-source
          + svn revert -R .
          Reverted 'hcatalog/core/src/main/java/org/apache/hcatalog/cli/SemanticAnalysis/HCatSemanticAnalyzerBase.java'
          Reverted 'hcatalog/core/src/main/java/org/apache/hcatalog/cli/SemanticAnalysis/HCatSemanticAnalyzer.java'
          Reverted 'hcatalog/core/src/main/java/org/apache/hive/hcatalog/cli/SemanticAnalysis/HCatSemanticAnalyzerBase.java'
          Reverted 'hcatalog/core/src/main/java/org/apache/hive/hcatalog/cli/SemanticAnalysis/HCatSemanticAnalyzer.java'
          Reverted 'cli/src/test/org/apache/hadoop/hive/cli/TestCliSessionState.java'
          Reverted 'cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java'
          Reverted 'cli/src/java/org/apache/hadoop/hive/cli/CliSessionState.java'
          Reverted 'ql/src/test/org/apache/hadoop/hive/ql/metadata/TestHive.java'
          Reverted 'ql/src/test/org/apache/hadoop/hive/ql/exec/TestExecDriver.java'
          Reverted 'ql/src/test/org/apache/hadoop/hive/ql/history/TestHiveHistory.java'
          Reverted 'ql/src/test/org/apache/hadoop/hive/ql/parse/TestMacroSemanticAnalyzer.java'
          Reverted 'ql/src/test/org/apache/hadoop/hive/ql/QTestUtil.java'
          Reverted 'ql/src/java/org/apache/hadoop/hive/ql/parse/ImportSemanticAnalyzer.java'
          Reverted 'ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java'
          Reverted 'ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java'
          Reverted 'ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java'
          Reverted 'ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java'
          Reverted 'ql/src/java/org/apache/hadoop/hive/ql/exec/ColumnStatsTask.java'
          Reverted 'ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java'
          Reverted 'ql/src/java/org/apache/hadoop/hive/ql/Driver.java'
          ++ awk '{print $2}'
          ++ egrep -v '^X|^Performing status on external'
          ++ svn status --no-ignore
          + rm -rf build hcatalog/build hcatalog/core/build hcatalog/storage-handlers/hbase/build hcatalog/server-extensions/build hcatalog/webhcat/svr/build hcatalog/webhcat/java-client/build hcatalog/hcatalog-pig-adapter/build common/src/gen ql/src/test/org/apache/hadoop/hive/ql/session
          + svn update
          U    hbase-handler/src/test/results/positive/external_table_ppd.q.out
          U    hbase-handler/src/test/results/positive/hbase_binary_storage_queries.q.out
          U    hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
          U    hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableOutputFormat.java
          U    hcatalog/core/src/main/java/org/apache/hcatalog/mapreduce/HCatStorageHandler.java
          D    hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/HCatStorageHandler.java
          U    hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java
          U    hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/HCatRecordReader.java
          U    hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/HCatBaseInputFormat.java
          U    hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/HCatBaseOutputFormat.java
          U    hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/HCatOutputFormat.java
          U    hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/FileOutputFormatContainer.java
          U    hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/FileRecordWriterContainer.java
          U    hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/InitializeInput.java
          U    hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/PartInfo.java
          U    hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/DefaultRecordWriterContainer.java
          U    hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/FosterStorageHandler.java
          U    hcatalog/core/src/main/java/org/apache/hive/hcatalog/cli/SemanticAnalysis/CreateTableHook.java
          U    hcatalog/core/src/main/java/org/apache/hive/hcatalog/security/StorageDelegationAuthorizationProvider.java
          U    hcatalog/core/src/main/java/org/apache/hive/hcatalog/common/HCatUtil.java
          D    hcatalog/core/src/test/java/org/apache/hive/hcatalog/cli/DummyStorageHandler.java
          U    hcatalog/storage-handlers/hbase/pom.xml
          A    hcatalog/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHiveHBaseTableOutputFormat.java
          A    hcatalog/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHiveHBaseStorageHandler.java
          A    hcatalog/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestPigHBaseStorageHandler.java
          U    hcatalog/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/SkeletonHBaseTest.java
          U    hcatalog/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseInputFormat.java
          A    hcatalog/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHCatHBaseInputFormat.java
          U    hcatalog/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/snapshot/RevisionManager.java
          U    hcatalog/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/HBaseHCatStorageHandler.java
          U    hcatalog/build-support/ant/checkstyle.xml
          U    ql/src/java/org/apache/hadoop/hive/ql/io/HiveFileFormatUtils.java
          A    ql/src/java/org/apache/hadoop/hive/ql/io/HivePassThroughOutputFormat.java
          A    ql/src/java/org/apache/hadoop/hive/ql/io/HivePassThroughRecordWriter.java
          U    ql/src/java/org/apache/hadoop/hive/ql/plan/CreateTableDesc.java
          U    ql/src/java/org/apache/hadoop/hive/ql/plan/TableDesc.java
          U    ql/src/java/org/apache/hadoop/hive/ql/plan/PlanUtils.java
          U    ql/src/java/org/apache/hadoop/hive/ql/plan/PartitionDesc.java
          U    ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
          U    ql/src/java/org/apache/hadoop/hive/ql/metadata/Partition.java
          U    ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java
          
          Fetching external item into 'hcatalog/src/test/e2e/harness'
          Updated external to revision 1522681.
          
          Updated to revision 1522681.
          + patchCommandPath=/data/hive-ptest/working/scratch/smart-apply-patch.sh
          + patchFilePath=/data/hive-ptest/working/scratch/build.patch
          + [[ -f /data/hive-ptest/working/scratch/build.patch ]]
          + chmod +x /data/hive-ptest/working/scratch/smart-apply-patch.sh
          + /data/hive-ptest/working/scratch/smart-apply-patch.sh /data/hive-ptest/working/scratch/build.patch
          The patch does not appear to apply with p0 to p2
          + exit 1
          '
          

          This message is automatically generated.

          Show
          Hive QA added a comment - Overall : -1 no tests executed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12602831/HIVE-5283.1.patch Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/713/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/713/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Tests failed with: NonZeroExitCodeException: Command 'bash /data/hive-ptest/working/scratch/source-prep.sh' failed with exit status 1 and output '+ [[ -n '' ]] + export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128' + ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128' + cd /data/hive-ptest/working/ + tee /data/hive-ptest/logs/PreCommit-HIVE-Build-713/source-prep.txt + mkdir -p maven ivy + [[ svn = \s\v\n ]] + [[ -n '' ]] + [[ -d apache-svn-trunk-source ]] + [[ ! -d apache-svn-trunk-source/.svn ]] + [[ ! -d apache-svn-trunk-source ]] + cd apache-svn-trunk-source + svn revert -R . Reverted 'hcatalog/core/src/main/java/org/apache/hcatalog/cli/SemanticAnalysis/HCatSemanticAnalyzerBase.java' Reverted 'hcatalog/core/src/main/java/org/apache/hcatalog/cli/SemanticAnalysis/HCatSemanticAnalyzer.java' Reverted 'hcatalog/core/src/main/java/org/apache/hive/hcatalog/cli/SemanticAnalysis/HCatSemanticAnalyzerBase.java' Reverted 'hcatalog/core/src/main/java/org/apache/hive/hcatalog/cli/SemanticAnalysis/HCatSemanticAnalyzer.java' Reverted 'cli/src/test/org/apache/hadoop/hive/cli/TestCliSessionState.java' Reverted 'cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java' Reverted 'cli/src/java/org/apache/hadoop/hive/cli/CliSessionState.java' Reverted 'ql/src/test/org/apache/hadoop/hive/ql/metadata/TestHive.java' Reverted 'ql/src/test/org/apache/hadoop/hive/ql/exec/TestExecDriver.java' Reverted 'ql/src/test/org/apache/hadoop/hive/ql/history/TestHiveHistory.java' Reverted 'ql/src/test/org/apache/hadoop/hive/ql/parse/TestMacroSemanticAnalyzer.java' Reverted 'ql/src/test/org/apache/hadoop/hive/ql/QTestUtil.java' Reverted 'ql/src/java/org/apache/hadoop/hive/ql/parse/ImportSemanticAnalyzer.java' Reverted 'ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java' Reverted 'ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java' Reverted 'ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java' Reverted 'ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java' Reverted 'ql/src/java/org/apache/hadoop/hive/ql/exec/ColumnStatsTask.java' Reverted 'ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java' Reverted 'ql/src/java/org/apache/hadoop/hive/ql/Driver.java' ++ awk '{print $2}' ++ egrep -v '^X|^Performing status on external' ++ svn status --no-ignore + rm -rf build hcatalog/build hcatalog/core/build hcatalog/storage-handlers/hbase/build hcatalog/server-extensions/build hcatalog/webhcat/svr/build hcatalog/webhcat/java-client/build hcatalog/hcatalog-pig-adapter/build common/src/gen ql/src/test/org/apache/hadoop/hive/ql/session + svn update U hbase-handler/src/test/results/positive/external_table_ppd.q.out U hbase-handler/src/test/results/positive/hbase_binary_storage_queries.q.out U hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java U hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableOutputFormat.java U hcatalog/core/src/main/java/org/apache/hcatalog/mapreduce/HCatStorageHandler.java D hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/HCatStorageHandler.java U hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java U hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/HCatRecordReader.java U hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/HCatBaseInputFormat.java U hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/HCatBaseOutputFormat.java U hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/HCatOutputFormat.java U hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/FileOutputFormatContainer.java U hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/FileRecordWriterContainer.java U hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/InitializeInput.java U hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/PartInfo.java U hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/DefaultRecordWriterContainer.java U hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/FosterStorageHandler.java U hcatalog/core/src/main/java/org/apache/hive/hcatalog/cli/SemanticAnalysis/CreateTableHook.java U hcatalog/core/src/main/java/org/apache/hive/hcatalog/security/StorageDelegationAuthorizationProvider.java U hcatalog/core/src/main/java/org/apache/hive/hcatalog/common/HCatUtil.java D hcatalog/core/src/test/java/org/apache/hive/hcatalog/cli/DummyStorageHandler.java U hcatalog/storage-handlers/hbase/pom.xml A hcatalog/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHiveHBaseTableOutputFormat.java A hcatalog/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHiveHBaseStorageHandler.java A hcatalog/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestPigHBaseStorageHandler.java U hcatalog/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/SkeletonHBaseTest.java U hcatalog/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseInputFormat.java A hcatalog/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHCatHBaseInputFormat.java U hcatalog/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/snapshot/RevisionManager.java U hcatalog/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/HBaseHCatStorageHandler.java U hcatalog/build-support/ant/checkstyle.xml U ql/src/java/org/apache/hadoop/hive/ql/io/HiveFileFormatUtils.java A ql/src/java/org/apache/hadoop/hive/ql/io/HivePassThroughOutputFormat.java A ql/src/java/org/apache/hadoop/hive/ql/io/HivePassThroughRecordWriter.java U ql/src/java/org/apache/hadoop/hive/ql/plan/CreateTableDesc.java U ql/src/java/org/apache/hadoop/hive/ql/plan/TableDesc.java U ql/src/java/org/apache/hadoop/hive/ql/plan/PlanUtils.java U ql/src/java/org/apache/hadoop/hive/ql/plan/PartitionDesc.java U ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java U ql/src/java/org/apache/hadoop/hive/ql/metadata/Partition.java U ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java Fetching external item into 'hcatalog/src/test/e2e/harness' Updated external to revision 1522681. Updated to revision 1522681. + patchCommandPath=/data/hive-ptest/working/scratch/smart-apply-patch.sh + patchFilePath=/data/hive-ptest/working/scratch/build.patch + [[ -f /data/hive-ptest/working/scratch/build.patch ]] + chmod +x /data/hive-ptest/working/scratch/smart-apply-patch.sh + /data/hive-ptest/working/scratch/smart-apply-patch.sh /data/hive-ptest/working/scratch/build.patch The patch does not appear to apply with p0 to p2 + exit 1 ' This message is automatically generated.
          Jitendra Nath Pandey made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Jitendra Nath Pandey made changes -
          Field Original Value New Value
          Attachment HIVE-5283.1.patch [ 12602831 ]
          Jitendra Nath Pandey created issue -

            People

            • Assignee:
              Jitendra Nath Pandey
              Reporter:
              Jitendra Nath Pandey
            • Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development