Hive
  1. Hive
  2. HIVE-5846

Analyze command fails with vectorization on

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.13.0
    • Component/s: None
    • Labels:
      None

      Description

      analyze table alltypesorc compute statistics; fails

        Activity

        Hide
        Ashutosh Chauhan added a comment -

        Stack trace:

        Caused by: java.lang.ClassCastException: org.apache.hadoop.hive.ql.exec.vector.VectorizedRowBatch cannot be cast to [Ljava.lang.Object;
                at org.apache.hadoop.hive.serde2.objectinspector.UnionStructObjectInspector.getStructFieldData(UnionStructObjectInspector.java:127)
                at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.partialCopyToStandardObject(ObjectInspectorUtils.java:216)
                at org.apache.hadoop.hive.ql.exec.TableScanOperator.gatherStats(TableScanOperator.java:178)
                at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:89)
                at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:489)
                at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:827)
                at org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:43)
        
        Show
        Ashutosh Chauhan added a comment - Stack trace: Caused by: java.lang.ClassCastException: org.apache.hadoop.hive.ql.exec.vector.VectorizedRowBatch cannot be cast to [Ljava.lang. Object ; at org.apache.hadoop.hive.serde2.objectinspector.UnionStructObjectInspector.getStructFieldData(UnionStructObjectInspector.java:127) at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.partialCopyToStandardObject(ObjectInspectorUtils.java:216) at org.apache.hadoop.hive.ql.exec.TableScanOperator.gatherStats(TableScanOperator.java:178) at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:89) at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:489) at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:827) at org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:43)
        Hide
        Remus Rusanu added a comment -

        This fixes the problem by making stats gathering path not vectorized.
        If we feel enabling vectorized execution for stats gathering is important, we can revisit and modify TableScanOperator to handle it properly.

        Show
        Remus Rusanu added a comment - This fixes the problem by making stats gathering path not vectorized. If we feel enabling vectorized execution for stats gathering is important, we can revisit and modify TableScanOperator to handle it properly.
        Hide
        Ashutosh Chauhan added a comment -

        +1

        Show
        Ashutosh Chauhan added a comment - +1
        Hide
        Hive QA added a comment -

        Overall: +1 all checks pass

        Here are the results of testing the latest attachment:
        https://issues.apache.org/jira/secure/attachment/12614620/HIVE-5846.1.patch

        SUCCESS: +1 4617 tests passed

        Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/365/testReport
        Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/365/console

        Messages:

        Executing org.apache.hive.ptest.execution.PrepPhase
        Executing org.apache.hive.ptest.execution.ExecutionPhase
        Executing org.apache.hive.ptest.execution.ReportingPhase
        

        This message is automatically generated.

        ATTACHMENT ID: 12614620

        Show
        Hive QA added a comment - Overall : +1 all checks pass Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12614620/HIVE-5846.1.patch SUCCESS: +1 4617 tests passed Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/365/testReport Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/365/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase This message is automatically generated. ATTACHMENT ID: 12614620
        Hide
        Ashutosh Chauhan added a comment -

        Committed to trunk. Thanks, Remus!

        Show
        Ashutosh Chauhan added a comment - Committed to trunk. Thanks, Remus!

          People

          • Assignee:
            Remus Rusanu
            Reporter:
            Ashutosh Chauhan
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development