Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-14006

Hive query with UNION ALL fails with ArrayIndexOutOfBoundsException

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.0
    • Fix Version/s: 2.1.0
    • Component/s: Hive
    • Labels:
      None

      Description

      set hive.cbo.enable=false;
      DROP VIEW IF EXISTS a_view;

      DROP TABLE IF EXISTS table_a1;
      DROP TABLE IF EXISTS table_a2;
      DROP TABLE IF EXISTS table_b1;
      DROP TABLE IF EXISTS table_b2;

      CREATE TABLE table_a1
      (composite_key STRING);

      CREATE TABLE table_a2
      (composite_key STRING);

      CREATE TABLE table_b1
      (composite_key STRING, col1 STRING);

      CREATE TABLE table_b2
      (composite_key STRING);

      CREATE VIEW a_view AS
      SELECT
      substring(a1.composite_key, 1, locate('|',a1.composite_key) - 1) AS autoname,
      NULL AS col1
      FROM table_a1 a1
      FULL OUTER JOIN table_a2 a2
      ON a1.composite_key = a2.composite_key
      UNION ALL
      SELECT
      substring(b1.composite_key, 1, locate('|',b1.composite_key) - 1) AS autoname,
      b1.col1 AS col1
      FROM table_b1 b1
      FULL OUTER JOIN table_b2 b2
      ON b1.composite_key = b2.composite_key;

      INSERT INTO TABLE table_b1
      SELECT * FROM (
      SELECT 'something|awful', 'col1'
      )s ;

      SELECT autoname
      FROM a_view
      WHERE autoname='something';

      fails with
      Diagnostic Messages for this Task:
      Error: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row

      {"_col0":"something"}

      at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:179)
      at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
      at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
      at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
      at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
      at java.security.AccessController.doPrivileged(Native Method)
      at javax.security.auth.Subject.doAs(Subject.java:415)
      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
      at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
      Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row

      {"_col0":"something"}

      at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:507)
      at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:170)
      ... 8 more
      Caused by: java.lang.ArrayIndexOutOfBoundsException: 0
      at org.apache.hadoop.hive.ql.exec.UnionOperator.processOp(UnionOperator.java:134)
      at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:815)
      at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:95)
      at org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.forward(MapOperator.java:157)
      at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:497)

      The same query succeeds when hive.ppd.remove.duplicatefilters=false with or without CBO on. It also succeeds with just CBO on.

        Attachments

        1. HIVE-14006.patch
          0.8 kB
          Naveen Gangam
        2. HIVE-14006.1.patch
          7 kB
          Naveen Gangam

          Activity

            People

            • Assignee:
              ngangam Naveen Gangam
              Reporter:
              ngangam Naveen Gangam
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: