Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-1901

ColumnPruner bug in case of LATERAL VIEW and UDTF

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      The following query

       EXPLAIN SELECT stat FROM (
      SELECT nt.value AS n, dt.value AS d_evt, nt.hr as hr
      FROM srcpart nt
      JOIN srcpart dt
      ON (nt.ds='2008-04-08' AND dt.ds='2008-04-08' AND nt.key=dt.key)
      ) joined
      LATERAL VIEW explode(array(n)) s AS stat;
      

      throws an exception:

      FAILED: Hive Internal Error: java.lang.ArrayIndexOutOfBoundsException(-1)
      java.lang.ArrayIndexOutOfBoundsException: -1
      at java.util.ArrayList.get(ArrayList.java:324)
      at org.apache.hadoop.hive.ql.optimizer.ColumnPrunerProcFactory$ColumnPrunerSelectProc.process(ColumnPrunerProcFactory.java:398)
      at org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:89)
      at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:88)
      at org.apache.hadoop.hive.ql.optimizer.ColumnPruner$ColumnPrunerWalker.walk(ColumnPruner.java:143)
      at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.startWalking(DefaultGraphWalker.java:102)
      at org.apache.hadoop.hive.ql.optimizer.ColumnPruner.transform(ColumnPruner.java:106)
      at org.apache.hadoop.hive.ql.optimizer.Optimizer.optimize(Optimizer.java:85)
      at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:6606)
      at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:238)
      at org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.analyzeInternal(ExplainSemanticAnalyzer.java:48)
      at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:238)
      at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:335)
      at org.apache.hadoop.hive.ql.Driver.run(Driver.java:686)
      at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:161)
      at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:235)
      at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:450)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      at java.lang.reflect.Method.invoke(Method.java:597)
      at org.apache.hadoop.util.RunJar.main(RunJar.java:156)

        Issue Links

          Activity

          Hide
          nzhang Ning Zhang added a comment -

          This bug is related to the unnecessary columns in the subquery (db.value and nt.hr). It works if removing these columns.

          Show
          nzhang Ning Zhang added a comment - This bug is related to the unnecessary columns in the subquery (db.value and nt.hr). It works if removing these columns.
          Hide
          nzhang Ning Zhang added a comment -

          Paul, can you take a look?

          Show
          nzhang Ning Zhang added a comment - Paul, can you take a look?

            People

            • Assignee:
              pauly Paul Yang
              Reporter:
              nzhang Ning Zhang
            • Votes:
              3 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:

                Development