Hive
  1. Hive
  2. HIVE-1320

NPE with lineage in a query of union alls on joins.

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.6.0
    • Fix Version/s: 0.6.0
    • Component/s: Query Processor
    • Labels:
      None

      Description

      The following query generates a NPE in the lineage ctx code

      EXPLAIN
      INSERT OVERWRITE TABLE dest_l1
      SELECT j.*
      FROM (SELECT t1.key, p1.value
      FROM src1 t1
      LEFT OUTER JOIN src p1
      ON (t1.key = p1.key)
      UNION ALL
      SELECT t2.key, p2.value
      FROM src1 t2
      LEFT OUTER JOIN src p2
      ON (t2.key = p2.key)) j;

      The stack trace is:

      FAILED: Hive Internal Error: java.lang.NullPointerException(null)
      java.lang.NullPointerException
      at org.apache.hadoop.hive.ql.optimizer.lineage.LineageCtx$Index.mergeDependency(LineageCtx.java:116)
      at org.apache.hadoop.hive.ql.optimizer.lineage.OpProcFactory$UnionLineage.process(OpProcFactory.java:396)
      at org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:89)
      at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:88)
      at org.apache.hadoop.hive.ql.lib.PreOrderWalker.walk(PreOrderWalker.java:54)
      at org.apache.hadoop.hive.ql.lib.PreOrderWalker.walk(PreOrderWalker.java:59)
      at org.apache.hadoop.hive.ql.lib.PreOrderWalker.walk(PreOrderWalker.java:59)
      at org.apache.hadoop.hive.ql.lib.PreOrderWalker.walk(PreOrderWalker.java:59)
      at org.apache.hadoop.hive.ql.lib.PreOrderWalker.walk(PreOrderWalker.java:59)
      at org.apache.hadoop.hive.ql.lib.PreOrderWalker.walk(PreOrderWalker.java:59)
      at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.startWalking(DefaultGraphWalker.java:102)
      at org.apache.hadoop.hive.ql.optimizer.lineage.Generator.transform(Generator.java:72)
      at org.apache.hadoop.hive.ql.optimizer.Optimizer.optimize(Optimizer.java:83)
      at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:5976)
      at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:126)
      at org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.analyzeInternal(ExplainSemanticAnalyzer.java:48)
      at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:126)

      1. HIVE-1320.patch
        13 kB
        Ashish Thusoo

        Activity

        Hide
        Ning Zhang added a comment -

        Committed. Thanks Ashish!

        Show
        Ning Zhang added a comment - Committed. Thanks Ashish!
        Hide
        Ning Zhang added a comment -

        +1 will commit after tests.

        Show
        Ning Zhang added a comment - +1 will commit after tests.
        Hide
        Ashish Thusoo added a comment -

        Fixed the NPE. The cause was that we were not checking for inp_dep to be null in the union all code path. We have to do that for all operators that have more than 1 parents.

        Show
        Ashish Thusoo added a comment - Fixed the NPE. The cause was that we were not checking for inp_dep to be null in the union all code path. We have to do that for all operators that have more than 1 parents.

          People

          • Assignee:
            Ashish Thusoo
            Reporter:
            Ashish Thusoo
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development