Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-21600

GenTezUtils.removeSemiJoinOperator may throw out of bounds exception for TS with multiple children

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.1.2, 3.2.0, 4.0.0
    • Component/s: None
    • Labels:
      None

      Description

      The method does not reset the context when it loops through the children of TS. Hence, if TS has multiple FIL children, we can end up with a mangled context.

      Stacktrace:

      ERROR : FAILED: ArrayIndexOutOfBoundsException -1
      java.lang.ArrayIndexOutOfBoundsException: -1
      	at java.util.ArrayList.elementData(ArrayList.java:418)
      	at java.util.ArrayList.remove(ArrayList.java:495)
      	at org.apache.hadoop.hive.ql.parse.GenTezUtils.removeSemiJoinOperator(GenTezUtils.java:653)
      	at org.apache.hadoop.hive.ql.optimizer.SharedWorkOptimizer.sharedWorkOptimization(SharedWorkOptimizer.java:452)
      	at org.apache.hadoop.hive.ql.optimizer.SharedWorkOptimizer.transform(SharedWorkOptimizer.java:151)
      	at org.apache.hadoop.hive.ql.parse.TezCompiler.optimizeOperatorPlan(TezCompiler.java:186)
      	at org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:148)
      	at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12495)
      	at org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:362)
      	at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:289)
      	at org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.analyzeInternal(ExplainSemanticAnalyzer.java:172)
      	at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:289)
      	at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:664)
      	at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1870)
      	at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1817)
      	at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1812)
      

        Attachments

        1. HIVE-21600.patch
          1.0 kB
          Jesus Camacho Rodriguez
        2. HIVE-21600.patch
          1.0 kB
          Jesus Camacho Rodriguez

          Activity

            People

            • Assignee:
              jcamachorodriguez Jesus Camacho Rodriguez
              Reporter:
              jcamachorodriguez Jesus Camacho Rodriguez
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: