Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-21600

GenTezUtils.removeSemiJoinOperator may throw out of bounds exception for TS with multiple children

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 3.1.2, 3.2.0, 4.0.0-alpha-1
    • None
    • None

    Description

      The method does not reset the context when it loops through the children of TS. Hence, if TS has multiple FIL children, we can end up with a mangled context.

      Stacktrace:

      ERROR : FAILED: ArrayIndexOutOfBoundsException -1
      java.lang.ArrayIndexOutOfBoundsException: -1
      	at java.util.ArrayList.elementData(ArrayList.java:418)
      	at java.util.ArrayList.remove(ArrayList.java:495)
      	at org.apache.hadoop.hive.ql.parse.GenTezUtils.removeSemiJoinOperator(GenTezUtils.java:653)
      	at org.apache.hadoop.hive.ql.optimizer.SharedWorkOptimizer.sharedWorkOptimization(SharedWorkOptimizer.java:452)
      	at org.apache.hadoop.hive.ql.optimizer.SharedWorkOptimizer.transform(SharedWorkOptimizer.java:151)
      	at org.apache.hadoop.hive.ql.parse.TezCompiler.optimizeOperatorPlan(TezCompiler.java:186)
      	at org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:148)
      	at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12495)
      	at org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:362)
      	at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:289)
      	at org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.analyzeInternal(ExplainSemanticAnalyzer.java:172)
      	at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:289)
      	at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:664)
      	at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1870)
      	at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1817)
      	at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1812)
      

      Attachments

        1. HIVE-21600.patch
          1.0 kB
          jcamachorodriguez
        2. HIVE-21600.patch
          1.0 kB
          jcamachorodriguez

        Activity

          People

            jcamacho Jesús Camacho Rodríguez
            jcamacho Jesús Camacho Rodríguez
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: