Hive
  1. Hive
  2. HIVE-6403

uncorrelated subquery is failing with auto.convert.join=true

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.13.0
    • Component/s: Query Processor
    • Labels:
      None
    • Release Note:
      Hide
      also added to 0.13
      thanks Navis
      Show
      also added to 0.13 thanks Navis

      Description

      Fixing HIVE-5690, I've found query in subquery_multiinsert.q is not working with hive.auto.convert.join=true

      set hive.auto.convert.join=true;
      hive> explain
          > from src b 
          > INSERT OVERWRITE TABLE src_4 
          >   select * 
          >   where b.key in 
          >    (select a.key 
          >     from src a 
          >     where b.value = a.value and a.key > '9'
          >    ) 
          > INSERT OVERWRITE TABLE src_5 
          >   select *  
          >   where b.key not in  ( select key from src s1 where s1.key > '2') 
          >   order by key 
          > ;
      java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
      	at java.util.ArrayList.rangeCheck(ArrayList.java:635)
      	at java.util.ArrayList.get(ArrayList.java:411)
      	at org.apache.hadoop.hive.ql.optimizer.MapJoinProcessor.genMapJoinLocalWork(MapJoinProcessor.java:149)
      	at org.apache.hadoop.hive.ql.optimizer.MapJoinProcessor.genLocalWorkForMapJoin(MapJoinProcessor.java:256)
      	at org.apache.hadoop.hive.ql.optimizer.MapJoinProcessor.genMapJoinOpAndLocalWork(MapJoinProcessor.java:248)
      	at org.apache.hadoop.hive.ql.optimizer.physical.CommonJoinTaskDispatcher.convertTaskToMapJoinTask(CommonJoinTaskDispatcher.java:191)
      	at org.apache.hadoop.hive.ql.optimizer.physical.CommonJoinTaskDispatcher.processCurrentTask(CommonJoinTaskDispatcher.java:481)
      	at org.apache.hadoop.hive.ql.optimizer.physical.AbstractJoinTaskDispatcher.dispatch(AbstractJoinTaskDispatcher.java:182)
      	at org.apache.hadoop.hive.ql.lib.TaskGraphWalker.dispatch(TaskGraphWalker.java:111)
      	at org.apache.hadoop.hive.ql.lib.TaskGraphWalker.walk(TaskGraphWalker.java:194)
      	at org.apache.hadoop.hive.ql.lib.TaskGraphWalker.startWalking(TaskGraphWalker.java:139)
      	at org.apache.hadoop.hive.ql.optimizer.physical.CommonJoinResolver.resolve(CommonJoinResolver.java:79)
      	at org.apache.hadoop.hive.ql.optimizer.physical.PhysicalOptimizer.optimize(PhysicalOptimizer.java:100)
      	at org.apache.hadoop.hive.ql.parse.MapReduceCompiler.optimizeTaskPlan(MapReduceCompiler.java:290)
      	at org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:216)
      	at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:9167)
      	at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:327)
      	at org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.analyzeInternal(ExplainSemanticAnalyzer.java:64)
      	at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:327)
      	at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:446)
      	at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:346)
      	at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1056)
      	at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1099)
      	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:992)
      	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:982)
      	at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:268)
      	at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220)
      	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:424)
      	at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:793)
      	at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:687)
      	at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:626)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:606)
      	at org.apache.hadoop.util.RunJar.main(RunJar.java:160)
      org.apache.hadoop.hive.ql.parse.SemanticException: Failed to generate new mapJoin operator by exception : Index: 0, Size: 0
      	at org.apache.hadoop.hive.ql.optimizer.MapJoinProcessor.genLocalWorkForMapJoin(MapJoinProcessor.java:266)
      	at org.apache.hadoop.hive.ql.optimizer.MapJoinProcessor.genMapJoinOpAndLocalWork(MapJoinProcessor.java:248)
      	at org.apache.hadoop.hive.ql.optimizer.physical.CommonJoinTaskDispatcher.convertTaskToMapJoinTask(CommonJoinTaskDispatcher.java:191)
      	at org.apache.hadoop.hive.ql.optimizer.physical.CommonJoinTaskDispatcher.processCurrentTask(CommonJoinTaskDispatcher.java:481)
      	at org.apache.hadoop.hive.ql.optimizer.physical.AbstractJoinTaskDispatcher.dispatch(AbstractJoinTaskDispatcher.java:182)
      	at org.apache.hadoop.hive.ql.lib.TaskGraphWalker.dispatch(TaskGraphWalker.java:111)
      	at org.apache.hadoop.hive.ql.lib.TaskGraphWalker.walk(TaskGraphWalker.java:194)
      	at org.apache.hadoop.hive.ql.lib.TaskGraphWalker.startWalking(TaskGraphWalker.java:139)
      	at org.apache.hadoop.hive.ql.optimizer.physical.CommonJoinResolver.resolve(CommonJoinResolver.java:79)
      	at org.apache.hadoop.hive.ql.optimizer.physical.PhysicalOptimizer.optimize(PhysicalOptimizer.java:100)
      	at org.apache.hadoop.hive.ql.parse.MapReduceCompiler.optimizeTaskPlan(MapReduceCompiler.java:290)
      	at org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:216)
      	at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:9167)
      	at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:327)
      	at org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.analyzeInternal(ExplainSemanticAnalyzer.java:64)
      	at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:327)
      	at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:446)
      	at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:346)
      	at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1056)
      	at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1099)
      	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:992)
      	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:982)
      	at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:268)
      	at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220)
      	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:424)
      	at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:793)
      	at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:687)
      	at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:626)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:606)
      	at org.apache.hadoop.util.RunJar.main(RunJar.java:160)
      FAILED: SemanticException Generate Map Join Task Error: Failed to generate new mapJoin operator by exception : Index: 0, Size: 0
      
      1. HIVE-6403.1.patch
        24 kB
        Harish Butani
      2. HIVE-6403.2.patch
        26 kB
        Harish Butani
      3. HIVE-6403.3.patch.txt
        71 kB
        Navis
      4. HIVE-6403.4.patch.txt
        75 kB
        Navis
      5. HIVE-6403.5.patch.txt
        119 kB
        Navis
      6. HIVE-6403.6.patch.txt
        119 kB
        Navis
      7. navis.patch
        4 kB
        Navis
      8. navis2.patch
        97 kB
        Navis

        Issue Links

          Activity

            People

            • Assignee:
              Navis
              Reporter:
              Navis
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development