[HIVE-8431] Enable smb_mapjoin_11.q and smb_mapjoin_12.q [Spark Branch]

• Type: Test
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.1.0
• Assignee: Szehon Ho
• Reporter: Chao Sun
• Created: 11/Oct/14 00:02
• Updated: 29/May/15 02:28
• Resolved: 05/Jan/15 19:25

Description

Currently, even with HIVE-8412, smb_mapjoin_11.q and smb_mapjoin_12.q will fail with the following exception:

2014-10-10 17:00:55,496 ERROR [main]: ql.Driver (SessionState.java:printError(829)) - FAILED: ArrayIndexOutOfBoundsException 1
java.lang.ArrayIndexOutOfBoundsException: 1
  at org.apache.hadoop.hive.ql.metadata.Partition.getBucketPath(Partition.java:411)
  at org.apache.hadoop.hive.ql.optimizer.SamplePruner.prune(SamplePruner.java:199)
  at org.apache.hadoop.hive.ql.optimizer.GenMapRedUtils.setMapWork(GenMapRedUtils.java:601)
  at org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.setupMapWork(GenSparkUtils.java:188)
  at org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.createMapWork(GenSparkUtils.java:165)
  at org.apache.hadoop.hive.ql.parse.spark.GenSparkWork.process(GenSparkWork.java:116)
  at org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90)
  at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:94)
  at org.apache.hadoop.hive.ql.parse.spark.GenSparkWorkWalker.walk(GenSparkWorkWalker.java:89)
  at org.apache.hadoop.hive.ql.parse.spark.GenSparkWorkWalker.walk(GenSparkWorkWalker.java:105)
  at org.apache.hadoop.hive.ql.parse.spark.GenSparkWorkWalker.walk(GenSparkWorkWalker.java:105)
  at org.apache.hadoop.hive.ql.parse.spark.GenSparkWorkWalker.startWalking(GenSparkWorkWalker.java:71)
  at org.apache.hadoop.hive.ql.parse.spark.SparkCompiler.generateTaskTree(SparkCompiler.java:221)
  at org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:202)
  at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10028)
  at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:221)
  at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:415)
  at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:303)
  at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1068)
  at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1130)
  at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1005)
  at org.apache.hadoop.hive.ql.Driver.run(Driver.java:995)
  at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:246)
  at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:198)
  at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:408)
  at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:781)
  at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:675)
  at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:614)
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Method.java:606)                                                                                                                                                                                        

We should investigate this.

Issue Links

• depends upon: HIVE-8640 Support hints of SMBJoin [Spark Branch]
• Is contained by: HIVE-7292 Hive on Spark
• is related to: HIVE-8422 Turn on all join .q tests [Spark Branch]