[HIVE-2778] Fail on table sampling

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.9.0
• Fix Version/s: 0.9.0
• Component/s: Query Processor

Environment

Reproduced only on hadoop-0.20.2-CDH3u1, work fine on hadoop-0.20.2

• Hadoop Flags: Reviewed
• Assignee: Navis Ryu
• Reporter: Navis Ryu
• Created: 06/Feb/12 02:36
• Updated: 02/May/13 02:30
• Resolved: 16/Mar/12 01:50

Description

Trying table sampling on any non-empty table throws NPE. This does not occur by test on mini-MR.

select count(*) from emp tablesample (0.1 percent);     
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
java.lang.NullPointerException
	at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.sampleSplits(CombineHiveInputFormat.java:450)
	at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getSplits(CombineHiveInputFormat.java:403)
	at org.apache.hadoop.mapred.JobClient.writeOldSplits(JobClient.java:971)
	at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:963)
	at org.apache.hadoop.mapred.JobClient.access$500(JobClient.java:170)
	at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:880)
	at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:833)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:396)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
	at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:833)
	at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:807)
	at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:432)
	at org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:136)
	at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:134)
	at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
	at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1332)
	at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1123)
	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:931)
	at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:255)
	at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:212)
	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
	at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:671)
	at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:554)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
Job Submission failed with exception 'java.lang.NullPointerException(null)'
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MapRedTask

Issue Links

• is duplicated by: HIVE-2784 Integrating with MapReduce2 get NPE throwed when executing a query with a "TABLESAMPLE(x percent)" clause
• is related to: HIVE-3257 Fix avro_joins.q testcase failure when building hive on hadoop0.23
• relates to: HIVE-2737 CombineFileInputFormat fails if mapred.job.tracker is set to local with a sub-query