[HADOOP-3299] org.apache.hadoop.mapred.join.CompositeInputFormat does not initialize TextInput format files with the configuration resulting in an NullPointerException

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.16.3
• Fix Version/s: 0.18.0
• Component/s: io
• Hadoop Flags: Reviewed
• Release Note: Changed the TextInputFormat and KeyValueTextInput classes to initialize the compressionCodecs member variable before dereferencing it.
• Assignee: Christopher Douglas
• Reporter: Jason
• Created: 22/Apr/08 23:45
• Updated: 08/Jul/09 16:52
• Resolved: 07/May/08 22:36

Description

The input formats are not initialized with the Configuration object before the isSplitable method is called.

bin/hadoop jar hadoop-0.16.3-examples.jar join -r 1 -inFormat org.apache.hadoop.mapred.KeyValueTextInputFormat -outFormat org.apache.hadoop.mapred.TextOutputFormat -joinOp outer datajoin/input datajoin/output -outKey org.apache.hadoop.io.Text -outValue org.apache.hadoop.mapred.join.TupleWritable
08/04/22 15:05:33 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
Job started: Tue Apr 22 15:05:33 GMT-08:00 2008
08/04/22 15:05:33 INFO jvm.JvmMetrics: Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
08/04/22 15:05:33 INFO mapred.FileInputFormat: Total input paths to process : 2
java.lang.NullPointerException
at org.apache.hadoop.mapred.KeyValueTextInputFormat.isSplitable(KeyValueTextInputFormat.java:44)
at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:185)
at org.apache.hadoop.mapred.join.Parser$WNode.getSplits(Parser.java:304)
at org.apache.hadoop.mapred.join.Parser$CNode.getSplits(Parser.java:374)
at org.apache.hadoop.mapred.join.CompositeInputFormat.getSplits(CompositeInputFormat.java:129)
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:542)
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:803)
at org.apache.hadoop.examples.Join.run(Join.java:149)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.examples.Join.main(Join.java:158)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:52)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:155)