Hadoop Common
  1. Hadoop Common
  2. HADOOP-3299

org.apache.hadoop.mapred.join.CompositeInputFormat does not initialize TextInput format files with the configuration resulting in an NullPointerException

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.16.3
    • Fix Version/s: 0.18.0
    • Component/s: io
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Changed the TextInputFormat and KeyValueTextInput classes to initialize the compressionCodecs member variable before dereferencing it.

      Description

      The input formats are not initialized with the Configuration object before the isSplitable method is called.

      bin/hadoop jar hadoop-0.16.3-examples.jar join -r 1 -inFormat org.apache.hadoop.mapred.KeyValueTextInputFormat -outFormat org.apache.hadoop.mapred.TextOutputFormat -joinOp outer datajoin/input datajoin/output -outKey org.apache.hadoop.io.Text -outValue org.apache.hadoop.mapred.join.TupleWritable
      08/04/22 15:05:33 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
      Job started: Tue Apr 22 15:05:33 GMT-08:00 2008
      08/04/22 15:05:33 INFO jvm.JvmMetrics: Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
      08/04/22 15:05:33 INFO mapred.FileInputFormat: Total input paths to process : 2
      java.lang.NullPointerException
      at org.apache.hadoop.mapred.KeyValueTextInputFormat.isSplitable(KeyValueTextInputFormat.java:44)
      at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:185)
      at org.apache.hadoop.mapred.join.Parser$WNode.getSplits(Parser.java:304)
      at org.apache.hadoop.mapred.join.Parser$CNode.getSplits(Parser.java:374)
      at org.apache.hadoop.mapred.join.CompositeInputFormat.getSplits(CompositeInputFormat.java:129)
      at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:542)
      at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:803)
      at org.apache.hadoop.examples.Join.run(Join.java:149)
      at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
      at org.apache.hadoop.examples.Join.main(Join.java:158)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      at java.lang.reflect.Method.invoke(Method.java:597)
      at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
      at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
      at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:52)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      at java.lang.reflect.Method.invoke(Method.java:597)
      at org.apache.hadoop.util.RunJar.main(RunJar.java:155)

      1. text.diff
        1 kB
        Jason
      2. keyvaluetext.diff
        1 kB
        Jason
      3. 3299-0.patch
        7 kB
        Chris Douglas

        Activity

        No work has yet been logged on this issue.

          People

          • Assignee:
            Chris Douglas
            Reporter:
            Jason
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development