Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3808

NPE in FileOutputCommitter when running a 0 reduce job

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.23.0, 2.0.0-alpha
    • Fix Version/s: 0.23.1
    • Component/s: mrv2
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Fixed an NPE in FileOutputCommitter for jobs with maps but no reduces.

      Description

      This was while running LoadGen.

      Error: java.lang.NullPointerException at org.apache.hadoop.fs.Path.<init>(Path.java:67) 
      at org.apache.hadoop.fs.Path.<init>(Path.java:56) 
      at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getPendingJobAttemptsPath(FileOutputCommitter.java:118) 
      at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getJobAttemptPath(FileOutputCommitter.java:167) 
      at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getJobAttemptPath(FileOutputCommitter.java:149) 
      at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getPendingTaskAttemptsPath(FileOutputCommitter.java:185) 
      at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:209) 
      at org.apache.hadoop.mapred.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:100) 
      at org.apache.hadoop.mapred.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:94) 
      at org.apache.hadoop.mapred.FileOutputCommitter.needsTaskCommit(FileOutputCommitter.java:176) 
      at org.apache.hadoop.mapred.OutputCommitter.needsTaskCommit(OutputCommitter.java:248) 
      at org.apache.hadoop.mapred.Task.isCommitRequired(Task.java:955) 
      at org.apache.hadoop.mapred.Task.done(Task.java:912) 
      at org.apache.hadoop.mapred.MapTask.run(MapTask.java:331) 
      at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:147) 
      at java.security.AccessController.doPrivileged(Native Method) 
      at javax.security.auth.Subject.doAs(Subject.java:396) 
      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1157) 
      at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:142)
      
      1. MR-3808.txt
        9 kB
        Robert Joseph Evans

        Issue Links

          Activity

            People

            • Assignee:
              Robert Joseph Evans
              Reporter:
              Siddharth Seth
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development