[HADOOP-2174] distcp throws a NullPointerException in the close() method of mapper class due to the Reporter becoming invalid

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 0.15.0
• Fix Version/s: 0.15.1
• Component/s: util
• Assignee: Christopher Douglas
• Reporter: Riccardo Boscolo
• Created: 08/Nov/07 23:11
• Updated: 28/Nov/07 22:32
• Resolved: 10/Nov/07 07:56

Description

distcp occasionally throws a NullPointerException in the close() method of the mapper class, when the copy of the Reporter handle becomes invalid:

java.lang.NullPointerException
at org.apache.hadoop.util.CopyFiles$FSCopyFilesMapper.updateStatus(CopyFiles.java:242)
at org.apache.hadoop.util.CopyFiles$FSCopyFilesMapper.close(CopyFiles.java:402)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:53)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:192)
at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1760)

This can easily be fixed by checking whether the Reporter is null before updating the status. Also, the status could be updated once the invocation of copy(srcstat, dstpath, out, reporter); returns on line 373 of CopyFiles.java. Marking this as critical for a 0.15.1 release as Chris requested.