[MAPREDUCE-5058] Job History files not copied

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.0.4
• Component/s: jobtracker
• Assignee: Unassigned
• Reporter: Aaron Zimmerman
• Created: 12/Mar/13 17:01
• Updated: 12/Mar/13 17:01

Description

When a job completes, instead of moving the job history files to the DONE folder, an error is thrown:
2013-03-12 15:03:52,111 ERROR org.apache.hadoop.mapred.JobHistory: Unable to move history file to DONE canonical subfolder.
java.io.FileNotFoundException: File </jobhistory/.....> does not exist.

I've confirmed that this file does exist, and can be pulled out of HDFS with hadoop fs -get.

I see this issue reported and resolved on other releases, such as: https://issues.apache.org/jira/browse/MAPREDUCE-2463

I believe the issue is that the move always tries to copy the folder from the local file system, regardless of whether or not the hadoop.job.history.location is in hdfs. This is in moveToDone() of org/apache/hadoop/mapred/JobHistory.java.