Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-5058

Job History files not copied

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 1.0.4
    • Fix Version/s: None
    • Component/s: jobtracker
    • Labels:
      None

      Description

      When a job completes, instead of moving the job history files to the DONE folder, an error is thrown:
      2013-03-12 15:03:52,111 ERROR org.apache.hadoop.mapred.JobHistory: Unable to move history file to DONE canonical subfolder.
      java.io.FileNotFoundException: File </jobhistory/.....> does not exist.

      I've confirmed that this file does exist, and can be pulled out of HDFS with hadoop fs -get.

      I see this issue reported and resolved on other releases, such as: https://issues.apache.org/jira/browse/MAPREDUCE-2463

      I believe the issue is that the move always tries to copy the folder from the local file system, regardless of whether or not the hadoop.job.history.location is in hdfs. This is in moveToDone() of org/apache/hadoop/mapred/JobHistory.java.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              az Aaron Zimmerman
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: