[MAPREDUCE-5679] TestJobHistoryParsing has race condition

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.2.0
• Fix Version/s: 2.3.0
• Hadoop Flags: Reviewed
• Assignee: Liyin Liang
• Reporter: Liyin Liang
• Created: 12/Dec/13 03:12
• Updated: 03/Sep/14 23:35
• Resolved: 17/Dec/13 16:51

Description

org.apache.hadoop.mapreduce.v2.hs.TestJobHistoryParsing can fail because of race condition.

testHistoryParsingWithParseErrors(org.apache.hadoop.mapreduce.v2.hs.TestJobHistoryParsing)  Time elapsed: 4.102 sec  <<< ERROR!
java.io.IOException: Unable to initialize History Viewer
        at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:520)
        at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:398)
        at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.<init>(ChecksumFileSystem.java:137)
        at org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:339)
        at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:798)
        at org.apache.hadoop.mapreduce.jobhistory.JobHistoryParser.<init>(JobHistoryParser.java:86)
        at org.apache.hadoop.mapreduce.jobhistory.HistoryViewer.<init>(HistoryViewer.java:85)
        at org.apache.hadoop.mapreduce.v2.hs.TestJobHistoryParsing.checkHistoryParsing(TestJobHistoryParsing.java:339)
        at org.apache.hadoop.mapreduce.v2.hs.TestJobHistoryParsing.testHistoryParsingWithParseErrors(TestJobHistoryParsing.java:125)

In the checkHistoryParsing() function, after

HistoryFileInfo fileInfo = jobHistory.getJobFileInfo(jobId);

a thread named MoveIntermediateToDone will be launched to move history file from done_intermediate to done directory.
If the history file is moved,

      HistoryViewer viewer = new HistoryViewer(fc.makeQualified(
          fileInfo.getHistoryFile()).toString(), conf, true);

will throw IOException，because the history file is not found.