[MAPREDUCE-5359] JobHistory should not use File.separator to match timestamp in path

• Type: Bug
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.1.0-beta, 3.0.0-alpha1
• Fix Version/s: 2.1.0-beta
• Target Version/s: 2.1.0-beta
• Hadoop Flags: Reviewed
• Assignee: Chuan Liu
• Reporter: Chuan Liu
• Created: 28/Jun/13 06:08
• Updated: 12/May/16 18:22
• Resolved: 03/Jul/13 00:08

Description

In HistoryFileManager.getTimestampPartFromPath() method, we use the following regular expression to match the timestamp in a Path object.

"\\d{4}" + "\\" + File.separator +  "\\d{2}" + "\\" + File.separator + "\\d{2}"

This is incorrect because Path uses backslash even for Windows path while File.separator is platform dependent, and is a forward slash on Windows.

This leads to failure matching the timestamp on Windows. One consequence is that addDirectoryToSerialNumberIndex() also failed. Later, getFileInfo() will fail if the job info is not in cache or intermediate directory.

The test case TestJobHistoryParsing.testScanningOldDirs() tests exactly the above scenario and fails on Windows.

Issue Links

• is part of: MAPREDUCE-4401 Enhancements to MapReduce for Windows Server and Windows Azure development and runtime environments