[MAPREDUCE-5224] JobTracker should allow the system directory to be in non-default FS

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1-win
• Fix Version/s: 1-win
• Component/s: jobtracker
• Target Version/s: 1-win
• Hadoop Flags: Reviewed
• Assignee: Douma Fang
• Reporter: Douma Fang
• Created: 08/May/13 18:42
• Updated: 16/Jun/13 04:18
• Resolved: 16/Jun/13 04:17

Description

JobTracker today expects the system directory to be in the default file system
if (fs == null) {
fs = mrOwner.doAs(new PrivilegedExceptionAction<FileSystem>() {
public FileSystem run() throws IOException {
return FileSystem.get(conf);
}});
}

...

public String getSystemDir()

{ Path sysDir = new Path(conf.get("mapred.system.dir", "/tmp/hadoop/mapred/system")); return fs.makeQualified(sysDir).toString(); }

In Cloud like Azure the default file system is set as ASV (Windows Azure Blob Storage), but we would still like the system directory to be in DFS. We should change JobTracker to allow that.

Issue Links

• is part of: HADOOP-8645 Stabilize branch-1-win
• relates to: MAPREDUCE-5277 Job history completed location cannot be on a file system other than default