[MAPREDUCE-1744] DistributedCache creates its own FileSytem instance when adding a file/archive to the path

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.23.1, 1.0.0
• Hadoop Flags: Reviewed
• Assignee: Dick King
• Reporter: Dick King
• Created: 29/Apr/10 22:36
• Updated: 05/Mar/12 02:49
• Resolved: 04/Jan/12 18:15

Description

According to the contract of UserGroupInformation.doAs() the only required operations within the doAs() block are the
creation of a JobClient or getting a FileSystem .

The DistributedCache.add(File/Archive)ToClasspath() methods create a FileSystem instance outside of the doAs() block,
this FileSystem instance is not in the scope of the proxy user but of the superuser and permissions may make the method
fail.

One option is to overload the methods above to receive a filesystem.

Another option is to do obtain the FileSystem within a doAs() block, for this it would be required to have the proxy
user set in the passed configuration.

The second option seems nicer, but I don't know if the proxy user is as a property in the jobconf.

Issue Links

• duplicates: MAPREDUCE-1862 Distributed Cache doesn't accept paths from non-default filesystems
• is related to: MAPREDUCE-950 Clean up unsymmetrical Job-API w.r.t DistributedCache