[MAPREDUCE-1435] symlinks in cwd of the task are not handled properly after MAPREDUCE-896

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.22.0
• Fix Version/s: 0.21.0
• Component/s: tasktracker
• Hadoop Flags: Reviewed
• Release Note: Fixes bugs in linux task controller and TaskRunner.setupWorkDir() related to handling of symlinks.
• Assignee: Ravi Gummadi
• Reporter: Amareshwari Sriramadasu
• Created: 01/Feb/10 09:34
• Updated: 24/Aug/10 21:20
• Resolved: 05/Mar/10 02:11

Description

With JVM reuse, TaskRunner.setupWorkDir() lists the contents of workDir and does a fs.delete on each path listed. If the listed file is a symlink to directory, it will delete the contents of those linked directories. This would delete files from distributed cache and jars directory,if mapred.create.symlink is true.
Changing ownership/permissions of symlinks through ENABLE_TASK_FOR_CLEANUP would change ownership/permissions of underlying files.

This is observed by Karam while running streaming jobs with DistributedCache and jvm reuse.

Issue Links

• is blocked by: HADOOP-6531 add FileUtil.fullyDeleteContents(dir) api to delete contents of a directory