[MAPREDUCE-4246] Failure in deleting user directories in Secure hadoop

• Type: Task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.22.0
• Fix Version/s: 0.22.1
• Component/s: security
• Target Version/s: 0.22.1
• Hadoop Flags: Reviewed
• Assignee: Benoy Antony
• Reporter: Benoy Antony
• Created: 10/May/12 01:42
• Updated: 05/Jun/12 13:53
• Resolved: 05/Jun/12 07:41

Description

This happens when security is enabled on 22

When TaskTracker starts p, it invokes MRAsyncDiskService moves the contents of scratch/tasktracker to toBeDeleted under a directory created with the current timestamp.
The owner of this directory is hadoop (owner of tasktracker)

The contents of this directory are various usernames and they are owned by individual users. But MRAsyncDiskService tries to delete the newly created directory as hadoop and it fails.

2012-05-01 16:04:58,805 DEBUG org.apache.hadoop.mapred.LinuxTaskController: deleteAsUser: [/apache/hadoop-assemble-argon-0.228/hadoop-0.22-argon-0.105/bin/../bin/task-controller, hadoop, 3, /hadoop00/scratch/toBeDeleted/2012-04-17_16-22-03.965_0]
2012-05-01 16:04:58,809 WARN org.apache.hadoop.mapreduce.util.MRAsyncDiskService: Failure in deletion of toBeDeleted/2012-04-17_16-22-03.965_0 on /hadoop00/scratch with original name /hadoop00/scratch/toBeDeleted/2012-04-17_16-22-03.965_0 with exception org.apache.hadoop.util.Shell$ExitCodeException:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:256)
at org.apache.hadoop.util.Shell.run(Shell.java:183)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:376)
at org.apache.hadoop.mapred.LinuxTaskController.deleteAsUser(LinuxTaskController.java:273)
at org.apache.hadoop.mapreduce.util.MRAsyncDiskService$DeleteTask.run(MRAsyncDiskService.java:237)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)

Issue Links

• is depended upon by: HADOOP-8357 Restore security in Hadoop 0.22 branch