[MAPREDUCE-1422] Changing permissions of files/dirs under job-work-dir may be needed sothat cleaning up of job-dir in all mapred-local-directories succeeds always

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.21.0
• Component/s: task-controller, tasktracker
• Hadoop Flags: Reviewed

Release Note

Introduced enableJobForCleanup() api in TaskController. This api enables deletion of stray files (with no write permissions for task-tracker) from job's work dir. Note that the behavior is similar to TaskController#enableTaskForCleanup() except the path on which the 'chmod' is done is the job's work dir.

• Assignee: Amar Kamat
• Reporter: Ravi Gummadi
• Created: 28/Jan/10 08:01
• Updated: 24/Aug/10 21:20
• Resolved: 10/Mar/10 05:27

Description

After MAPREDUCE-896, if LinuxTaskController is set in config, task-controller binary is launched for changing permissions of taskAttemptDir and taskWorkDir before cleaning up of these directories sothat cleanup will be succeeded even if user had created files/dirs under taskAttemptDir or taskWorkDir with non-writable permissions. Users can't create files/dirs under job-dir directly as we set 2570 for job-dir. But as job-work-dir has 2770 permissions and user can create files/dirs under job-work-dir with non-writable permissions, Changing permissions of files/dirs under job-work-dir may be needed sothat cleaning up of job-dir in all mapred-local-directories succeeds always.

Issue Links

• is depended upon by: MAPREDUCE-331 Make jobtracker resilient to memory issues
• is related to: MAPREDUCE-896 Users can set non-writable permissions on temporary files for TT and can abuse disk usage.