[MAPREDUCE-1609] TaskTracker.localizeJob should not set permissions on job log directory recursively

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.22.0
• Fix Version/s: 0.21.0
• Component/s: tasktracker
• Hadoop Flags: Reviewed
• Release Note: Fixed TaskTracker so that it does not set permissions on job-log directory recursively. This fix both improves the performance of job localization as well as avoids a bug related to launching of task-cleanup attempts after TaskTracker's restart.
• Assignee: Amareshwari Sriramadasu
• Reporter: Amareshwari Sriramadasu
• Created: 19/Mar/10 03:52
• Updated: 24/Aug/10 21:21
• Resolved: 28/Apr/10 04:29

Description

Currently TaskTracker.localizeJob sets permissions (570 with LinuxTaskController) on job log directory recursively. When the tracker restarts/reinits, if there are new tasks for the job, localizeJob would find attempt log directories for the earlier tasks. If the job has many tasks, it would spend significant time in chmod'ing.

Also, if a cleanup attempt is launched after the restart/reinit, we would hit MAPREDUCE-1607.

This problem is missed by the patch for MAPREDUCE-927. The above problem never existed before MAPREDUCE:927.

Issue Links

• incorporates: MAPREDUCE-1595 LinuxTaskController is too strict on the initial ownership of files/dir.