Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1609

TaskTracker.localizeJob should not set permissions on job log directory recursively

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.22.0
    • Fix Version/s: 0.21.0
    • Component/s: tasktracker
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Fixed TaskTracker so that it does not set permissions on job-log directory recursively. This fix both improves the performance of job localization as well as avoids a bug related to launching of task-cleanup attempts after TaskTracker's restart.

      Description

      Currently TaskTracker.localizeJob sets permissions (570 with LinuxTaskController) on job log directory recursively. When the tracker restarts/reinits, if there are new tasks for the job, localizeJob would find attempt log directories for the earlier tasks. If the job has many tasks, it would spend significant time in chmod'ing.

      Also, if a cleanup attempt is launched after the restart/reinit, we would hit MAPREDUCE-1607.

      This problem is missed by the patch for MAPREDUCE-927. The above problem never existed before MAPREDUCE:927.

        Attachments

        1. patch-1609-1.txt
          18 kB
          Amareshwari Sriramadasu
        2. patch-1609.txt
          18 kB
          Amareshwari Sriramadasu
        3. MAPREDUCE-1609-20-1.patch
          21 kB
          Hemanth Yamijala

          Issue Links

            Activity

              People

              • Assignee:
                amareshwari Amareshwari Sriramadasu
                Reporter:
                amareshwari Amareshwari Sriramadasu
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: