Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1435

symlinks in cwd of the task are not handled properly after MAPREDUCE-896

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.22.0
    • Fix Version/s: 0.21.0
    • Component/s: tasktracker
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Fixes bugs in linux task controller and TaskRunner.setupWorkDir() related to handling of symlinks.

      Description

      With JVM reuse, TaskRunner.setupWorkDir() lists the contents of workDir and does a fs.delete on each path listed. If the listed file is a symlink to directory, it will delete the contents of those linked directories. This would delete files from distributed cache and jars directory,if mapred.create.symlink is true.
      Changing ownership/permissions of symlinks through ENABLE_TASK_FOR_CLEANUP would change ownership/permissions of underlying files.

      This is observed by Karam while running streaming jobs with DistributedCache and jvm reuse.

      1. MR-1435-y20s-1.txt
        21 kB
        Hemanth Yamijala
      2. MR-1435-y20s.patch
        13 kB
        Hemanth Yamijala
      3. 1435.v4.patch
        27 kB
        Hemanth Yamijala
      4. 1435.v3.patch
        25 kB
        Hemanth Yamijala
      5. 1435.v2.patch
        19 kB
        Ravi Gummadi
      6. 1435.v1.patch
        12 kB
        Ravi Gummadi
      7. 1435.patch
        7 kB
        Ravi Gummadi

        Issue Links

          Activity

          No work has yet been logged on this issue.

            People

            • Assignee:
              Ravi Gummadi
              Reporter:
              Amareshwari Sriramadasu
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development