Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1435

symlinks in cwd of the task are not handled properly after MAPREDUCE-896

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.22.0
    • Fix Version/s: 0.21.0
    • Component/s: tasktracker
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Fixes bugs in linux task controller and TaskRunner.setupWorkDir() related to handling of symlinks.

      Description

      With JVM reuse, TaskRunner.setupWorkDir() lists the contents of workDir and does a fs.delete on each path listed. If the listed file is a symlink to directory, it will delete the contents of those linked directories. This would delete files from distributed cache and jars directory,if mapred.create.symlink is true.
      Changing ownership/permissions of symlinks through ENABLE_TASK_FOR_CLEANUP would change ownership/permissions of underlying files.

      This is observed by Karam while running streaming jobs with DistributedCache and jvm reuse.

        Attachments

        1. MR-1435-y20s-1.txt
          21 kB
          Hemanth Yamijala
        2. MR-1435-y20s.patch
          13 kB
          Hemanth Yamijala
        3. 1435.v4.patch
          27 kB
          Hemanth Yamijala
        4. 1435.v3.patch
          25 kB
          Hemanth Yamijala
        5. 1435.v2.patch
          19 kB
          Ravi Gummadi
        6. 1435.v1.patch
          12 kB
          Ravi Gummadi
        7. 1435.patch
          7 kB
          Ravi Gummadi

          Issue Links

            Activity

              People

              • Assignee:
                ravidotg Ravi Gummadi
                Reporter:
                amareshwari Amareshwari Sriramadasu
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: