Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-563 Security features for Map/Reduce
  3. MAPREDUCE-856

Localized files from DistributedCache should have right access-control

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.21.0
    • tasktracker
    • None
    • Reviewed
    • Hide
      Fixed TaskTracker and related classes so as to set correct and most restrictive access control for DistributedCache files/archives.
       - To do this, it changed the directory structure of per-job local files on a TaskTracker to the following:
      $mapred.local.dir
         `-- taskTracker
              `-- $user
                     |- distcache
                     `-- jobcache
       - Distributed cache files/archives are now user-owned by the job-owner and the group-owned by the special group-owner of the task-controller binary. The files/archives are set most private permissions possible, and as soon as possible, immediately after the files/dirs are first localized on the TT.
       - As depicted by the new directory structure, a directory corresponding to each user is created on each TT when that particular user's first task are assigned to the corresponding TT. These user directories remain on the TT forever are not cleaned when unused, which is targeted to be fixed via MAPREDUCE-1019.
       - The distributed cache files are now accessible _only_ by the user who first localized them. Sharing of these files across users is no longer possible, but is targeted for future versions via MAPREDUCE-744.
      Show
      Fixed TaskTracker and related classes so as to set correct and most restrictive access control for DistributedCache files/archives.  - To do this, it changed the directory structure of per-job local files on a TaskTracker to the following: $mapred.local.dir    `-- taskTracker         `-- $user                |- distcache                `-- jobcache  - Distributed cache files/archives are now user-owned by the job-owner and the group-owned by the special group-owner of the task-controller binary. The files/archives are set most private permissions possible, and as soon as possible, immediately after the files/dirs are first localized on the TT.  - As depicted by the new directory structure, a directory corresponding to each user is created on each TT when that particular user's first task are assigned to the corresponding TT. These user directories remain on the TT forever are not cleaned when unused, which is targeted to be fixed via MAPREDUCE-1019 .  - The distributed cache files are now accessible _only_ by the user who first localized them. Sharing of these files across users is no longer possible, but is targeted for future versions via MAPREDUCE-744 .

    Attachments

      1. MAPREDUCE-856-20090908-y20.txt
        151 kB
        Hemanth Yamijala
      2. MAPREDUCE-856-20090908.txt
        153 kB
        Vinod Kumar Vavilapalli
      3. MAPREDUCE-856-20090907.txt
        153 kB
        Vinod Kumar Vavilapalli
      4. MAPREDUCE-856-20090907.1.txt
        153 kB
        Vinod Kumar Vavilapalli
      5. MAPREDUCE-856-20090904.txt
        150 kB
        Vinod Kumar Vavilapalli
      6. MAPREDUCE-856-20090904.1.txt
        151 kB
        Vinod Kumar Vavilapalli
      7. MAPREDUCE-856-20090903.txt
        149 kB
        Vinod Kumar Vavilapalli
      8. MAPREDUCE-856-20090827.txt
        148 kB
        Vinod Kumar Vavilapalli
      9. MAPREDUCE-856-20090825.3.txt
        138 kB
        Vinod Kumar Vavilapalli
      10. MAPREDUCE-856-20090821.txt
        116 kB
        Vinod Kumar Vavilapalli
      11. MAPREDUCE-856-20090820.txt
        116 kB
        Vinod Kumar Vavilapalli

      Issue Links

        blocks

        Sub-task - The sub-task of the issue MAPREDUCE-744 Support in DistributedCache to share cache files with other users after HADOOP-4493

        • Major - Major loss of function.
        • Closed

        Bug - A problem which impairs or prevents the functions of the product. MAPREDUCE-890 After HADOOP-4491, the user who started mapred system is not able to run job.

        • Blocker - Blocks development and/or testing work, production could not run
        • Closed
        is blocked by

        Sub-task - The sub-task of the issue MAPREDUCE-871 Job/Task local files have incorrect group ownership set by LinuxTaskController binary

        • Major - Major loss of function.
        • Closed

        Improvement - An improvement or enhancement to an existing feature or task. MAPREDUCE-711 Move Distributed Cache from Common to Map/Reduce

        • Major - Major loss of function.
        • Closed

        Improvement - An improvement or enhancement to an existing feature or task. MAPREDUCE-476 extend DistributedCache to work locally (LocalJobRunner)

        • Minor - Minor loss of function, or other problem where easy workaround is present.
        • Closed
        is related to

        Sub-task - The sub-task of the issue HADOOP-4490 Map and Reduce tasks should run as the user who submitted the job

        • Major - Major loss of function.
        • Closed

        Activity

          People

            vinodkv Vinod Kumar Vavilapalli
            acmurthy Arun Murthy
            Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: