Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1744

DistributedCache creates its own FileSytem instance when adding a file/archive to the path

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.23.1, 1.0.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      According to the contract of UserGroupInformation.doAs() the only required operations within the doAs() block are the
      creation of a JobClient or getting a FileSystem .

      The DistributedCache.add(File/Archive)ToClasspath() methods create a FileSystem instance outside of the doAs() block,
      this FileSystem instance is not in the scope of the proxy user but of the superuser and permissions may make the method
      fail.

      One option is to overload the methods above to receive a filesystem.

      Another option is to do obtain the FileSystem within a doAs() block, for this it would be required to have the proxy
      user set in the passed configuration.

      The second option seems nicer, but I don't know if the proxy user is as a property in the jobconf.

        Attachments

        1. MAPREDUCE-1744-0.23-trunk.patch
          5 kB
          Alejandro Abdelnur
        2. MAPREDUCE-1744-0.23-trunk.patch
          6 kB
          Alejandro Abdelnur
        3. mapred-1744-3.patch
          7 kB
          Krishna Ramachandran
        4. mapred-1744-2.patch
          7 kB
          Krishna Ramachandran
        5. mapred-1744-1.patch
          7 kB
          Krishna Ramachandran
        6. mapred-1744.patch
          8 kB
          Krishna Ramachandran
        7. h1744.patch
          7 kB
          Krishna Ramachandran
        8. MAPREDUCE-1744.patch
          7 kB
          Arun C Murthy
        9. BZ-3503564--2010-05-06.patch
          14 kB
          Dick King

          Issue Links

            Activity

              People

              • Assignee:
                dking Dick King
                Reporter:
                dking Dick King
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: