Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1744

DistributedCache creates its own FileSytem instance when adding a file/archive to the path

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.23.1, 1.0.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      According to the contract of UserGroupInformation.doAs() the only required operations within the doAs() block are the
      creation of a JobClient or getting a FileSystem .

      The DistributedCache.add(File/Archive)ToClasspath() methods create a FileSystem instance outside of the doAs() block,
      this FileSystem instance is not in the scope of the proxy user but of the superuser and permissions may make the method
      fail.

      One option is to overload the methods above to receive a filesystem.

      Another option is to do obtain the FileSystem within a doAs() block, for this it would be required to have the proxy
      user set in the passed configuration.

      The second option seems nicer, but I don't know if the proxy user is as a property in the jobconf.

      1. BZ-3503564--2010-05-06.patch
        14 kB
        Dick King
      2. MAPREDUCE-1744.patch
        7 kB
        Arun C Murthy
      3. h1744.patch
        7 kB
        Krishna Ramachandran
      4. mapred-1744.patch
        8 kB
        Krishna Ramachandran
      5. mapred-1744-1.patch
        7 kB
        Krishna Ramachandran
      6. mapred-1744-2.patch
        7 kB
        Krishna Ramachandran
      7. mapred-1744-3.patch
        7 kB
        Krishna Ramachandran
      8. MAPREDUCE-1744-0.23-trunk.patch
        6 kB
        Alejandro Abdelnur
      9. MAPREDUCE-1744-0.23-trunk.patch
        5 kB
        Alejandro Abdelnur

        Issue Links

          Activity

          Arun C Murthy made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Alejandro Abdelnur made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags Reviewed [ 10343 ]
          Fix Version/s 0.23.1 [ 12318883 ]
          Fix Version/s 1.0.0 [ 12318240 ]
          Resolution Fixed [ 1 ]
          Alejandro Abdelnur made changes -
          Assignee Alejandro Abdelnur [ tucu00 ] Dick King [ dking ]
          Alejandro Abdelnur made changes -
          Attachment MAPREDUCE-1744-0.23-trunk.patch [ 12509361 ]
          Alejandro Abdelnur made changes -
          Status Reopened [ 4 ] Patch Available [ 10002 ]
          Alejandro Abdelnur made changes -
          Attachment MAPREDUCE-1744-0.23-trunk.patch [ 12509328 ]
          Alejandro Abdelnur made changes -
          Resolution Invalid [ 6 ]
          Status Resolved [ 5 ] Reopened [ 4 ]
          Assignee Krishna Ramachandran [ ramach ] Alejandro Abdelnur [ tucu00 ]
          Tom White made changes -
          Link This issue is related to MAPREDUCE-950 [ MAPREDUCE-950 ]
          Krishna Ramachandran made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Resolution Invalid [ 6 ]
          Krishna Ramachandran made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Joydeep Sen Sarma made changes -
          Link This issue duplicates MAPREDUCE-1862 [ MAPREDUCE-1862 ]
          Krishna Ramachandran made changes -
          Attachment mapred-1744-3.patch [ 12446813 ]
          Arun C Murthy made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Krishna Ramachandran made changes -
          Attachment mapred-1744-2.patch [ 12446592 ]
          Krishna Ramachandran made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Krishna Ramachandran made changes -
          Attachment mapred-1744-1.patch [ 12446558 ]
          Chris Douglas made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Krishna Ramachandran made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Krishna Ramachandran made changes -
          Status In Progress [ 3 ] Open [ 1 ]
          Krishna Ramachandran made changes -
          Status Patch Available [ 10002 ] In Progress [ 3 ]
          Krishna Ramachandran made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Assignee Krishna Ramachandran [ ramach ]
          Dick King made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Krishna Ramachandran made changes -
          Attachment mapred-1744.patch [ 12445391 ]
          Chris Douglas made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Chris Douglas made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Krishna Ramachandran made changes -
          Attachment h1744.patch [ 12444582 ]
          Krishna Ramachandran made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Arun C Murthy made changes -
          Attachment MAPREDUCE-1744.patch [ 12444060 ]
          Dick King made changes -
          Field Original Value New Value
          Attachment BZ-3503564--2010-05-06.patch [ 12443902 ]
          Dick King created issue -

            People

            • Assignee:
              Dick King
              Reporter:
              Dick King
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development