Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1550

UGI.doAs should not be used for getting the history file of jobs

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Invalid
    • Affects Version/s: 0.22.0
    • Fix Version/s: 0.22.0
    • Component/s: jobtracker
    • Labels:
      None

      Description

      When the jobtracker tries to open a job history file it does a doAs to get the filesystem for the user (that had submitted the job). This should not be done since the job history files are owned by the jobtracker.

      1. 1550.trunk.patch
        3 kB
        Ravi Gummadi
      2. 1550-2.1.patch
        0.8 kB
        Devaraj Das
      3. 1550-2.patch
        2 kB
        Devaraj Das
      4. 1550-1.patch
        0.9 kB
        Devaraj Das

        Issue Links

          Activity

          Hide
          Devaraj Das added a comment -

          This fixes the problem. The problem is that, in a secure env, the FileSystem.get() would require user's credentials to talk to the NameNode if the get() was done within a UGI.doAs. Removing the doAs would mean that the JobTracker's credentials would be used which is the right thing to do since the JobHistory files are owned by the JobTracker anyway..

          Show
          Devaraj Das added a comment - This fixes the problem. The problem is that, in a secure env, the FileSystem.get() would require user's credentials to talk to the NameNode if the get() was done within a UGI.doAs. Removing the doAs would mean that the JobTracker's credentials would be used which is the right thing to do since the JobHistory files are owned by the JobTracker anyway..
          Hide
          Devaraj Das added a comment -

          I should add that at the point where the ugi for the remote user is constructed, the JobTracker doesn't have the tokens to add to it. In contrast, during job submission (in the JobInProgress constructor), the remote user's credentials are available and the ugi created there is populated with it. That ugi is then used to talk to the namenode.

          Show
          Devaraj Das added a comment - I should add that at the point where the ugi for the remote user is constructed, the JobTracker doesn't have the tokens to add to it. In contrast, during job submission (in the JobInProgress constructor), the remote user's credentials are available and the ugi created there is populated with it. That ugi is then used to talk to the namenode.
          Hide
          Devaraj Das added a comment -

          This patch fixes a bug to do with getting the right user for the authorization check.

          Show
          Devaraj Das added a comment - This patch fixes a bug to do with getting the right user for the authorization check.
          Hide
          Devaraj Das added a comment -

          Fixes a bug in the earlier patch (assumes the earlier patch)

          Show
          Devaraj Das added a comment - Fixes a bug in the earlier patch (assumes the earlier patch)
          Hide
          Vinod Kumar Vavilapalli added a comment -

          This bug is there not on trunk but on the ydist branch for which patch was attached at MAPREDUCE-1455. Closing this as invalid.

          Show
          Vinod Kumar Vavilapalli added a comment - This bug is there not on trunk but on the ydist branch for which patch was attached at MAPREDUCE-1455 . Closing this as invalid.
          Hide
          Ravi Gummadi added a comment -

          ugi.doAs() is there JSPUtil.getJobInfo() in current trunk also. So this fix needs to go into trunk also.

          Show
          Ravi Gummadi added a comment - ugi.doAs() is there JSPUtil.getJobInfo() in current trunk also. So this fix needs to go into trunk also.
          Hide
          Ravi Gummadi added a comment -

          Attaching patch for trunk.
          This is forward port of merge of 1550-2.patch and 1550-2.1.patch to trunk. A small change needed to TestJobRetire because JSPUtil.getJobInfo() is used there.

          Show
          Ravi Gummadi added a comment - Attaching patch for trunk. This is forward port of merge of 1550-2.patch and 1550-2.1.patch to trunk. A small change needed to TestJobRetire because JSPUtil.getJobInfo() is used there.
          Hide
          Vinod Kumar Vavilapalli added a comment -

          Ravi and I sat together to see if it is really valid on trunk. And we reached the conclusion that it isn't after an elaborate analysis of the order of patches that went into trunk and yahoo distribution.

          Show
          Vinod Kumar Vavilapalli added a comment - Ravi and I sat together to see if it is really valid on trunk. And we reached the conclusion that it isn't after an elaborate analysis of the order of patches that went into trunk and yahoo distribution.

            People

            • Assignee:
              Devaraj Das
              Reporter:
              Devaraj Das
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development