Hadoop Common
  1. Hadoop Common
  2. HADOOP-7101

UserGroupInformation.getCurrentUser() fails when called from non-Hadoop JAAS context

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.22.0
    • Fix Version/s: 1.2.0, 0.23.0
    • Component/s: security
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      If a Hadoop client is run from inside a container like Tomcat, and the current AccessControlContext has a Subject associated with it that is not created by Hadoop, then UserGroupInformation.getCurrentUser() will throw NoSuchElementException, since it assumes that any Subject will have a hadoop User principal.

      1. hadoop-7101.branch-1.patch
        2 kB
        Suresh Srinivas
      2. hadoop-7101.txt
        3 kB
        Todd Lipcon

        Issue Links

          Activity

          Hide
          Suresh Srinivas added a comment -

          OK, but current maven hadoop version (1.1.2)

          Yes. The change is not in 1.1.2. It is in next minor release 1.2.0.

          Show
          Suresh Srinivas added a comment - OK, but current maven hadoop version (1.1.2) Yes. The change is not in 1.1.2. It is in next minor release 1.2.0.
          Hide
          Mikhail Baturov added a comment -

          0.23.x - OK, but current maven hadoop version (1.1.2) still has this bug, I got it in OSGI with hbase + hadoop-core.

          Show
          Mikhail Baturov added a comment - 0.23.x - OK, but current maven hadoop version (1.1.2) still has this bug, I got it in OSGI with hbase + hadoop-core.
          Hide
          Suresh Srinivas added a comment -

          I committed the patch to branch-1 and branch-1.2.

          Show
          Suresh Srinivas added a comment - I committed the patch to branch-1 and branch-1.2.
          Hide
          Suresh Srinivas added a comment -

          Changing the fix version from 0.22 to 0.23. I also changed CHANGES.txt in the current branches of development to reflect this.

          Show
          Suresh Srinivas added a comment - Changing the fix version from 0.22 to 0.23. I also changed CHANGES.txt in the current branches of development to reflect this.
          Hide
          Matthew Farrellee added a comment -

          Suresh Srinivas

          Re CLOSED - ok, my background w/ workflows is that once CLOSED leave closed, open a new issue

          Re version - I've only been able to find the fix for HADOOP-7101 on the 0.23.* and 2.0.* release tags. That may mean this issue was not actually fixed in 0.22.0. However, I'm not familiar enough with the version streams to say for sure.

          Show
          Matthew Farrellee added a comment - Suresh Srinivas Re CLOSED - ok, my background w/ workflows is that once CLOSED leave closed, open a new issue Re version - I've only been able to find the fix for HADOOP-7101 on the 0.23.* and 2.0.* release tags. That may mean this issue was not actually fixed in 0.22.0. However, I'm not familiar enough with the version streams to say for sure.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          +1 the branch-1 patch looks good.

          Show
          Tsz Wo Nicholas Sze added a comment - +1 the branch-1 patch looks good.
          Hide
          Suresh Srinivas added a comment -

          Here is a branch-1 patch for this issue.

          Show
          Suresh Srinivas added a comment - Here is a branch-1 patch for this issue.
          Hide
          Suresh Srinivas added a comment -

          Let's continue the discussion there, instead of on this CLOSED issue.

          Sure. Generally if the port is straightforward, this issue could also be opened to attach the branch-1 patch.

          Show
          Suresh Srinivas added a comment - Let's continue the discussion there, instead of on this CLOSED issue. Sure. Generally if the port is straightforward, this issue could also be opened to attach the branch-1 patch.
          Hide
          Matthew Farrellee added a comment -

          I've posted a port of HADOOP-7101 over on HADOOP-9280. Let's continue the discussion there, instead of on this CLOSED issue.

          Show
          Matthew Farrellee added a comment - I've posted a port of HADOOP-7101 over on HADOOP-9280 . Let's continue the discussion there, instead of on this CLOSED issue.
          Hide
          Torsten Mielke added a comment -

          Filling in for Claus Ibsen here as well:

          Can you tell me the branch name and commit revision for this?

          The latest 0.23.5 release has this fix included.
          However we did not find any 1.x release that contains this code fix. See my comment from 01/Feb/13.
          I checked the source code of all relevant 1.x releases and did not see the fix.
          The fix is present in 2.0 branch however there has not been any release of 2.0 yet.

          So trying to use the Hadoop client libs of any 1.x version when running inside another container (e.g. OSGi) fails.

          Show
          Torsten Mielke added a comment - Filling in for Claus Ibsen here as well: Can you tell me the branch name and commit revision for this? The latest 0.23.5 release has this fix included. However we did not find any 1.x release that contains this code fix. See my comment from 01/Feb/13. I checked the source code of all relevant 1.x releases and did not see the fix. The fix is present in 2.0 branch however there has not been any release of 2.0 yet. So trying to use the Hadoop client libs of any 1.x version when running inside another container (e.g. OSGi) fails.
          Hide
          Suresh Srinivas added a comment -

          Claus, when you say 0.2, I am thrown off. 0.2 is the release from probably 6 years ago. Do you mean 0.20?

          eg the fix only went into the 0.2 code branch

          Can you tell me the branch name and commit revision for this?

          Show
          Suresh Srinivas added a comment - Claus, when you say 0.2, I am thrown off. 0.2 is the release from probably 6 years ago. Do you mean 0.20? eg the fix only went into the 0.2 code branch Can you tell me the branch name and commit revision for this?
          Hide
          Claus Ibsen added a comment -

          I mean when people uses Hadoop 0.2 and it works. Then if they upgrade to Hadoop 1.x and it no longer works.
          eg the fix only went into the 0.2 code branch, and not the 1.x code branch.

          Show
          Claus Ibsen added a comment - I mean when people uses Hadoop 0.2 and it works. Then if they upgrade to Hadoop 1.x and it no longer works. eg the fix only went into the 0.2 code branch, and not the 1.x code branch.
          Hide
          Suresh Srinivas added a comment -

          I logged a new ticket HADOOP-9280 as this is critical problem for people trying to upgrade from 0.2 to 1.x

          What do you mean by 0.2 to 1.x?

          Show
          Suresh Srinivas added a comment - I logged a new ticket HADOOP-9280 as this is critical problem for people trying to upgrade from 0.2 to 1.x What do you mean by 0.2 to 1.x?
          Hide
          Claus Ibsen added a comment -

          I logged a new ticket HADOOP-9280 as this is critical problem for people trying to upgrade from 0.2 to 1.x

          Show
          Claus Ibsen added a comment - I logged a new ticket HADOOP-9280 as this is critical problem for people trying to upgrade from 0.2 to 1.x
          Hide
          Torsten Mielke added a comment -

          Any idea why this bug got never fixed on any 1.x release?
          When running inside an OSGi container one also suffers this bug.
          1.x is the latest stable release but does not contain this fix which is almost a year old.

          Or will you release 2.0 some time soon?

          Show
          Torsten Mielke added a comment - Any idea why this bug got never fixed on any 1.x release? When running inside an OSGi container one also suffers this bug. 1.x is the latest stable release but does not contain this fix which is almost a year old. Or will you release 2.0 some time soon?
          Hide
          Luke Lu added a comment -

          We should probably fix this in 1.0 branch as well.

          Show
          Luke Lu added a comment - We should probably fix this in 1.0 branch as well.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #479 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk-Commit/479/)

          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #479 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk-Commit/479/ )
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk #576 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk/576/)
          HADOOP-7101. UserGroupInformation.getCurrentUser() fails when called from non-Hadoop JAAS context. Contributed by Todd Lipcon

          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk #576 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk/576/ ) HADOOP-7101 . UserGroupInformation.getCurrentUser() fails when called from non-Hadoop JAAS context. Contributed by Todd Lipcon
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-22-branch #15 (See https://hudson.apache.org/hudson/job/Hadoop-Common-22-branch/15/)
          HADOOP-7101. UserGroupInformation.getCurrentUser() fails when called from non-Hadoop JAAS context. Contributed by Todd Lipcon

          Show
          Hudson added a comment - Integrated in Hadoop-Common-22-branch #15 (See https://hudson.apache.org/hudson/job/Hadoop-Common-22-branch/15/ ) HADOOP-7101 . UserGroupInformation.getCurrentUser() fails when called from non-Hadoop JAAS context. Contributed by Todd Lipcon
          Hide
          Todd Lipcon added a comment -

          Yes, I agree the whole concept of loginUser fallback feels wrong... but like you said, outside the scope of this patch.

          To add a datapoint about testing - the user who was experiencing this issue inside tomcat tested the patch and reports the problem is indeed fixed.

          I'll commit to 0.22 and trunk shortly.

          Show
          Todd Lipcon added a comment - Yes, I agree the whole concept of loginUser fallback feels wrong... but like you said, outside the scope of this patch. To add a datapoint about testing - the user who was experiencing this issue inside tomcat tested the patch and reports the problem is indeed fixed. I'll commit to 0.22 and trunk shortly.
          Hide
          Kan Zhang added a comment -

          > And if the caller wants to use the credentials of the loginUser, call loginUser.doAs() explicitly.

          I meant "call getLoginUser().doAs() explicitly".

          Show
          Kan Zhang added a comment - > And if the caller wants to use the credentials of the loginUser, call loginUser.doAs() explicitly. I meant "call getLoginUser().doAs() explicitly".
          Hide
          Kan Zhang added a comment -

          I should note that the original code has an inconsistency in that the caller of getCurrentUser() never knows whether the returned currentUser is actually associated with the current AccessControlContext. If it is called in a doAs() block, then the returned currentUser is associated with the current AccessControlContext. Whereas if it is not called in a doAs() block, loginUser is returned (login is performed if needed) and this loginUser is NOT associated with the current AccessControlContext. This matters when we want to invoke, for example, Java GSS/Kerberos library, since these Java libraries will only check the current AccessControlContext for credentials. A cleaner approach would have been returning null when we're not in a doAs() block, to say that there is no currentUser being associated with current AccessControlContext. And if the caller wants to use the credentials of the loginUser, call loginUser.doAs() explicitly.

          It's not the purpose of this patch to fix the above inconsistency. This patch simply extends the current semantics. So +1.

          Show
          Kan Zhang added a comment - I should note that the original code has an inconsistency in that the caller of getCurrentUser() never knows whether the returned currentUser is actually associated with the current AccessControlContext. If it is called in a doAs() block, then the returned currentUser is associated with the current AccessControlContext. Whereas if it is not called in a doAs() block, loginUser is returned (login is performed if needed) and this loginUser is NOT associated with the current AccessControlContext. This matters when we want to invoke, for example, Java GSS/Kerberos library, since these Java libraries will only check the current AccessControlContext for credentials. A cleaner approach would have been returning null when we're not in a doAs() block, to say that there is no currentUser being associated with current AccessControlContext. And if the caller wants to use the credentials of the loginUser, call loginUser.doAs() explicitly. It's not the purpose of this patch to fix the above inconsistency. This patch simply extends the current semantics. So +1.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12468097/hadoop-7101.txt
          against trunk revision 1057970.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          +1 system test framework. The patch passed system test framework compile.

          Test results: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/172//testReport/
          Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/172//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Console output: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/172//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12468097/hadoop-7101.txt against trunk revision 1057970. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/172//testReport/ Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/172//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/172//console This message is automatically generated.

            People

            • Assignee:
              Todd Lipcon
              Reporter:
              Todd Lipcon
            • Votes:
              0 Vote for this issue
              Watchers:
              16 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development