Hadoop Common
  1. Hadoop Common
  2. HADOOP-6670

UserGroupInformation doesn't support use in hash tables

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.22.0
    • Component/s: security
    • Labels:
      None

      Description

      The UserGroupInformation objects are mutable, but they are used as keys in hash tables. This leads to serious problems in the FileSystem cache and RPC connection cache. We need to change the hashCode to be the identity hash code of the Subject and change equals to use == between the Subjects.

      1. fs-close.patch
        13 kB
        Owen O'Malley
      2. fs-close.patch
        9 kB
        Owen O'Malley
      3. fs-close.patch
        12 kB
        Owen O'Malley
      4. 6670-gridmix-test.bugfix.patch
        2 kB
        Devaraj Das
      5. c6670-02.patch
        6 kB
        Kan Zhang
      6. c6670-03.patch
        6 kB
        Kan Zhang

        Activity

        Hide
        Owen O'Malley added a comment -

        This patch fixes the problem and the testcases. I'm still getting a test case failure in the dist cache one.

        Show
        Owen O'Malley added a comment - This patch fixes the problem and the testcases. I'm still getting a test case failure in the dist cache one.
        Hide
        Owen O'Malley added a comment -

        This patch changes UGI.hashCode to be the identity hash of the Subject and equals to check == between the subjects.

        Show
        Owen O'Malley added a comment - This patch changes UGI.hashCode to be the identity hash of the Subject and equals to check == between the subjects.
        Hide
        Owen O'Malley added a comment -

        This is the updated patch for y20. Not for inclusion.

        Show
        Owen O'Malley added a comment - This is the updated patch for y20. Not for inclusion.
        Hide
        Devaraj Das added a comment -

        Fixes a testcase that got affected by the UGI changes in hashcode/equals methods.

        Show
        Devaraj Das added a comment - Fixes a testcase that got affected by the UGI changes in hashcode/equals methods.
        Hide
        Kan Zhang added a comment -

        Attaching a straight-forward port for trunk. Changes made to TestFileSystem in the original patch is now applied to TestFileSystemCaching. Changes made to FileSystem.java are ignored since only debug messages were added. Changes made to TestTrackerDistributedCacheManager are also ignored for the same reason. Devaraj's bug fix for Gridmix is not ported at this time since related Gridmix files are not forward ported yet. It's better to incorporate the bug fix when porting Gridmix.

        Show
        Kan Zhang added a comment - Attaching a straight-forward port for trunk. Changes made to TestFileSystem in the original patch is now applied to TestFileSystemCaching. Changes made to FileSystem.java are ignored since only debug messages were added. Changes made to TestTrackerDistributedCacheManager are also ignored for the same reason. Devaraj's bug fix for Gridmix is not ported at this time since related Gridmix files are not forward ported yet. It's better to incorporate the bug fix when porting Gridmix.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12449172/c6670-02.patch
        against trunk revision 963490.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 6 new or modified tests.

        -1 javadoc. The javadoc tool appears to have generated 1 warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/609/testReport/
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/609/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/609/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/609/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12449172/c6670-02.patch against trunk revision 963490. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. -1 javadoc. The javadoc tool appears to have generated 1 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/609/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/609/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/609/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/609/console This message is automatically generated.
        Hide
        Kan Zhang added a comment -

        The javadoc warnings are due to KerberosName.java and SecurityUtil.java, and not related to this patch.

        Show
        Kan Zhang added a comment - The javadoc warnings are due to KerberosName.java and SecurityUtil.java, and not related to this patch.
        Hide
        Jitendra Nath Pandey added a comment -

        In TestUserGroupInformation.java there is a repeated check for equality of two UGIs with same subject which can be removed.

        Show
        Jitendra Nath Pandey added a comment - In TestUserGroupInformation.java there is a repeated check for equality of two UGIs with same subject which can be removed.
        Hide
        Kan Zhang added a comment -

        Attaching a new patch that removes the redundant checking in test per Jitendra's comment. Otherwise, it stays the same.

        Show
        Kan Zhang added a comment - Attaching a new patch that removes the redundant checking in test per Jitendra's comment. Otherwise, it stays the same.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12449720/c6670-03.patch
        against trunk revision 964939.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 6 new or modified tests.

        -1 javadoc. The javadoc tool appears to have generated 1 warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/620/testReport/
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/620/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/620/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/620/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12449720/c6670-03.patch against trunk revision 964939. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. -1 javadoc. The javadoc tool appears to have generated 1 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/620/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/620/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/620/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/620/console This message is automatically generated.
        Hide
        Devaraj Das added a comment -

        I just committed this. Thanks, Owen and Kan!

        Show
        Devaraj Das added a comment - I just committed this. Thanks, Owen and Kan!
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk-Commit #329 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk-Commit/329/)
        HADOOP-6670. Use the UserGroupInformation's Subject as the criteria for equals and hashCode. Contributed by Owen O'Malley and Kan Zhang.

        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #329 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk-Commit/329/ ) HADOOP-6670 . Use the UserGroupInformation's Subject as the criteria for equals and hashCode. Contributed by Owen O'Malley and Kan Zhang.
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk #395 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk/395/)
        HADOOP-6670. Use the UserGroupInformation's Subject as the criteria for equals and hashCode. Contributed by Owen O'Malley and Kan Zhang.

        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk #395 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk/395/ ) HADOOP-6670 . Use the UserGroupInformation's Subject as the criteria for equals and hashCode. Contributed by Owen O'Malley and Kan Zhang.

          People

          • Assignee:
            Owen O'Malley
            Reporter:
            Owen O'Malley
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development