Hadoop Common
  1. Hadoop Common
  2. HADOOP-8088

User-group mapping cache incorrectly does negative caching on transient failures

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.20.205.0, 1.0.0, 1.1.0, 0.23.1, 0.24.0
    • Fix Version/s: 0.23.3, 2.0.2-alpha
    • Component/s: security
    • Labels:
      None

      Description

      We've seen a case where some getGroups() calls fail when the ldap server or the network is having transient failures. Looking at the code, the shell-based and the JNI-based implementations swallow exceptions and return an empty or partial list. The caller, Groups#getGroups() adds this likely empty list into the mapping cache for the user. This will function as negative caching until the cache expires. I don't think we want negative caching here, but even if we do, it should be intelligent enough to distinguish transient failures from ENOENT. The log message in the jni-based impl also needs an improvement. It should print what exception it encountered instead of just saying one happened.

      1. hadoop-8088-branch-1.patch
        9 kB
        Kihwal Lee
      2. hadoop-8088-trunk.patch
        9 kB
        Kihwal Lee
      3. hadoop-8088-trunk.patch
        9 kB
        Kihwal Lee
      4. hadoop-8088-branch-1.patch
        5 kB
        Kihwal Lee
      5. hadoop-8088-trunk.patch
        5 kB
        Kihwal Lee

        Issue Links

          Activity

          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #1041 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1041/)
          moved HADOOP-8088 to 0.23.3 (Revision 1309436)

          Result = SUCCESS
          bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1309436
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1041 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1041/ ) moved HADOOP-8088 to 0.23.3 (Revision 1309436) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1309436 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #1006 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1006/)
          moved HADOOP-8088 to 0.23.3 (Revision 1309436)

          Result = FAILURE
          bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1309436
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1006 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1006/ ) moved HADOOP-8088 to 0.23.3 (Revision 1309436) Result = FAILURE bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1309436 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-0.23-Build #219 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/219/)
          svn merge -c 1302062 trunk to branch-0.23 FIXES HADOOP-8088. User-group mapping cache incorrectly does negative caching on transient failures (Revision 1309431)

          Result = SUCCESS
          bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1309431
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/Groups.java
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestGroupsCaching.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #219 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/219/ ) svn merge -c 1302062 trunk to branch-0.23 FIXES HADOOP-8088 . User-group mapping cache incorrectly does negative caching on transient failures (Revision 1309431) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1309431 Files : /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/Groups.java /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestGroupsCaching.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #2003 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2003/)
          moved HADOOP-8088 to 0.23.3 (Revision 1309436)

          Result = ABORTED
          bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1309436
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #2003 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2003/ ) moved HADOOP-8088 to 0.23.3 (Revision 1309436) Result = ABORTED bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1309436 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #1990 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1990/)
          moved HADOOP-8088 to 0.23.3 (Revision 1309436)

          Result = SUCCESS
          bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1309436
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #1990 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1990/ ) moved HADOOP-8088 to 0.23.3 (Revision 1309436) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1309436 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #2065 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2065/)
          moved HADOOP-8088 to 0.23.3 (Revision 1309436)

          Result = SUCCESS
          bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1309436
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2065 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2065/ ) moved HADOOP-8088 to 0.23.3 (Revision 1309436) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1309436 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          Robert Joseph Evans added a comment -

          I pulled this into 0.23.3. It was listed as being in 0.23.2, but it only made it into branch-0.23, which was renamed to branch-2.

          Show
          Robert Joseph Evans added a comment - I pulled this into 0.23.3. It was listed as being in 0.23.2, but it only made it into branch-0.23, which was renamed to branch-2.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12518767/hadoop-8088-trunk.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/760//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12518767/hadoop-8088-trunk.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/760//console This message is automatically generated.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #1023 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1023/)
          HADOOP-8088. User-group mapping cache incorrectly does negative caching on transient failures (Khiwal Lee via bobby) (Revision 1302062)

          Result = SUCCESS
          bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1302062
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/Groups.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestGroupsCaching.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1023 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1023/ ) HADOOP-8088 . User-group mapping cache incorrectly does negative caching on transient failures (Khiwal Lee via bobby) (Revision 1302062) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1302062 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/Groups.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestGroupsCaching.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-0.23-Build #229 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/229/)
          svn merge -c 1302062 trunk to branch-0.23 FIXES HADOOP-8088. User-group mapping cache incorrectly does negative caching on transient failures (Khiwal Lee via bobby) (Revision 1302063)

          Result = FAILURE
          bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1302063
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/Groups.java
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestGroupsCaching.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Build #229 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/229/ ) svn merge -c 1302062 trunk to branch-0.23 FIXES HADOOP-8088 . User-group mapping cache incorrectly does negative caching on transient failures (Khiwal Lee via bobby) (Revision 1302063) Result = FAILURE bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1302063 Files : /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/Groups.java /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestGroupsCaching.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-0.23-Build #201 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/201/)
          svn merge -c 1302062 trunk to branch-0.23 FIXES HADOOP-8088. User-group mapping cache incorrectly does negative caching on transient failures (Khiwal Lee via bobby) (Revision 1302063)

          Result = SUCCESS
          bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1302063
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/Groups.java
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestGroupsCaching.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #201 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/201/ ) svn merge -c 1302062 trunk to branch-0.23 FIXES HADOOP-8088 . User-group mapping cache incorrectly does negative caching on transient failures (Khiwal Lee via bobby) (Revision 1302063) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1302063 Files : /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/Groups.java /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestGroupsCaching.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #988 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/988/)
          HADOOP-8088. User-group mapping cache incorrectly does negative caching on transient failures (Khiwal Lee via bobby) (Revision 1302062)

          Result = SUCCESS
          bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1302062
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/Groups.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestGroupsCaching.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #988 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/988/ ) HADOOP-8088 . User-group mapping cache incorrectly does negative caching on transient failures (Khiwal Lee via bobby) (Revision 1302062) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1302062 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/Groups.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestGroupsCaching.java
          Hide
          Matt Foley added a comment -

          Just squeaked into 1.0.2-rc1.

          Show
          Matt Foley added a comment - Just squeaked into 1.0.2-rc1.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-0.23-Commit #708 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/708/)
          svn merge -c 1302062 trunk to branch-0.23 FIXES HADOOP-8088. User-group mapping cache incorrectly does negative caching on transient failures (Khiwal Lee via bobby) (Revision 1302063)

          Result = ABORTED
          bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1302063
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/Groups.java
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestGroupsCaching.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Commit #708 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/708/ ) svn merge -c 1302062 trunk to branch-0.23 FIXES HADOOP-8088 . User-group mapping cache incorrectly does negative caching on transient failures (Khiwal Lee via bobby) (Revision 1302063) Result = ABORTED bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1302063 Files : /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/Groups.java /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestGroupsCaching.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #1905 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1905/)
          HADOOP-8088. User-group mapping cache incorrectly does negative caching on transient failures (Khiwal Lee via bobby) (Revision 1302062)

          Result = ABORTED
          bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1302062
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/Groups.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestGroupsCaching.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #1905 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1905/ ) HADOOP-8088 . User-group mapping cache incorrectly does negative caching on transient failures (Khiwal Lee via bobby) (Revision 1302062) Result = ABORTED bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1302062 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/Groups.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestGroupsCaching.java
          Hide
          Robert Joseph Evans added a comment -

          All of my tests passed.

          Thanks Kihwal. I just checked this into trunk, branch-0.23, branch-1 and branch-1.0. I plan to check this into branch-0.23.2 shortly too, and I will resolve the JIRA once I put it into 0.23.2

          Show
          Robert Joseph Evans added a comment - All of my tests passed. Thanks Kihwal. I just checked this into trunk, branch-0.23, branch-1 and branch-1.0. I plan to check this into branch-0.23.2 shortly too, and I will resolve the JIRA once I put it into 0.23.2
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-0.23-Commit #700 (See https://builds.apache.org/job/Hadoop-Common-0.23-Commit/700/)
          svn merge -c 1302062 trunk to branch-0.23 FIXES HADOOP-8088. User-group mapping cache incorrectly does negative caching on transient failures (Khiwal Lee via bobby) (Revision 1302063)

          Result = SUCCESS
          bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1302063
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/Groups.java
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestGroupsCaching.java
          Show
          Hudson added a comment - Integrated in Hadoop-Common-0.23-Commit #700 (See https://builds.apache.org/job/Hadoop-Common-0.23-Commit/700/ ) svn merge -c 1302062 trunk to branch-0.23 FIXES HADOOP-8088 . User-group mapping cache incorrectly does negative caching on transient failures (Khiwal Lee via bobby) (Revision 1302063) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1302063 Files : /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/Groups.java /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestGroupsCaching.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #1897 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1897/)
          HADOOP-8088. User-group mapping cache incorrectly does negative caching on transient failures (Khiwal Lee via bobby) (Revision 1302062)

          Result = SUCCESS
          bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1302062
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/Groups.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestGroupsCaching.java
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #1897 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1897/ ) HADOOP-8088 . User-group mapping cache incorrectly does negative caching on transient failures (Khiwal Lee via bobby) (Revision 1302062) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1302062 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/Groups.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestGroupsCaching.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-0.23-Commit #691 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/691/)
          svn merge -c 1302062 trunk to branch-0.23 FIXES HADOOP-8088. User-group mapping cache incorrectly does negative caching on transient failures (Khiwal Lee via bobby) (Revision 1302063)

          Result = SUCCESS
          bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1302063
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/Groups.java
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestGroupsCaching.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Commit #691 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/691/ ) svn merge -c 1302062 trunk to branch-0.23 FIXES HADOOP-8088 . User-group mapping cache incorrectly does negative caching on transient failures (Khiwal Lee via bobby) (Revision 1302063) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1302063 Files : /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/Groups.java /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestGroupsCaching.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #1971 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1971/)
          HADOOP-8088. User-group mapping cache incorrectly does negative caching on transient failures (Khiwal Lee via bobby) (Revision 1302062)

          Result = SUCCESS
          bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1302062
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/Groups.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestGroupsCaching.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #1971 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1971/ ) HADOOP-8088 . User-group mapping cache incorrectly does negative caching on transient failures (Khiwal Lee via bobby) (Revision 1302062) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1302062 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/Groups.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestGroupsCaching.java
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12518767/hadoop-8088-trunk.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in .

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/724//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/724//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12518767/hadoop-8088-trunk.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/724//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/724//console This message is automatically generated.
          Hide
          Aaron T. Myers added a comment -

          Hudson appears to be down.

          I actually think Jenkins itself is fine, but for some reason test-patch builds aren't getting triggered automatically when new patches are uploaded. I've just kicked Jenkins manually to run test-patch for this JIRA: https://builds.apache.org/view/G-L/view/Hadoop/job/PreCommit-HADOOP-Build/724/

          Show
          Aaron T. Myers added a comment - Hudson appears to be down. I actually think Jenkins itself is fine, but for some reason test-patch builds aren't getting triggered automatically when new patches are uploaded. I've just kicked Jenkins manually to run test-patch for this JIRA: https://builds.apache.org/view/G-L/view/Hadoop/job/PreCommit-HADOOP-Build/724/
          Hide
          Robert Joseph Evans added a comment -

          Hudson appears to be down. I looked over the patch and it looks good. +1 assuming that the tests pass when I run them manually.

          Show
          Robert Joseph Evans added a comment - Hudson appears to be down. I looked over the patch and it looks good. +1 assuming that the tests pass when I run them manually.
          Hide
          Kihwal Lee added a comment -

          The new patch removes the negative caching behavior. The test case was modified accordingly.

          Show
          Kihwal Lee added a comment - The new patch removes the negative caching behavior. The test case was modified accordingly.
          Hide
          Jitendra Nath Pandey added a comment -

          I agree with devaraj's proposal. Too many queries for a non-existent user should happen only in case of a security attack, but authentication should prevent any rogue user if security is enabled.

          Show
          Jitendra Nath Pandey added a comment - I agree with devaraj's proposal. Too many queries for a non-existent user should happen only in case of a security attack, but authentication should prevent any rogue user if security is enabled.
          Hide
          Kihwal Lee added a comment -

          Thinking about it, it may make sense to not cache negatives at all.

          I initially thought so, but was not sure about the original design goal of the group caching. On many systems, nscd or equivalent can do negative caching to reduce unnecessary network traffic to ldap server, etc. So removing the behavior might not be too much of concern.

          Show
          Kihwal Lee added a comment - Thinking about it, it may make sense to not cache negatives at all. I initially thought so, but was not sure about the original design goal of the group caching. On many systems, nscd or equivalent can do negative caching to reduce unnecessary network traffic to ldap server, etc. So removing the behavior might not be too much of concern.
          Hide
          Devaraj Das added a comment -

          Thinking about it, it may make sense to not cache negatives at all (I don't think the original implementation aimed to handle this use-case). I can't imagine a situation where we'd have many queries for non-existent users (so we'd still be fine with not overloading the infrastructure, like ldap, which was the primary motivation for the cache). Does that make sense?

          Show
          Devaraj Das added a comment - Thinking about it, it may make sense to not cache negatives at all (I don't think the original implementation aimed to handle this use-case). I can't imagine a situation where we'd have many queries for non-existent users (so we'd still be fine with not overloading the infrastructure, like ldap, which was the primary motivation for the cache). Does that make sense?
          Hide
          Aaron T. Myers added a comment -

          I do agree with you on the problem of having too many configs, but we often end up going this route in (sometimes justifiable) fear of breaking compatibility even if the bug is clearly there and in need of fix. I would appreciate your further input on how we can address the issue with the minimum negative impact.

          The problem of operability is not in having too many configs, but in having too many configs that one needs to tweak. I think having a config for this is fine, as long as we choose the correct default. If after deploying this on a cluster an operator decides they want to change the behavior, they'll be very happy there's a config to do it, so they won't have to recompile/redeploy.

          Show
          Aaron T. Myers added a comment - I do agree with you on the problem of having too many configs, but we often end up going this route in (sometimes justifiable) fear of breaking compatibility even if the bug is clearly there and in need of fix. I would appreciate your further input on how we can address the issue with the minimum negative impact. The problem of operability is not in having too many configs, but in having too many configs that one needs to tweak. I think having a config for this is fine, as long as we choose the correct default. If after deploying this on a cluster an operator decides they want to change the behavior, they'll be very happy there's a config to do it, so they won't have to recompile/redeploy.
          Hide
          Kihwal Lee added a comment -

          We use a refresh cycle of 4 hrs. It becomes a serious issue when headless users associated with tight SLA jobs get in there. This behavior is not a hypothetical one. We've seen this in production.

          I do agree with you on the problem of having too many configs, but we often end up going this route in (sometimes justifiable) fear of breaking compatibility even if the bug is clearly there and in need of fix. I would appreciate your further input on how we can address the issue with the minimum negative impact.

          Show
          Kihwal Lee added a comment - We use a refresh cycle of 4 hrs. It becomes a serious issue when headless users associated with tight SLA jobs get in there. This behavior is not a hypothetical one. We've seen this in production. I do agree with you on the problem of having too many configs, but we often end up going this route in (sometimes justifiable) fear of breaking compatibility even if the bug is clearly there and in need of fix. I would appreciate your further input on how we can address the issue with the minimum negative impact.
          Hide
          Devaraj Das added a comment -

          The config for negative cache seems like an overkill. I'd suggest that we just handle the transient failures better.

          Show
          Devaraj Das added a comment - The config for negative cache seems like an overkill. I'd suggest that we just handle the transient failures better.
          Hide
          Jitendra Nath Pandey added a comment -

          I am concerned about adding another configuration for negative caching. Every new configuration parameter adds to operational complexity. I was wondering how serious is the issue that empty groups are cached? The groups will be recovered after cache expires. How long is the cache expiry?

          Show
          Jitendra Nath Pandey added a comment - I am concerned about adding another configuration for negative caching. Every new configuration parameter adds to operational complexity. I was wondering how serious is the issue that empty groups are cached? The groups will be recovered after cache expires. How long is the cache expiry?
          Hide
          Kihwal Lee added a comment -

          Resubmiting the same trunk patch for jenkins to pick up.

          Show
          Kihwal Lee added a comment - Resubmiting the same trunk patch for jenkins to pick up.
          Hide
          Kihwal Lee added a comment -

          Another patch for TRUNK.

          Show
          Kihwal Lee added a comment - Another patch for TRUNK.
          Hide
          Kihwal Lee added a comment -

          For the branch-1 patch

               [exec] -1 overall.  
               [exec] 
               [exec]     +1 @author.  The patch does not contain any @author tags.
               [exec] 
               [exec]     +1 tests included.  The patch appears to include 13 new or modified tests.
               [exec] 
               [exec]     +1 javadoc.  The javadoc tool did not generate any warning messages.
               [exec] 
               [exec]     +1 javac.  The applied patch does not increase the total number of javac compiler warnings.
               [exec] 
               [exec]     -1 findbugs.  The patch appears to introduce 6 new Findbugs (version 1.3.9) warnings.
          

          I manually ran findbugs again. There is no difference before/after the patch: Both have the same 217 warnings.

          Show
          Kihwal Lee added a comment - For the branch-1 patch [exec] -1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] +1 tests included. The patch appears to include 13 new or modified tests. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] -1 findbugs. The patch appears to introduce 6 new Findbugs (version 1.3.9) warnings. I manually ran findbugs again. There is no difference before/after the patch: Both have the same 217 warnings.

            People

            • Assignee:
              Kihwal Lee
              Reporter:
              Kihwal Lee
            • Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development