Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-12450

UserGroupInformation should not log at WARN level if no groups are found

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: security
    • Labels:
      None

      Description

      HBase tries to get the groups of a user on every request. That user may or may not exist on the box running Hadoop/HBase.

      If that user doesn't exist currently Hadoop will log at the WARN level everytime. This leads to gigs of log spam and serious GC issues.

      1. HADOOP-12450-v1.patch
        1 kB
        Elliott Clark
      2. HADOOP-12450-v2.patch
        1 kB
        Elliott Clark

        Issue Links

          Activity

          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2446 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2446/)
          HADOOP-12450. UserGroupInformation should not log at WARN level if no (stevel: rev e286512a7143427f2975ec92cdc4fad0a093a456)

          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2446 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2446/ ) HADOOP-12450 . UserGroupInformation should not log at WARN level if no (stevel: rev e286512a7143427f2975ec92cdc4fad0a093a456) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #509 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/509/)
          HADOOP-12450. UserGroupInformation should not log at WARN level if no (stevel: rev e286512a7143427f2975ec92cdc4fad0a093a456)

          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #509 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/509/ ) HADOOP-12450 . UserGroupInformation should not log at WARN level if no (stevel: rev e286512a7143427f2975ec92cdc4fad0a093a456) hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #547 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/547/)
          HADOOP-12450. UserGroupInformation should not log at WARN level if no (stevel: rev e286512a7143427f2975ec92cdc4fad0a093a456)

          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #547 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/547/ ) HADOOP-12450 . UserGroupInformation should not log at WARN level if no (stevel: rev e286512a7143427f2975ec92cdc4fad0a093a456) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2495 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2495/)
          HADOOP-12450. UserGroupInformation should not log at WARN level if no (stevel: rev e286512a7143427f2975ec92cdc4fad0a093a456)

          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2495 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2495/ ) HADOOP-12450 . UserGroupInformation should not log at WARN level if no (stevel: rev e286512a7143427f2975ec92cdc4fad0a093a456) hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #1283 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1283/)
          HADOOP-12450. UserGroupInformation should not log at WARN level if no (stevel: rev e286512a7143427f2975ec92cdc4fad0a093a456)

          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #1283 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1283/ ) HADOOP-12450 . UserGroupInformation should not log at WARN level if no (stevel: rev e286512a7143427f2975ec92cdc4fad0a093a456) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #561 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/561/)
          HADOOP-12450. UserGroupInformation should not log at WARN level if no (stevel: rev e286512a7143427f2975ec92cdc4fad0a093a456)

          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #561 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/561/ ) HADOOP-12450 . UserGroupInformation should not log at WARN level if no (stevel: rev e286512a7143427f2975ec92cdc4fad0a093a456) hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #8657 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8657/)
          HADOOP-12450. UserGroupInformation should not log at WARN level if no (stevel: rev e286512a7143427f2975ec92cdc4fad0a093a456)

          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8657 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8657/ ) HADOOP-12450 . UserGroupInformation should not log at WARN level if no (stevel: rev e286512a7143427f2975ec92cdc4fad0a093a456) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          stevel@apache.org Steve Loughran added a comment -

          +1, committed —thanks!

          Show
          stevel@apache.org Steve Loughran added a comment - +1, committed —thanks!
          Hide
          nkeywal Nicolas Liochon added a comment -

          +1 (non binding)...

          Show
          nkeywal Nicolas Liochon added a comment - +1 (non binding)...
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          -1 pre-patch 17m 38s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 javac 8m 1s There were no new javac warning messages.
          +1 javadoc 10m 19s There were no new javadoc warning messages.
          +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 1m 5s There were no new checkstyle issues.
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 28s mvn install still works.
          +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
          +1 findbugs 1m 56s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          -1 common tests 6m 36s Tests failed in hadoop-common.
              48m 4s  



          Reason Tests
          Failed unit tests hadoop.fs.sftp.TestSFTPFileSystem
            hadoop.net.TestDNS



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12764300/HADOOP-12450-v2.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 80d33b5
          Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HADOOP-Build/7739/artifact/patchprocess/trunkFindbugsWarningshadoop-common.html
          hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7739/artifact/patchprocess/testrun_hadoop-common.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7739/testReport/
          Java 1.7.0_55
          uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7739/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 17m 38s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 8m 1s There were no new javac warning messages. +1 javadoc 10m 19s There were no new javadoc warning messages. +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 1m 5s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 28s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 1m 56s The patch does not introduce any new Findbugs (version 3.0.0) warnings. -1 common tests 6m 36s Tests failed in hadoop-common.     48m 4s   Reason Tests Failed unit tests hadoop.fs.sftp.TestSFTPFileSystem   hadoop.net.TestDNS Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12764300/HADOOP-12450-v2.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 80d33b5 Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HADOOP-Build/7739/artifact/patchprocess/trunkFindbugsWarningshadoop-common.html hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7739/artifact/patchprocess/testrun_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7739/testReport/ Java 1.7.0_55 uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7739/console This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          -1 pre-patch 19m 18s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 javac 9m 2s There were no new javac warning messages.
          +1 javadoc 11m 56s There were no new javadoc warning messages.
          +1 release audit 0m 25s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 1m 16s There were no new checkstyle issues.
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 35s mvn install still works.
          +1 eclipse:eclipse 0m 37s The patch built with eclipse:eclipse.
          +1 findbugs 2m 3s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 common tests 7m 59s Tests passed in hadoop-common.
              54m 17s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12764284/HADOOP-12450-v1.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 80d33b5
          Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HADOOP-Build/7738/artifact/patchprocess/trunkFindbugsWarningshadoop-common.html
          hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7738/artifact/patchprocess/testrun_hadoop-common.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7738/testReport/
          Java 1.7.0_55
          uname Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7738/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 19m 18s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 9m 2s There were no new javac warning messages. +1 javadoc 11m 56s There were no new javadoc warning messages. +1 release audit 0m 25s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 1m 16s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 35s mvn install still works. +1 eclipse:eclipse 0m 37s The patch built with eclipse:eclipse. +1 findbugs 2m 3s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 common tests 7m 59s Tests passed in hadoop-common.     54m 17s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12764284/HADOOP-12450-v1.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 80d33b5 Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HADOOP-Build/7738/artifact/patchprocess/trunkFindbugsWarningshadoop-common.html hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7738/artifact/patchprocess/testrun_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7738/testReport/ Java 1.7.0_55 uname Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7738/console This message was automatically generated.
          Hide
          eclark Elliott Clark added a comment -

          Patch with log at debug.

          Show
          eclark Elliott Clark added a comment - Patch with log at debug.
          Hide
          stevel@apache.org Steve Loughran added a comment -
          1. it should really be at debug; trace doesn't get used in the hadoop code.
          2. Given it's also swallowing the exception (which may be a sign of something fundamental), maybe the logging should also include the nested exception.

          What do others think?

          Show
          stevel@apache.org Steve Loughran added a comment - it should really be at debug; trace doesn't get used in the hadoop code. Given it's also swallowing the exception (which may be a sign of something fundamental), maybe the logging should also include the nested exception. What do others think?

            People

            • Assignee:
              eclark Elliott Clark
              Reporter:
              eclark Elliott Clark
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development