Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-11962

Sasl message with MD5 challenge text shouldn't be LOG out even in debug level.

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 2.6.0
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: ipc, security
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      Some log examples:

      2014-09-24 05:42:12,975 DEBUG security.SaslRpcServer (SaslRpcServer.java:create(174)) - Created SASL server with mechanism = DIGEST-MD5
      2014-09-24 05:42:12,977 DEBUG ipc.Server (Server.java:doSaslReply(1424)) - Sending sasl message state: NEGOTIATE
      auths {
        method: "TOKEN"
        mechanism: "DIGEST-MD5"
        protocol: ""
        serverId: "default"
        challenge: "realm=\"default\",nonce=\"yIvZDpbzGGq3yIrMynVKnEv9Z0qw6lxpr9nZxm0r\",qop=\"auth\",charset=utf-8,algorithm=md5-sess"
      }
      ...
      ...
      2014-09-24 06:21:59,146 DEBUG ipc.Server (Server.java:doSaslReply(1424)) - Sending sasl message state: CHALLENGE
      token: "`l\006\t*\206H\206\367\022\001\002\002\002\000o]0[\240\003\002\001\005\241\003\002\001\017\242O0M\240\003\002\001\020\242F\004D#\030\336|kb\232\033V\340\342F\334\230\347\230\362)u!=\215\271\006\244:\244\221vn\215*\323\353\360\350\3006\366\3340\245\371Ri\273\374\307\017\207Z\233\326\217\224!yo$\373\233\315:JsY!^?"
      

      We should get rid of this kind of log in production environment even under debug log level.

      1. HADOOP-11962.patch
        0.8 kB
        Junping Du
      2. HADOOP-11962-v2.patch
        2 kB
        Junping Du

        Activity

        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk #2142 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2142/)
        HADOOP-11962. Sasl message with MD5 challenge text shouldn't be LOG out even in debug level. Contributed by Junping Du. (wheat9: rev 2f4b6d1157f280c8a6e1b2e7217fd2ec16991985)

        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/SaslRpcClient.java
        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java
        • hadoop-common-project/hadoop-common/CHANGES.txt
        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2142 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2142/ ) HADOOP-11962 . Sasl message with MD5 challenge text shouldn't be LOG out even in debug level. Contributed by Junping Du. (wheat9: rev 2f4b6d1157f280c8a6e1b2e7217fd2ec16991985) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/SaslRpcClient.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #194 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/194/)
        HADOOP-11962. Sasl message with MD5 challenge text shouldn't be LOG out even in debug level. Contributed by Junping Du. (wheat9: rev 2f4b6d1157f280c8a6e1b2e7217fd2ec16991985)

        • hadoop-common-project/hadoop-common/CHANGES.txt
        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java
        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/SaslRpcClient.java
        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #194 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/194/ ) HADOOP-11962 . Sasl message with MD5 challenge text shouldn't be LOG out even in debug level. Contributed by Junping Du. (wheat9: rev 2f4b6d1157f280c8a6e1b2e7217fd2ec16991985) hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/SaslRpcClient.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #184 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/184/)
        HADOOP-11962. Sasl message with MD5 challenge text shouldn't be LOG out even in debug level. Contributed by Junping Du. (wheat9: rev 2f4b6d1157f280c8a6e1b2e7217fd2ec16991985)

        • hadoop-common-project/hadoop-common/CHANGES.txt
        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java
        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/SaslRpcClient.java
        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #184 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/184/ ) HADOOP-11962 . Sasl message with MD5 challenge text shouldn't be LOG out even in debug level. Contributed by Junping Du. (wheat9: rev 2f4b6d1157f280c8a6e1b2e7217fd2ec16991985) hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/SaslRpcClient.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk #2124 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2124/)
        HADOOP-11962. Sasl message with MD5 challenge text shouldn't be LOG out even in debug level. Contributed by Junping Du. (wheat9: rev 2f4b6d1157f280c8a6e1b2e7217fd2ec16991985)

        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java
        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/SaslRpcClient.java
        • hadoop-common-project/hadoop-common/CHANGES.txt
        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2124 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2124/ ) HADOOP-11962 . Sasl message with MD5 challenge text shouldn't be LOG out even in debug level. Contributed by Junping Du. (wheat9: rev 2f4b6d1157f280c8a6e1b2e7217fd2ec16991985) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/SaslRpcClient.java hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Yarn-trunk #926 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/926/)
        HADOOP-11962. Sasl message with MD5 challenge text shouldn't be LOG out even in debug level. Contributed by Junping Du. (wheat9: rev 2f4b6d1157f280c8a6e1b2e7217fd2ec16991985)

        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java
        • hadoop-common-project/hadoop-common/CHANGES.txt
        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java
        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/SaslRpcClient.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk #926 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/926/ ) HADOOP-11962 . Sasl message with MD5 challenge text shouldn't be LOG out even in debug level. Contributed by Junping Du. (wheat9: rev 2f4b6d1157f280c8a6e1b2e7217fd2ec16991985) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/SaslRpcClient.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #195 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/195/)
        HADOOP-11962. Sasl message with MD5 challenge text shouldn't be LOG out even in debug level. Contributed by Junping Du. (wheat9: rev 2f4b6d1157f280c8a6e1b2e7217fd2ec16991985)

        • hadoop-common-project/hadoop-common/CHANGES.txt
        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java
        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/SaslRpcClient.java
        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #195 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/195/ ) HADOOP-11962 . Sasl message with MD5 challenge text shouldn't be LOG out even in debug level. Contributed by Junping Du. (wheat9: rev 2f4b6d1157f280c8a6e1b2e7217fd2ec16991985) hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/SaslRpcClient.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java
        Hide
        djp Junping Du added a comment -

        Thanks Haohui Mai and Surendra Singh Lilhore for review and comments!

        Show
        djp Junping Du added a comment - Thanks Haohui Mai and Surendra Singh Lilhore for review and comments!
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #7808 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7808/)
        HADOOP-11962. Sasl message with MD5 challenge text shouldn't be LOG out even in debug level. Contributed by Junping Du. (wheat9: rev 2f4b6d1157f280c8a6e1b2e7217fd2ec16991985)

        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java
        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/SaslRpcClient.java
        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java
        • hadoop-common-project/hadoop-common/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #7808 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7808/ ) HADOOP-11962 . Sasl message with MD5 challenge text shouldn't be LOG out even in debug level. Contributed by Junping Du. (wheat9: rev 2f4b6d1157f280c8a6e1b2e7217fd2ec16991985) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/SaslRpcClient.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java hadoop-common-project/hadoop-common/CHANGES.txt
        Hide
        wheat9 Haohui Mai added a comment -

        I've committed the patch to trunk and branch-2. Thanks Junping Du for the contribution.

        Show
        wheat9 Haohui Mai added a comment - I've committed the patch to trunk and branch-2. Thanks Junping Du for the contribution.
        Hide
        wheat9 Haohui Mai added a comment -

        +1. I'll commit it shortly.

        Show
        wheat9 Haohui Mai added a comment - +1. I'll commit it shortly.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 14m 47s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 javac 7m 32s There were no new javac warning messages.
        +1 javadoc 9m 34s There were no new javadoc warning messages.
        +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
        -1 checkstyle 1m 5s The applied patch generated 1 new checkstyle issues (total was 424, now 424).
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 34s mvn install still works.
        +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
        +1 findbugs 1m 41s The patch does not introduce any new Findbugs (version 2.0.3) warnings.
        +1 common tests 22m 29s Tests passed in hadoop-common.
            59m 40s  



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12732254/HADOOP-11962-v2.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 6d5da94
        checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/6655/artifact/patchprocess/diffcheckstylehadoop-common.txt
        hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/6655/artifact/patchprocess/testrun_hadoop-common.txt
        Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/6655/testReport/
        Java 1.7.0_55
        uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/6655/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 47s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 32s There were no new javac warning messages. +1 javadoc 9m 34s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 5s The applied patch generated 1 new checkstyle issues (total was 424, now 424). +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 34s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 1m 41s The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 common tests 22m 29s Tests passed in hadoop-common.     59m 40s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12732254/HADOOP-11962-v2.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 6d5da94 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/6655/artifact/patchprocess/diffcheckstylehadoop-common.txt hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/6655/artifact/patchprocess/testrun_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/6655/testReport/ Java 1.7.0_55 uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/6655/console This message was automatically generated.
        Hide
        djp Junping Du added a comment -

        In these two class also we can avoid printing secure info.

        Nice catch, Surendra Singh Lilhore! Removing debug info in these two places in v2 patch.

        Show
        djp Junping Du added a comment - In these two class also we can avoid printing secure info. Nice catch, Surendra Singh Lilhore ! Removing debug info in these two places in v2 patch.
        Hide
        surendrasingh Surendra Singh Lilhore added a comment -

        Junping Du Thanks for reporting this issue.. we also got internally same issue.

        In these two class also we can avoid printing secure info.

        In SaslRpcClient.java sasl message

              
              if (LOG.isDebugEnabled()) {
                LOG.debug("Received SASL message "+saslMessage);
              }
        

        and in UserGroupInformation.java ticket info.

                if (LOG.isDebugEnabled()) {
                  LOG.debug("Found tgt " + ticket);
                }
        
        Show
        surendrasingh Surendra Singh Lilhore added a comment - Junping Du Thanks for reporting this issue.. we also got internally same issue. In these two class also we can avoid printing secure info. In SaslRpcClient.java sasl message if (LOG.isDebugEnabled()) { LOG.debug( "Received SASL message " +saslMessage); } and in UserGroupInformation.java ticket info. if (LOG.isDebugEnabled()) { LOG.debug( "Found tgt " + ticket); }
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 14m 56s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 javac 7m 32s There were no new javac warning messages.
        +1 javadoc 9m 57s There were no new javadoc warning messages.
        +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
        -1 checkstyle 1m 3s The applied patch generated 1 new checkstyle issues (total was 218, now 218).
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 33s mvn install still works.
        +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
        +1 findbugs 1m 42s The patch does not introduce any new Findbugs (version 2.0.3) warnings.
        +1 common tests 22m 33s Tests passed in hadoop-common.
            60m 15s  



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12732220/HADOOP-11962.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 8badd82
        checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/6652/artifact/patchprocess/diffcheckstylehadoop-common.txt
        hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/6652/artifact/patchprocess/testrun_hadoop-common.txt
        Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/6652/testReport/
        Java 1.7.0_55
        uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/6652/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 56s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 32s There were no new javac warning messages. +1 javadoc 9m 57s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 3s The applied patch generated 1 new checkstyle issues (total was 218, now 218). +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 33s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 1m 42s The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 common tests 22m 33s Tests passed in hadoop-common.     60m 15s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12732220/HADOOP-11962.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 8badd82 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/6652/artifact/patchprocess/diffcheckstylehadoop-common.txt hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/6652/artifact/patchprocess/testrun_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/6652/testReport/ Java 1.7.0_55 uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/6652/console This message was automatically generated.
        Hide
        djp Junping Du added a comment -

        Simply remove LOG.debug for sasl message. No need unit test to cover this.

        Show
        djp Junping Du added a comment - Simply remove LOG.debug for sasl message. No need unit test to cover this.

          People

          • Assignee:
            djp Junping Du
            Reporter:
            djp Junping Du
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development