Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-12901

Add warning log when KMSClientProvider cannot create a connection to the KMS server

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: None
    • Labels:

      Description

      Currently when failed to connect, one can only see logs at debug level with a vague SocketTimeoutException. (Luckily in the env I saw this, HADOOP-10015 was not present, so the last WARN log was printed.)

      2015-12-17 12:28:01,410 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:hdfs (auth:SIMPLE) from:org.apache.hadoop.crypto.key.kms.KMSClientProvider.createConnection(KMSClientProvider.java:477) 
      2015-12-17 12:28:01,469 WARN org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:hdfs (auth:SIMPLE) cause:java.net.SocketTimeoutException: connect timed out 
      

      The issue was then fixed by opening up the KMS server port.

      This jira is to propose we add specific loggings in this case, to help user easily identify the problem.

        Activity

        Hide
        xiaochen Xiao Chen added a comment -

        Patch 1 adds a WARN log on this. No stacktrace is printed in the log here, because IMO just the host information should be enough.

        I verified the logging by programmatically injecting a SocketTimeoutException in the doAs block, and running a TestKMS test case.

        Show
        xiaochen Xiao Chen added a comment - Patch 1 adds a WARN log on this. No stacktrace is printed in the log here, because IMO just the host information should be enough. I verified the logging by programmatically injecting a SocketTimeoutException in the doAs block, and running a TestKMS test case.
        Hide
        andrew.wang Andrew Wang added a comment -

        +1 LGTM pending Jenkins, thanks Xiao!

        Show
        andrew.wang Andrew Wang added a comment - +1 LGTM pending Jenkins, thanks Xiao!
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 11s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 6m 36s trunk passed
        +1 compile 5m 55s trunk passed with JDK v1.8.0_74
        +1 compile 6m 54s trunk passed with JDK v1.7.0_95
        +1 checkstyle 0m 20s trunk passed
        +1 mvnsite 0m 57s trunk passed
        +1 mvneclipse 0m 13s trunk passed
        +1 findbugs 1m 38s trunk passed
        +1 javadoc 0m 51s trunk passed with JDK v1.8.0_74
        +1 javadoc 1m 3s trunk passed with JDK v1.7.0_95
        +1 mvninstall 0m 39s the patch passed
        +1 compile 6m 29s the patch passed with JDK v1.8.0_74
        +1 javac 6m 29s the patch passed
        +1 compile 7m 11s the patch passed with JDK v1.7.0_95
        +1 javac 7m 11s the patch passed
        +1 checkstyle 0m 21s the patch passed
        +1 mvnsite 0m 56s the patch passed
        +1 mvneclipse 0m 13s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 1m 55s the patch passed
        +1 javadoc 1m 1s the patch passed with JDK v1.8.0_74
        +1 javadoc 1m 1s the patch passed with JDK v1.7.0_95
        +1 unit 6m 42s hadoop-common in the patch passed with JDK v1.8.0_74.
        +1 unit 7m 8s hadoop-common in the patch passed with JDK v1.7.0_95.
        +1 asflicense 0m 22s Patch does not generate ASF License warnings.
        59m 45s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12791815/HADOOP-12901.01.patch
        JIRA Issue HADOOP-12901
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux db885f2b7b9f 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 724d229
        Default Java 1.7.0_95
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
        findbugs v3.0.0
        JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8809/testReport/
        modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8809/console
        Powered by Apache Yetus 0.3.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 11s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 6m 36s trunk passed +1 compile 5m 55s trunk passed with JDK v1.8.0_74 +1 compile 6m 54s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 20s trunk passed +1 mvnsite 0m 57s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 38s trunk passed +1 javadoc 0m 51s trunk passed with JDK v1.8.0_74 +1 javadoc 1m 3s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 39s the patch passed +1 compile 6m 29s the patch passed with JDK v1.8.0_74 +1 javac 6m 29s the patch passed +1 compile 7m 11s the patch passed with JDK v1.7.0_95 +1 javac 7m 11s the patch passed +1 checkstyle 0m 21s the patch passed +1 mvnsite 0m 56s the patch passed +1 mvneclipse 0m 13s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 1m 55s the patch passed +1 javadoc 1m 1s the patch passed with JDK v1.8.0_74 +1 javadoc 1m 1s the patch passed with JDK v1.7.0_95 +1 unit 6m 42s hadoop-common in the patch passed with JDK v1.8.0_74. +1 unit 7m 8s hadoop-common in the patch passed with JDK v1.7.0_95. +1 asflicense 0m 22s Patch does not generate ASF License warnings. 59m 45s Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12791815/HADOOP-12901.01.patch JIRA Issue HADOOP-12901 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux db885f2b7b9f 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 724d229 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8809/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8809/console Powered by Apache Yetus 0.3.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        andrew.wang Andrew Wang added a comment -

        Pushed to trunk, branch-2, branch-2.8. Thanks Xiao for the find and fix!

        Show
        andrew.wang Andrew Wang added a comment - Pushed to trunk, branch-2, branch-2.8. Thanks Xiao for the find and fix!
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #9439 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9439/)
        HADOOP-12901. Add warning log when KMSClientProvider cannot create a (wang: rev 391da36d93358038c50c15d91543f6c765fa0471)

        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/kms/KMSClientProvider.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #9439 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9439/ ) HADOOP-12901 . Add warning log when KMSClientProvider cannot create a (wang: rev 391da36d93358038c50c15d91543f6c765fa0471) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/kms/KMSClientProvider.java
        Hide
        xiaochen Xiao Chen added a comment -

        Thanks Andrew for the prompt review and commit.

        Show
        xiaochen Xiao Chen added a comment - Thanks Andrew for the prompt review and commit.

          People

          • Assignee:
            xiaochen Xiao Chen
            Reporter:
            xiaochen Xiao Chen
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development