Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-12901

Add warning log when KMSClientProvider cannot create a connection to the KMS server

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • None
    • 2.8.0, 3.0.0-alpha1
    • None

    Description

      Currently when failed to connect, one can only see logs at debug level with a vague SocketTimeoutException. (Luckily in the env I saw this, HADOOP-10015 was not present, so the last WARN log was printed.)

      2015-12-17 12:28:01,410 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:hdfs (auth:SIMPLE) from:org.apache.hadoop.crypto.key.kms.KMSClientProvider.createConnection(KMSClientProvider.java:477) 
      2015-12-17 12:28:01,469 WARN org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:hdfs (auth:SIMPLE) cause:java.net.SocketTimeoutException: connect timed out 
      

      The issue was then fixed by opening up the KMS server port.

      This jira is to propose we add specific loggings in this case, to help user easily identify the problem.

      Attachments

        1. HADOOP-12901.01.patch
          2 kB
          Xiao Chen

        Activity

          xiaochen Xiao Chen added a comment -

          Patch 1 adds a WARN log on this. No stacktrace is printed in the log here, because IMO just the host information should be enough.

          I verified the logging by programmatically injecting a SocketTimeoutException in the doAs block, and running a TestKMS test case.

          xiaochen Xiao Chen added a comment - Patch 1 adds a WARN log on this. No stacktrace is printed in the log here, because IMO just the host information should be enough. I verified the logging by programmatically injecting a SocketTimeoutException in the doAs block, and running a TestKMS test case.
          andrew.wang Andrew Wang added a comment -

          +1 LGTM pending Jenkins, thanks Xiao!

          andrew.wang Andrew Wang added a comment - +1 LGTM pending Jenkins, thanks Xiao!
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 11s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 6m 36s trunk passed
          +1 compile 5m 55s trunk passed with JDK v1.8.0_74
          +1 compile 6m 54s trunk passed with JDK v1.7.0_95
          +1 checkstyle 0m 20s trunk passed
          +1 mvnsite 0m 57s trunk passed
          +1 mvneclipse 0m 13s trunk passed
          +1 findbugs 1m 38s trunk passed
          +1 javadoc 0m 51s trunk passed with JDK v1.8.0_74
          +1 javadoc 1m 3s trunk passed with JDK v1.7.0_95
          +1 mvninstall 0m 39s the patch passed
          +1 compile 6m 29s the patch passed with JDK v1.8.0_74
          +1 javac 6m 29s the patch passed
          +1 compile 7m 11s the patch passed with JDK v1.7.0_95
          +1 javac 7m 11s the patch passed
          +1 checkstyle 0m 21s the patch passed
          +1 mvnsite 0m 56s the patch passed
          +1 mvneclipse 0m 13s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 1m 55s the patch passed
          +1 javadoc 1m 1s the patch passed with JDK v1.8.0_74
          +1 javadoc 1m 1s the patch passed with JDK v1.7.0_95
          +1 unit 6m 42s hadoop-common in the patch passed with JDK v1.8.0_74.
          +1 unit 7m 8s hadoop-common in the patch passed with JDK v1.7.0_95.
          +1 asflicense 0m 22s Patch does not generate ASF License warnings.
          59m 45s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12791815/HADOOP-12901.01.patch
          JIRA Issue HADOOP-12901
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux db885f2b7b9f 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 724d229
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8809/testReport/
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8809/console
          Powered by Apache Yetus 0.3.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 11s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 6m 36s trunk passed +1 compile 5m 55s trunk passed with JDK v1.8.0_74 +1 compile 6m 54s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 20s trunk passed +1 mvnsite 0m 57s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 38s trunk passed +1 javadoc 0m 51s trunk passed with JDK v1.8.0_74 +1 javadoc 1m 3s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 39s the patch passed +1 compile 6m 29s the patch passed with JDK v1.8.0_74 +1 javac 6m 29s the patch passed +1 compile 7m 11s the patch passed with JDK v1.7.0_95 +1 javac 7m 11s the patch passed +1 checkstyle 0m 21s the patch passed +1 mvnsite 0m 56s the patch passed +1 mvneclipse 0m 13s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 1m 55s the patch passed +1 javadoc 1m 1s the patch passed with JDK v1.8.0_74 +1 javadoc 1m 1s the patch passed with JDK v1.7.0_95 +1 unit 6m 42s hadoop-common in the patch passed with JDK v1.8.0_74. +1 unit 7m 8s hadoop-common in the patch passed with JDK v1.7.0_95. +1 asflicense 0m 22s Patch does not generate ASF License warnings. 59m 45s Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12791815/HADOOP-12901.01.patch JIRA Issue HADOOP-12901 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux db885f2b7b9f 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 724d229 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8809/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8809/console Powered by Apache Yetus 0.3.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          andrew.wang Andrew Wang added a comment -

          Pushed to trunk, branch-2, branch-2.8. Thanks Xiao for the find and fix!

          andrew.wang Andrew Wang added a comment - Pushed to trunk, branch-2, branch-2.8. Thanks Xiao for the find and fix!
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #9439 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9439/)
          HADOOP-12901. Add warning log when KMSClientProvider cannot create a (wang: rev 391da36d93358038c50c15d91543f6c765fa0471)

          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/kms/KMSClientProvider.java
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #9439 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9439/ ) HADOOP-12901 . Add warning log when KMSClientProvider cannot create a (wang: rev 391da36d93358038c50c15d91543f6c765fa0471) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/kms/KMSClientProvider.java
          xiaochen Xiao Chen added a comment -

          Thanks Andrew for the prompt review and commit.

          xiaochen Xiao Chen added a comment - Thanks Andrew for the prompt review and commit.

          People

            xiaochen Xiao Chen
            xiaochen Xiao Chen
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: