Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-13890

Maintain HTTP/host as SPNEGO SPN support and fix KerberosName parsing

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha2
    • Component/s: test
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      HADOOP-13565 introduced an incompatible check that disallowed principal like HTTP/host from being used as SPNEGO SPN.
      This breaks the following test in trunk: TestWebDelegationToken, TestKMS , TestTrashWithSecureEncryptionZones and TestSecureEncryptionZoneWithKMS because they used HTTP/localhost as SPNEGO SPN assuming the default realm. This ticket is opened to bring back the support of HTTP/host as valid SPNEGO SPN.

      KerberosName parsing bug was discovered, fixed and included as a necessary part of this ticket along with additional unit test to cover parsing different form of principals.

      Jenkins URL
      https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/251/testReport/
      https://builds.apache.org/job/PreCommit-HADOOP-Build/11240/testReport/

      1. HADOOP-13890.00.patch
        9 kB
        Xiaoyu Yao
      2. HADOOP-13890.01.patch
        11 kB
        Xiaoyu Yao
      3. HADOOP-13890.02.patch
        4 kB
        Xiaoyu Yao
      4. HADOOP-13890.03.patch
        4 kB
        Xiaoyu Yao
      5. test-failure.txt
        19 kB
        Yuanbo Liu
      6. HADOOP-13890.04.patch
        5 kB
        Xiaoyu Yao
      7. test_failure_1.txt
        34 kB
        Yuanbo Liu
      8. HADOOP-13890.05.patch
        7 kB
        Xiaoyu Yao

        Issue Links

          Activity

          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10998 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10998/)
          HADOOP-13890. Maintain HTTP/host as SPNEGO SPN support and fix (xyao: rev f5e0bd30fde654ed48fe73e5c0523030365385a4)

          • (edit) hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/server/KerberosAuthenticationHandler.java
          • (edit) hadoop-common-project/hadoop-auth/src/test/java/org/apache/hadoop/security/authentication/util/TestKerberosName.java
          • (edit) hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/util/KerberosName.java
          • (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/token/delegation/web/TestWebDelegationToken.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10998 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10998/ ) HADOOP-13890 . Maintain HTTP/host as SPNEGO SPN support and fix (xyao: rev f5e0bd30fde654ed48fe73e5c0523030365385a4) (edit) hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/server/KerberosAuthenticationHandler.java (edit) hadoop-common-project/hadoop-auth/src/test/java/org/apache/hadoop/security/authentication/util/TestKerberosName.java (edit) hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/util/KerberosName.java (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/token/delegation/web/TestWebDelegationToken.java
          Hide
          xyao Xiaoyu Yao added a comment -

          Thanks all for the reviews and discussions. I've commit the patch to trunk, branch-2 and branch-2.8

          I also updated the title/description of the incompatible issue introduced by HADOOP-13565 and fixed with this ticket.

          Show
          xyao Xiaoyu Yao added a comment - Thanks all for the reviews and discussions. I've commit the patch to trunk, branch-2 and branch-2.8 I also updated the title/description of the incompatible issue introduced by HADOOP-13565 and fixed with this ticket.
          Hide
          xyao Xiaoyu Yao added a comment - - edited

          Brahma Reddy Battula, thanks for the review and tried the fix on windows.
          The windows failure is not related to HADOOP-13565 or HADOOP-13890. I tried the TestWebDelegationToken with both changes reverted on Windows. It failed with the same error due to an issue with KerberosUtil#getDefaultRealm(). I filed a follow up ticket HADOOP-13907 to investigate and fix KerberosUtil#getDefaultRealm() on Windows.

          java.lang.IllegalArgumentException: Can't get Kerberos realm
          
          	at org.apache.hadoop.security.HadoopKerberosName.setConfiguration(HadoopKerberosName.java:65)
          	at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:305)
          	at org.apache.hadoop.security.UserGroupInformation.setConfiguration(UserGroupInformation.java:351)
          	at org.apache.hadoop.security.token.delegation.web.TestWebDelegationToken.testKerberosDelegationTokenAuthenticator(TestWebDelegationToken.java:746)
          	at org.apache.hadoop.security.token.delegation.web.TestWebDelegationToken.testKerberosDelegationTokenAuthenticator(TestWebDelegationToken.java:729)
          	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
          	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
          	at java.lang.reflect.Method.invoke(Method.java:498)
          	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
          	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
          	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
          	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
          	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
          	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
          	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
          	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
          	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
          	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
          	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
          	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
          	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
          	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
          	at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
          	at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
          	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:117)
          	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:42)
          	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:262)
          	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:84)
          	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
          	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
          	at java.lang.reflect.Method.invoke(Method.java:498)
          	at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147)
          Caused by: java.lang.reflect.InvocationTargetException
          	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
          	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
          	at java.lang.reflect.Method.invoke(Method.java:498)
          	at org.apache.hadoop.security.authentication.util.KerberosUtil.getDefaultRealm(KerberosUtil.java:88)
          	at org.apache.hadoop.security.HadoopKerberosName.setConfiguration(HadoopKerberosName.java:63)
          	... 33 more
          Caused by: KrbException: Cannot locate default realm
          	at sun.security.krb5.Config.getDefaultRealm(Config.java:1029)
          	... 39 more
          
          Show
          xyao Xiaoyu Yao added a comment - - edited Brahma Reddy Battula , thanks for the review and tried the fix on windows. The windows failure is not related to HADOOP-13565 or HADOOP-13890 . I tried the TestWebDelegationToken with both changes reverted on Windows. It failed with the same error due to an issue with KerberosUtil#getDefaultRealm(). I filed a follow up ticket HADOOP-13907 to investigate and fix KerberosUtil#getDefaultRealm() on Windows. java.lang.IllegalArgumentException: Can't get Kerberos realm at org.apache.hadoop.security.HadoopKerberosName.setConfiguration(HadoopKerberosName.java:65) at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:305) at org.apache.hadoop.security.UserGroupInformation.setConfiguration(UserGroupInformation.java:351) at org.apache.hadoop.security.token.delegation.web.TestWebDelegationToken.testKerberosDelegationTokenAuthenticator(TestWebDelegationToken.java:746) at org.apache.hadoop.security.token.delegation.web.TestWebDelegationToken.testKerberosDelegationTokenAuthenticator(TestWebDelegationToken.java:729) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229) at org.junit.runners.ParentRunner.run(ParentRunner.java:309) at org.junit.runner.JUnitCore.run(JUnitCore.java:160) at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:117) at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:42) at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:262) at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:84) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147) Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.security.authentication.util.KerberosUtil.getDefaultRealm(KerberosUtil.java:88) at org.apache.hadoop.security.HadoopKerberosName.setConfiguration(HadoopKerberosName.java:63) ... 33 more Caused by: KrbException: Cannot locate default realm at sun.security.krb5.Config.getDefaultRealm(Config.java:1029) ... 39 more
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          My late +1, Xiaoyu Yao thanks for taking care..Now it's addressed the incompatible which is introduced in HADOOP-13565 , Daryn Sharp do you think same..?

          Still tests can still fail in windows,may we can investigate further in follow up jira's..

          I feel,,,,It will be always good to run all the tests when we change common part ( which we feel,it can impact other projects).May be we change one line each project,Let jenkins run in all the projects which could have avoid this jira..

          Show
          brahmareddy Brahma Reddy Battula added a comment - My late +1, Xiaoyu Yao thanks for taking care..Now it's addressed the incompatible which is introduced in HADOOP-13565 , Daryn Sharp do you think same..? Still tests can still fail in windows,may we can investigate further in follow up jira's.. I feel,,,,It will be always good to run all the tests when we change common part ( which we feel,it can impact other projects).May be we change one line each project,Let jenkins run in all the projects which could have avoid this jira..
          Hide
          xyao Xiaoyu Yao added a comment -

          cc: Brahma Reddy Battula and Daryn Sharp, I plan to commit HADOOP-13890 shortly to unblock Jenkins and QE/test pipelines.
          If you have additional comments regarding HADOOP-13565, we can discuss and address them with followup JIRAs.

          Show
          xyao Xiaoyu Yao added a comment - cc: Brahma Reddy Battula and Daryn Sharp , I plan to commit HADOOP-13890 shortly to unblock Jenkins and QE/test pipelines. If you have additional comments regarding HADOOP-13565 , we can discuss and address them with followup JIRAs.
          Hide
          arpitagarwal Arpit Agarwal added a comment -

          +1 for the v5 patch.

          Show
          arpitagarwal Arpit Agarwal added a comment - +1 for the v5 patch.
          Hide
          xyao Xiaoyu Yao added a comment -

          Since John Zhuge, you and Apache Jenkins have reported that the test failures are passed, I'm strongly doubting these failures are related to my laptop environment. Please go ahead if you're suspended by my comment

          Agree. We can investigate your case as a separate issue.
          Appreciate if folks on the watch list can review this fix to unblock Jenkins. Thanks in advance!

          Show
          xyao Xiaoyu Yao added a comment - Since John Zhuge, you and Apache Jenkins have reported that the test failures are passed, I'm strongly doubting these failures are related to my laptop environment. Please go ahead if you're suspended by my comment Agree. We can investigate your case as a separate issue. Appreciate if folks on the watch list can review this fix to unblock Jenkins. Thanks in advance!
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 22s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
          0 mvndep 0m 8s Maven dependency ordering for branch
          +1 mvninstall 7m 17s trunk passed
          +1 compile 9m 33s trunk passed
          +1 checkstyle 0m 32s trunk passed
          +1 mvnsite 1m 27s trunk passed
          +1 mvneclipse 0m 36s trunk passed
          +1 findbugs 1m 54s trunk passed
          +1 javadoc 1m 5s trunk passed
          0 mvndep 0m 7s Maven dependency ordering for patch
          +1 mvninstall 0m 52s the patch passed
          +1 compile 9m 37s the patch passed
          +1 javac 9m 37s the patch passed
          -0 checkstyle 0m 33s hadoop-common-project: The patch generated 2 new + 66 unchanged - 1 fixed = 68 total (was 67)
          +1 mvnsite 1m 24s the patch passed
          +1 mvneclipse 0m 36s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 2m 9s the patch passed
          +1 javadoc 1m 6s the patch passed
          +1 unit 3m 33s hadoop-auth in the patch passed.
          -1 unit 8m 15s hadoop-common in the patch failed.
          +1 asflicense 0m 32s The patch does not generate ASF License warnings.
          56m 7s



          Reason Tests
          Failed junit tests hadoop.fs.viewfs.TestViewFsTrash
            hadoop.ha.TestZKFailoverController
            hadoop.net.TestDNS



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HADOOP-13890
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12843256/HADOOP-13890.05.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux f64fccf9a37b 3.13.0-103-generic #150-Ubuntu SMP Thu Nov 24 10:34:17 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 72bff19
          Default Java 1.8.0_111
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/11276/artifact/patchprocess/diff-checkstyle-hadoop-common-project.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/11276/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11276/testReport/
          modules C: hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-common U: hadoop-common-project
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11276/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 22s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. 0 mvndep 0m 8s Maven dependency ordering for branch +1 mvninstall 7m 17s trunk passed +1 compile 9m 33s trunk passed +1 checkstyle 0m 32s trunk passed +1 mvnsite 1m 27s trunk passed +1 mvneclipse 0m 36s trunk passed +1 findbugs 1m 54s trunk passed +1 javadoc 1m 5s trunk passed 0 mvndep 0m 7s Maven dependency ordering for patch +1 mvninstall 0m 52s the patch passed +1 compile 9m 37s the patch passed +1 javac 9m 37s the patch passed -0 checkstyle 0m 33s hadoop-common-project: The patch generated 2 new + 66 unchanged - 1 fixed = 68 total (was 67) +1 mvnsite 1m 24s the patch passed +1 mvneclipse 0m 36s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 2m 9s the patch passed +1 javadoc 1m 6s the patch passed +1 unit 3m 33s hadoop-auth in the patch passed. -1 unit 8m 15s hadoop-common in the patch failed. +1 asflicense 0m 32s The patch does not generate ASF License warnings. 56m 7s Reason Tests Failed junit tests hadoop.fs.viewfs.TestViewFsTrash   hadoop.ha.TestZKFailoverController   hadoop.net.TestDNS Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HADOOP-13890 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12843256/HADOOP-13890.05.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux f64fccf9a37b 3.13.0-103-generic #150-Ubuntu SMP Thu Nov 24 10:34:17 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 72bff19 Default Java 1.8.0_111 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/11276/artifact/patchprocess/diff-checkstyle-hadoop-common-project.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/11276/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11276/testReport/ modules C: hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-common U: hadoop-common-project Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11276/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          xyao Xiaoyu Yao added a comment - - edited

          Yuanbo Liu, here is what happened in your case.

          1. hostname localhost is mapped to principal HTTP/localhost during KerberosAuthenticationHandler.java:init.

          2016-12-14 15:48:34,459 TRACE server.KerberosAuthenticationHandler (KerberosAuthenticationHandler.java:init(279)) - Map server: localhost to principal: HTTP/localhost
          

          2. authenticate request comes in

          2016-12-14 15:48:34,482 TRACE server.KerberosAuthenticationHandler (KerberosAuthenticationHandler.java:authenticate(400)) - SPNEGO starting for url: http://localhost:39910/foo/bar
          

          3. The localhost to principal lookup somehow failed with an empty principal as shown below, which failed the test.

          2016-12-14 15:48:34,495 TRACE server.KerberosAuthenticationHandler (KerberosAuthenticationHandler.java:run(421)) - SPNEGO with principals: []
          

          The only difference is in all the pass cases the HashMap lookup successfully find the right principal. I can't see obvious reason why the single principle is not being added into the HashMap during init(). I attach a new patch with additional tracing. Yuanbo Liu, can you try it out and post the result?

          2016-12-13 21:12:43,918 TRACE server.KerberosAuthenticationHandler (KerberosAuthenticationHandler.java:run(421)) - SPNEGO with principals: [HTTP/localhost]
          
          Show
          xyao Xiaoyu Yao added a comment - - edited Yuanbo Liu , here is what happened in your case. 1. hostname localhost is mapped to principal HTTP/localhost during KerberosAuthenticationHandler.java:init. 2016-12-14 15:48:34,459 TRACE server.KerberosAuthenticationHandler (KerberosAuthenticationHandler.java:init(279)) - Map server: localhost to principal: HTTP/localhost 2. authenticate request comes in 2016-12-14 15:48:34,482 TRACE server.KerberosAuthenticationHandler (KerberosAuthenticationHandler.java:authenticate(400)) - SPNEGO starting for url: http: //localhost:39910/foo/bar 3. The localhost to principal lookup somehow failed with an empty principal as shown below, which failed the test. 2016-12-14 15:48:34,495 TRACE server.KerberosAuthenticationHandler (KerberosAuthenticationHandler.java:run(421)) - SPNEGO with principals: [] The only difference is in all the pass cases the HashMap lookup successfully find the right principal. I can't see obvious reason why the single principle is not being added into the HashMap during init(). I attach a new patch with additional tracing. Yuanbo Liu , can you try it out and post the result? 2016-12-13 21:12:43,918 TRACE server.KerberosAuthenticationHandler (KerberosAuthenticationHandler.java:run(421)) - SPNEGO with principals: [HTTP/localhost]
          Hide
          yuanbo Yuanbo Liu added a comment -

          Xiaoyu Yao Thanks for your new patch and explanation.
          My java version is oracle-1.8, the details are here:

          java version "1.8.0_101"
          Java(TM) SE Runtime Environment (build 1.8.0_101-b13)
          Java HotSpot(TM) 64-Bit Server VM (build 25.101-b13, mixed mode)
          
          ll |grep java_sdk_1.8.0
          lrwxrwxrwx. 1 root root 58 Sep  6 16:06 java_sdk_1.8.0 -> /usr/lib/jvm/java-1.8.0-oracle-1.8.0.101-1jpp.1.el7.x86_64
          

          The exception from IBM JDK seems not to be "Invalid SPNEGO sequence" exception.
          FYI, I have attached a new log file: test_failure_1.txt.
          Since John Zhuge, you and Apache Jenkins have reported that the test failures are passed, I'm strongly doubting these failures are related to my laptop environment. Please go ahead if you're suspended by my comment.

          Show
          yuanbo Yuanbo Liu added a comment - Xiaoyu Yao Thanks for your new patch and explanation. My java version is oracle-1.8, the details are here: java version "1.8.0_101" Java(TM) SE Runtime Environment (build 1.8.0_101-b13) Java HotSpot(TM) 64-Bit Server VM (build 25.101-b13, mixed mode) ll |grep java_sdk_1.8.0 lrwxrwxrwx. 1 root root 58 Sep 6 16:06 java_sdk_1.8.0 -> /usr/lib/jvm/java-1.8.0-oracle-1.8.0.101-1jpp.1.el7.x86_64 The exception from IBM JDK seems not to be "Invalid SPNEGO sequence" exception. FYI, I have attached a new log file: test_failure_1.txt. Since John Zhuge , you and Apache Jenkins have reported that the test failures are passed, I'm strongly doubting these failures are related to my laptop environment. Please go ahead if you're suspended by my comment.
          Hide
          xyao Xiaoyu Yao added a comment -

          Yuanbo Liu, the issue you hit is IBM JDK specific. TestWebDelegationToken was failing even without HADOOP-13565.
          Based on that, I think the failure is not related to either HADOOP-13565 or HADOOP-13890 we are trying to solve here.
          If you want, we could fix IBM JDK issue for TestWebDelegationToken in a separate ticket later.

          IBM JDK 8

          [root@c6404 hadoop]# /opt/ibm/java-x86_64-80/bin/java -version
          java version "1.8.0"
          Java(TM) SE Runtime Environment (build pxa6480sr3fp12-20160919_01(SR3 FP12))
          IBM J9 VM (build 2.8, JRE 1.8.0 Linux amd64-64 Compressed References 20160915_318796 (JIT enabled, AOT enabled)
          J9VM - R28_Java8_SR3_20160915_0912_B318796
          JIT  - tr.r14.java.green_20160818_122998
          GC   - R28_Java8_SR3_20160915_0912_B318796_CMPRSS
          J9CL - 20160915_318796)
          JCL - 20160914_01 based on Oracle jdk8u101-b13
          

          Git info after revert HADOOP-13565.

          commit b5a719486112fa1ca60bee5eec81f41a0828b928
          Author: root <root@c6404.ambari.apache.org>
          Date:   Wed Dec 14 07:10:36 2016 +0000
              Revert "HADOOP-13565. KerberosAuthenticationHandler#authenticate should not rebuild SPN based on client request. Contributed by Xiaoyu Yao."
              
              This reverts commit 4c38f11cec0664b70e52f9563052dca8fb17c33f.
          

          The test still failed even after revert HADOOP-13565.

          Tests run: 12, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 10.748 sec <<< FAILURE! - in org.apache.hadoop.security.token.delegation.web.TestWebDelegationToken
          testKerberosDelegationTokenAuthenticator(org.apache.hadoop.security.token.delegation.web.TestWebDelegationToken)  Time elapsed: 1.571 sec  <<< ERROR!
          javax.security.auth.login.LoginException: Bad JAAS configuration: unrecognized option: isInitiator
                  at com.ibm.security.jgss.i18n.I18NException.throwLoginException(I18NException.java:23)
                  at com.ibm.security.auth.module.Krb5LoginModule.d(Krb5LoginModule.java:57)
                  at com.ibm.security.auth.module.Krb5LoginModule.a(Krb5LoginModule.java:686)
                  at com.ibm.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:214)
                  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
                  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:95)
                  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
                  at java.lang.reflect.Method.invoke(Method.java:508)
                  at javax.security.auth.login.LoginContext.invoke(LoginContext.java:788)
                  at javax.security.auth.login.LoginContext.access$000(LoginContext.java:196)
                  at javax.security.auth.login.LoginContext$5.run(LoginContext.java:721)
                  at javax.security.auth.login.LoginContext$5.run(LoginContext.java:719)
                  at java.security.AccessController.doPrivileged(AccessController.java:686)
                  at javax.security.auth.login.LoginContext.invokeCreatorPriv(LoginContext.java:719)
                  at javax.security.auth.login.LoginContext.login(LoginContext.java:593)
                  at org.apache.hadoop.security.token.delegation.web.TestWebDelegationToken.doAsKerberosUser(TestWebDelegationToken.java:710)
                  at org.apache.hadoop.security.token.delegation.web.TestWebDelegationToken.testKerberosDelegationTokenAuthenticator(TestWebDelegationToken.java:778)
                  at org.apache.hadoop.security.token.delegation.web.TestWebDelegationToken.testKerberosDelegationTokenAuthenticator(TestWebDelegationToken.java:729)
          
          Show
          xyao Xiaoyu Yao added a comment - Yuanbo Liu , the issue you hit is IBM JDK specific. TestWebDelegationToken was failing even without HADOOP-13565 . Based on that, I think the failure is not related to either HADOOP-13565 or HADOOP-13890 we are trying to solve here. If you want, we could fix IBM JDK issue for TestWebDelegationToken in a separate ticket later. IBM JDK 8 [root@c6404 hadoop]# /opt/ibm/java-x86_64-80/bin/java -version java version "1.8.0" Java(TM) SE Runtime Environment (build pxa6480sr3fp12-20160919_01(SR3 FP12)) IBM J9 VM (build 2.8, JRE 1.8.0 Linux amd64-64 Compressed References 20160915_318796 (JIT enabled, AOT enabled) J9VM - R28_Java8_SR3_20160915_0912_B318796 JIT - tr.r14.java.green_20160818_122998 GC - R28_Java8_SR3_20160915_0912_B318796_CMPRSS J9CL - 20160915_318796) JCL - 20160914_01 based on Oracle jdk8u101-b13 Git info after revert HADOOP-13565 . commit b5a719486112fa1ca60bee5eec81f41a0828b928 Author: root <root@c6404.ambari.apache.org> Date: Wed Dec 14 07:10:36 2016 +0000 Revert "HADOOP-13565. KerberosAuthenticationHandler#authenticate should not rebuild SPN based on client request. Contributed by Xiaoyu Yao." This reverts commit 4c38f11cec0664b70e52f9563052dca8fb17c33f. The test still failed even after revert HADOOP-13565 . Tests run: 12, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 10.748 sec <<< FAILURE! - in org.apache.hadoop.security.token.delegation.web.TestWebDelegationToken testKerberosDelegationTokenAuthenticator(org.apache.hadoop.security.token.delegation.web.TestWebDelegationToken) Time elapsed: 1.571 sec <<< ERROR! javax.security.auth.login.LoginException: Bad JAAS configuration: unrecognized option: isInitiator at com.ibm.security.jgss.i18n.I18NException.throwLoginException(I18NException.java:23) at com.ibm.security.auth.module.Krb5LoginModule.d(Krb5LoginModule.java:57) at com.ibm.security.auth.module.Krb5LoginModule.a(Krb5LoginModule.java:686) at com.ibm.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:214) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:95) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55) at java.lang.reflect.Method.invoke(Method.java:508) at javax.security.auth.login.LoginContext.invoke(LoginContext.java:788) at javax.security.auth.login.LoginContext.access$000(LoginContext.java:196) at javax.security.auth.login.LoginContext$5.run(LoginContext.java:721) at javax.security.auth.login.LoginContext$5.run(LoginContext.java:719) at java.security.AccessController.doPrivileged(AccessController.java:686) at javax.security.auth.login.LoginContext.invokeCreatorPriv(LoginContext.java:719) at javax.security.auth.login.LoginContext.login(LoginContext.java:593) at org.apache.hadoop.security.token.delegation.web.TestWebDelegationToken.doAsKerberosUser(TestWebDelegationToken.java:710) at org.apache.hadoop.security.token.delegation.web.TestWebDelegationToken.testKerberosDelegationTokenAuthenticator(TestWebDelegationToken.java:778) at org.apache.hadoop.security.token.delegation.web.TestWebDelegationToken.testKerberosDelegationTokenAuthenticator(TestWebDelegationToken.java:729)
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 15s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
          0 mvndep 0m 8s Maven dependency ordering for branch
          +1 mvninstall 6m 59s trunk passed
          +1 compile 10m 8s trunk passed
          +1 checkstyle 0m 33s trunk passed
          +1 mvnsite 1m 30s trunk passed
          +1 mvneclipse 0m 36s trunk passed
          +1 findbugs 1m 51s trunk passed
          +1 javadoc 1m 6s trunk passed
          0 mvndep 0m 7s Maven dependency ordering for patch
          +1 mvninstall 0m 53s the patch passed
          +1 compile 9m 50s the patch passed
          +1 javac 9m 50s the patch passed
          -0 checkstyle 0m 36s hadoop-common-project: The patch generated 1 new + 66 unchanged - 1 fixed = 67 total (was 67)
          +1 mvnsite 1m 35s the patch passed
          +1 mvneclipse 0m 36s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 2m 37s the patch passed
          +1 javadoc 1m 2s the patch passed
          +1 unit 3m 35s hadoop-auth in the patch passed.
          -1 unit 9m 51s hadoop-common in the patch failed.
          +1 asflicense 0m 35s The patch does not generate ASF License warnings.
          58m 47s



          Reason Tests
          Failed junit tests hadoop.ha.TestZKFailoverController
            hadoop.net.TestDNS



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HADOOP-13890
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12843154/HADOOP-13890.04.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 8836f9967b6d 3.13.0-103-generic #150-Ubuntu SMP Thu Nov 24 10:34:17 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / ada876c
          Default Java 1.8.0_111
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/11270/artifact/patchprocess/diff-checkstyle-hadoop-common-project.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/11270/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11270/testReport/
          modules C: hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-common U: hadoop-common-project
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11270/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. 0 mvndep 0m 8s Maven dependency ordering for branch +1 mvninstall 6m 59s trunk passed +1 compile 10m 8s trunk passed +1 checkstyle 0m 33s trunk passed +1 mvnsite 1m 30s trunk passed +1 mvneclipse 0m 36s trunk passed +1 findbugs 1m 51s trunk passed +1 javadoc 1m 6s trunk passed 0 mvndep 0m 7s Maven dependency ordering for patch +1 mvninstall 0m 53s the patch passed +1 compile 9m 50s the patch passed +1 javac 9m 50s the patch passed -0 checkstyle 0m 36s hadoop-common-project: The patch generated 1 new + 66 unchanged - 1 fixed = 67 total (was 67) +1 mvnsite 1m 35s the patch passed +1 mvneclipse 0m 36s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 2m 37s the patch passed +1 javadoc 1m 2s the patch passed +1 unit 3m 35s hadoop-auth in the patch passed. -1 unit 9m 51s hadoop-common in the patch failed. +1 asflicense 0m 35s The patch does not generate ASF License warnings. 58m 47s Reason Tests Failed junit tests hadoop.ha.TestZKFailoverController   hadoop.net.TestDNS Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HADOOP-13890 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12843154/HADOOP-13890.04.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 8836f9967b6d 3.13.0-103-generic #150-Ubuntu SMP Thu Nov 24 10:34:17 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / ada876c Default Java 1.8.0_111 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/11270/artifact/patchprocess/diff-checkstyle-hadoop-common-project.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/11270/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11270/testReport/ modules C: hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-common U: hadoop-common-project Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11270/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          xyao Xiaoyu Yao added a comment -

          Attach patch v04 to enable KerberosAuthenticationHandler trace log for tests in TestWebDelegationToken.java.

          Show
          xyao Xiaoyu Yao added a comment - Attach patch v04 to enable KerberosAuthenticationHandler trace log for tests in TestWebDelegationToken.java.
          Hide
          xyao Xiaoyu Yao added a comment -

          Yuanbo Liu, do you use Oracle JDK or IBM JDK? What's your JDK version? It passed on my machine with Oracle JDK 1.8.

          Can you enable trace log for KerberosAuthenticatonHandler during the test run by
          1. adding the following to the end of TestWebDelegationToken#setUp()

              GenericTestUtils.setLogLevel(KerberosAuthenticationHandler.LOG, Level.TRACE);
          

          2. change KerberosAuthenticationHandler.LOG to public in KerberosAuthenticator.java.
          and reattach the log with additional trace enabled? This will reveal the cause of SPNEGO failure from server side. Thanks in advance!

          Show
          xyao Xiaoyu Yao added a comment - Yuanbo Liu , do you use Oracle JDK or IBM JDK? What's your JDK version? It passed on my machine with Oracle JDK 1.8. Can you enable trace log for KerberosAuthenticatonHandler during the test run by 1. adding the following to the end of TestWebDelegationToken#setUp() GenericTestUtils.setLogLevel(KerberosAuthenticationHandler.LOG, Level.TRACE); 2. change KerberosAuthenticationHandler.LOG to public in KerberosAuthenticator.java. and reattach the log with additional trace enabled? This will reveal the cause of SPNEGO failure from server side. Thanks in advance!
          Hide
          yuanbo Yuanbo Liu added a comment -

          Xiaoyu Yao Thanks for your response.
          I've attached my test failure information and this is my git status info:

          # Changes not staged for commit:
          #   (use "git add <file>..." to update what will be committed)
          #   (use "git checkout -- <file>..." to discard changes in working directory)
          #
          #	modified:   hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/server/KerberosAuthenticationHandler.java
          #	modified:   hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/util/KerberosName.java
          #	modified:   hadoop-common-project/hadoop-auth/src/test/java/org/apache/hadoop/security/authentication/util/TestKerberosName.java
          #
          no changes added to commit (use "git add" and/or "git commit -a")
          
          Show
          yuanbo Yuanbo Liu added a comment - Xiaoyu Yao Thanks for your response. I've attached my test failure information and this is my git status info: # Changes not staged for commit: # (use "git add <file>..." to update what will be committed) # (use "git checkout -- <file>..." to discard changes in working directory) # # modified: hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/server/KerberosAuthenticationHandler.java # modified: hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/util/KerberosName.java # modified: hadoop-common-project/hadoop-auth/src/test/java/org/apache/hadoop/security/authentication/util/TestKerberosName.java # no changes added to commit (use "git add" and/or "git commit -a" )
          Hide
          xyao Xiaoyu Yao added a comment -

          Yuanbo Liu, thanks for trying the patch. Can you post or attach the test logs of the failed test? Here is the result on my local machine after v3 patch, which has all passed.

          -------------------------------------------------------
           T E S T S
          -------------------------------------------------------
          Running org.apache.hadoop.security.token.delegation.web.TestWebDelegationToken
          Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.559 sec - in org.apache.hadoop.security.token.delegation.web.TestWebDelegationToken
          
          Results :
          
          Tests run: 12, Failures: 0, Errors: 0, Skipped: 0
          
          
          Show
          xyao Xiaoyu Yao added a comment - Yuanbo Liu , thanks for trying the patch. Can you post or attach the test logs of the failed test? Here is the result on my local machine after v3 patch, which has all passed. ------------------------------------------------------- T E S T S ------------------------------------------------------- Running org.apache.hadoop.security.token.delegation.web.TestWebDelegationToken Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.559 sec - in org.apache.hadoop.security.token.delegation.web.TestWebDelegationToken Results : Tests run: 12, Failures: 0, Errors: 0, Skipped: 0
          Hide
          yuanbo Yuanbo Liu added a comment -

          Xiaoyu Yao Thanks for working on this JIRA.
          After applying your v3 patch to my local trunk branch, the "Invalid SPNEGO sequence" exception still exist in TestWebDelegationToken.
          Have I missed something?

          Show
          yuanbo Yuanbo Liu added a comment - Xiaoyu Yao Thanks for working on this JIRA. After applying your v3 patch to my local trunk branch, the "Invalid SPNEGO sequence" exception still exist in TestWebDelegationToken . Have I missed something?
          Hide
          jzhuge John Zhuge added a comment -

          +1 LGTM (non-binding). All hadoop-kms and hadoop-httpfs tests passed.

          Show
          jzhuge John Zhuge added a comment - +1 LGTM (non-binding). All hadoop-kms and hadoop-httpfs tests passed.
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 15s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 6m 42s trunk passed
          +1 compile 9m 29s trunk passed
          +1 checkstyle 0m 18s trunk passed
          +1 mvnsite 0m 25s trunk passed
          +1 mvneclipse 0m 18s trunk passed
          +1 findbugs 0m 27s trunk passed
          +1 javadoc 0m 18s trunk passed
          +1 mvninstall 0m 14s the patch passed
          +1 compile 9m 5s the patch passed
          +1 javac 9m 5s the patch passed
          +1 checkstyle 0m 18s hadoop-common-project/hadoop-auth: The patch generated 0 new + 18 unchanged - 1 fixed = 18 total (was 19)
          +1 mvnsite 0m 24s the patch passed
          +1 mvneclipse 0m 18s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 0m 34s the patch passed
          +1 javadoc 0m 18s the patch passed
          +1 unit 3m 33s hadoop-auth in the patch passed.
          +1 asflicense 0m 31s The patch does not generate ASF License warnings.
          35m 11s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HADOOP-13890
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12842917/HADOOP-13890.03.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 9a16cbc0bbf6 3.13.0-103-generic #150-Ubuntu SMP Thu Nov 24 10:34:17 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 754f15b
          Default Java 1.8.0_111
          findbugs v3.0.0
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11257/testReport/
          modules C: hadoop-common-project/hadoop-auth U: hadoop-common-project/hadoop-auth
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11257/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 42s trunk passed +1 compile 9m 29s trunk passed +1 checkstyle 0m 18s trunk passed +1 mvnsite 0m 25s trunk passed +1 mvneclipse 0m 18s trunk passed +1 findbugs 0m 27s trunk passed +1 javadoc 0m 18s trunk passed +1 mvninstall 0m 14s the patch passed +1 compile 9m 5s the patch passed +1 javac 9m 5s the patch passed +1 checkstyle 0m 18s hadoop-common-project/hadoop-auth: The patch generated 0 new + 18 unchanged - 1 fixed = 18 total (was 19) +1 mvnsite 0m 24s the patch passed +1 mvneclipse 0m 18s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 34s the patch passed +1 javadoc 0m 18s the patch passed +1 unit 3m 33s hadoop-auth in the patch passed. +1 asflicense 0m 31s The patch does not generate ASF License warnings. 35m 11s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HADOOP-13890 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12842917/HADOOP-13890.03.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 9a16cbc0bbf6 3.13.0-103-generic #150-Ubuntu SMP Thu Nov 24 10:34:17 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 754f15b Default Java 1.8.0_111 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11257/testReport/ modules C: hadoop-common-project/hadoop-auth U: hadoop-common-project/hadoop-auth Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11257/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          xyao Xiaoyu Yao added a comment -

          In the patch v2, we remove the check for realm but keep the check for host as required based on RFC-4559.

             When the Kerberos Version 5 GSSAPI mechanism [RFC4121] is being used,
             the HTTP server will be using a principal name of the form of
             "HTTP/hostname".
          

          In other words, some valid UPN (User Principal Name) without hostname like HTTP@EXAMPLE.COM will be invalid for HTTP SPNEGO SPN (Service Principal Name). The RFC does not mention any requirement on realm. But based on many articles on multi-realm deployment, it is recommend to have HTTP/FQDN@Realm configured to avoid ambiguity and authentication problem in multi-realm use cases.

          Show
          xyao Xiaoyu Yao added a comment - In the patch v2, we remove the check for realm but keep the check for host as required based on RFC-4559 . When the Kerberos Version 5 GSSAPI mechanism [RFC4121] is being used, the HTTP server will be using a principal name of the form of "HTTP/hostname" . In other words, some valid UPN (User Principal Name) without hostname like HTTP@EXAMPLE.COM will be invalid for HTTP SPNEGO SPN (Service Principal Name). The RFC does not mention any requirement on realm. But based on many articles on multi-realm deployment, it is recommend to have HTTP/FQDN@Realm configured to avoid ambiguity and authentication problem in multi-realm use cases.
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 15s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 8m 16s trunk passed
          +1 compile 11m 28s trunk passed
          +1 checkstyle 0m 18s trunk passed
          +1 mvnsite 0m 27s trunk passed
          +1 mvneclipse 0m 18s trunk passed
          +1 findbugs 0m 28s trunk passed
          +1 javadoc 0m 19s trunk passed
          +1 mvninstall 0m 16s the patch passed
          +1 compile 10m 13s the patch passed
          +1 javac 10m 13s the patch passed
          -0 checkstyle 0m 18s hadoop-common-project/hadoop-auth: The patch generated 1 new + 18 unchanged - 1 fixed = 19 total (was 19)
          +1 mvnsite 0m 25s the patch passed
          +1 mvneclipse 0m 18s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 0m 35s the patch passed
          +1 javadoc 0m 18s the patch passed
          +1 unit 3m 33s hadoop-auth in the patch passed.
          +1 asflicense 0m 31s The patch does not generate ASF License warnings.
          40m 1s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HADOOP-13890
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12842911/HADOOP-13890.02.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 95763bec7229 3.13.0-103-generic #150-Ubuntu SMP Thu Nov 24 10:34:17 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / c6a3923
          Default Java 1.8.0_111
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/11256/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-auth.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11256/testReport/
          modules C: hadoop-common-project/hadoop-auth U: hadoop-common-project/hadoop-auth
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11256/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 8m 16s trunk passed +1 compile 11m 28s trunk passed +1 checkstyle 0m 18s trunk passed +1 mvnsite 0m 27s trunk passed +1 mvneclipse 0m 18s trunk passed +1 findbugs 0m 28s trunk passed +1 javadoc 0m 19s trunk passed +1 mvninstall 0m 16s the patch passed +1 compile 10m 13s the patch passed +1 javac 10m 13s the patch passed -0 checkstyle 0m 18s hadoop-common-project/hadoop-auth: The patch generated 1 new + 18 unchanged - 1 fixed = 19 total (was 19) +1 mvnsite 0m 25s the patch passed +1 mvneclipse 0m 18s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 35s the patch passed +1 javadoc 0m 18s the patch passed +1 unit 3m 33s hadoop-auth in the patch passed. +1 asflicense 0m 31s The patch does not generate ASF License warnings. 40m 1s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HADOOP-13890 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12842911/HADOOP-13890.02.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 95763bec7229 3.13.0-103-generic #150-Ubuntu SMP Thu Nov 24 10:34:17 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / c6a3923 Default Java 1.8.0_111 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/11256/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-auth.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11256/testReport/ modules C: hadoop-common-project/hadoop-auth U: hadoop-common-project/hadoop-auth Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11256/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 14s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 7m 18s trunk passed
          +1 compile 9m 41s trunk passed
          +1 checkstyle 0m 18s trunk passed
          +1 mvnsite 0m 24s trunk passed
          +1 mvneclipse 0m 18s trunk passed
          +1 findbugs 0m 29s trunk passed
          +1 javadoc 0m 18s trunk passed
          +1 mvninstall 0m 15s the patch passed
          +1 compile 9m 12s the patch passed
          +1 javac 9m 12s the patch passed
          -0 checkstyle 0m 18s hadoop-common-project/hadoop-auth: The patch generated 1 new + 18 unchanged - 1 fixed = 19 total (was 19)
          +1 mvnsite 0m 24s the patch passed
          +1 mvneclipse 0m 16s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 0m 36s the patch passed
          +1 javadoc 0m 17s the patch passed
          +1 unit 3m 32s hadoop-auth in the patch passed.
          +1 asflicense 0m 31s The patch does not generate ASF License warnings.
          36m 6s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HADOOP-13890
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12842911/HADOOP-13890.02.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 6c2be5edf51e 3.13.0-103-generic #150-Ubuntu SMP Thu Nov 24 10:34:17 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / c6a3923
          Default Java 1.8.0_111
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/11255/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-auth.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11255/testReport/
          modules C: hadoop-common-project/hadoop-auth U: hadoop-common-project/hadoop-auth
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11255/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 18s trunk passed +1 compile 9m 41s trunk passed +1 checkstyle 0m 18s trunk passed +1 mvnsite 0m 24s trunk passed +1 mvneclipse 0m 18s trunk passed +1 findbugs 0m 29s trunk passed +1 javadoc 0m 18s trunk passed +1 mvninstall 0m 15s the patch passed +1 compile 9m 12s the patch passed +1 javac 9m 12s the patch passed -0 checkstyle 0m 18s hadoop-common-project/hadoop-auth: The patch generated 1 new + 18 unchanged - 1 fixed = 19 total (was 19) +1 mvnsite 0m 24s the patch passed +1 mvneclipse 0m 16s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 36s the patch passed +1 javadoc 0m 17s the patch passed +1 unit 3m 32s hadoop-auth in the patch passed. +1 asflicense 0m 31s The patch does not generate ASF License warnings. 36m 6s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HADOOP-13890 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12842911/HADOOP-13890.02.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 6c2be5edf51e 3.13.0-103-generic #150-Ubuntu SMP Thu Nov 24 10:34:17 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / c6a3923 Default Java 1.8.0_111 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/11255/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-auth.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11255/testReport/ modules C: hadoop-common-project/hadoop-auth U: hadoop-common-project/hadoop-auth Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11255/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          xyao Xiaoyu Yao added a comment -

          I guess these changes (hadoop-auth) are unlikely to trigger hadoop-common and hadoop-hdfs tests by Jenkins.

          Show
          xyao Xiaoyu Yao added a comment - I guess these changes (hadoop-auth) are unlikely to trigger hadoop-common and hadoop-hdfs tests by Jenkins.
          Hide
          xyao Xiaoyu Yao added a comment -

          Post a patch that fix the KerberosName parsing and release the checking to require a realm from KerberosAuthenticationHandler without modifying the failed unit tests. This way, we won't break compatibility for use case that use SPN in the form of HTTP/host assuming local realm like the failed unit tests.

          I've tested the patch locally against the failed tests and all of them passed. Please review, thanks!

          Show
          xyao Xiaoyu Yao added a comment - Post a patch that fix the KerberosName parsing and release the checking to require a realm from KerberosAuthenticationHandler without modifying the failed unit tests. This way, we won't break compatibility for use case that use SPN in the form of HTTP/host assuming local realm like the failed unit tests. I've tested the patch locally against the failed tests and all of them passed. Please review, thanks!
          Hide
          xyao Xiaoyu Yao added a comment -

          Thanks Daryn Sharp for the comments. Yes, HADOOP-13565 enforces the check of SPNEGO SPNs to have three parts HTTP/host/realm. This will be incompatible which previously allows principal like HTTP/host before HADOOP-13565, assuming the default realm at authentication time.

          Since HTTP/host is a legitimate use case as you commented on HADOOP-13891, we can loosen the check added by HADOOP-13565 to allow it from KerberosAuthenticationHandler without modify the unit tests.

          Show
          xyao Xiaoyu Yao added a comment - Thanks Daryn Sharp for the comments. Yes, HADOOP-13565 enforces the check of SPNEGO SPNs to have three parts HTTP/host/realm. This will be incompatible which previously allows principal like HTTP/host before HADOOP-13565 , assuming the default realm at authentication time. Since HTTP/host is a legitimate use case as you commented on HADOOP-13891 , we can loosen the check added by HADOOP-13565 to allow it from KerberosAuthenticationHandler without modify the unit tests.
          Hide
          daryn Daryn Sharp added a comment -

          This appears to be an attempt to hide an incompatibility introduced by HADOOP-13565. Generally, when a legitimate test breaks, the solution isn't to alter the test to conform to new incompatible behavior.

          Show
          daryn Daryn Sharp added a comment - This appears to be an attempt to hide an incompatibility introduced by HADOOP-13565 . Generally, when a legitimate test breaks, the solution isn't to alter the test to conform to new incompatible behavior.
          Hide
          xyao Xiaoyu Yao added a comment -

          I plan to commit the patch by EOD today to fix the Jenkins issues unless Brahma Reddy Battula or other folks on the watchlist have additional comments.

          Show
          xyao Xiaoyu Yao added a comment - I plan to commit the patch by EOD today to fix the Jenkins issues unless Brahma Reddy Battula or other folks on the watchlist have additional comments.
          Hide
          arpitagarwal Arpit Agarwal added a comment -

          +1 for the patch. Verified it fixes the unit tests. Brahma Reddy Battula, do you have any additional comments?

          Show
          arpitagarwal Arpit Agarwal added a comment - +1 for the patch. Verified it fixes the unit tests. Brahma Reddy Battula , do you have any additional comments?
          Hide
          xyao Xiaoyu Yao added a comment -

          The failed unit test is not related to this change. It is tracked by https://issues.apache.org/jira/browse/HDFS-11131.

          Show
          xyao Xiaoyu Yao added a comment - The failed unit test is not related to this change. It is tracked by https://issues.apache.org/jira/browse/HDFS-11131 .
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 17s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 4 new or modified test files.
          0 mvndep 0m 16s Maven dependency ordering for branch
          +1 mvninstall 8m 4s trunk passed
          +1 compile 11m 19s trunk passed
          +1 checkstyle 1m 42s trunk passed
          +1 mvnsite 2m 43s trunk passed
          +1 mvneclipse 0m 57s trunk passed
          +1 findbugs 4m 12s trunk passed
          +1 javadoc 2m 6s trunk passed
          0 mvndep 0m 19s Maven dependency ordering for patch
          +1 mvninstall 2m 12s the patch passed
          +1 compile 11m 16s the patch passed
          +1 javac 11m 16s the patch passed
          +1 checkstyle 1m 47s root: The patch generated 0 new + 153 unchanged - 2 fixed = 153 total (was 155)
          +1 mvnsite 2m 54s the patch passed
          +1 mvneclipse 1m 9s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 4m 14s the patch passed
          +1 javadoc 2m 5s the patch passed
          +1 unit 8m 25s hadoop-common in the patch passed.
          +1 unit 2m 18s hadoop-kms in the patch passed.
          -1 unit 67m 28s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 39s The patch does not generate ASF License warnings.
          160m 15s



          Reason Tests
          Failed junit tests hadoop.hdfs.server.datanode.checker.TestThrottledAsyncChecker



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HADOOP-13890
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12842818/HADOOP-13890.01.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 1ac439b2ce6d 3.13.0-103-generic #150-Ubuntu SMP Thu Nov 24 10:34:17 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / f66f618
          Default Java 1.8.0_111
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/11248/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11248/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms hadoop-hdfs-project/hadoop-hdfs U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11248/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 4 new or modified test files. 0 mvndep 0m 16s Maven dependency ordering for branch +1 mvninstall 8m 4s trunk passed +1 compile 11m 19s trunk passed +1 checkstyle 1m 42s trunk passed +1 mvnsite 2m 43s trunk passed +1 mvneclipse 0m 57s trunk passed +1 findbugs 4m 12s trunk passed +1 javadoc 2m 6s trunk passed 0 mvndep 0m 19s Maven dependency ordering for patch +1 mvninstall 2m 12s the patch passed +1 compile 11m 16s the patch passed +1 javac 11m 16s the patch passed +1 checkstyle 1m 47s root: The patch generated 0 new + 153 unchanged - 2 fixed = 153 total (was 155) +1 mvnsite 2m 54s the patch passed +1 mvneclipse 1m 9s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 4m 14s the patch passed +1 javadoc 2m 5s the patch passed +1 unit 8m 25s hadoop-common in the patch passed. +1 unit 2m 18s hadoop-kms in the patch passed. -1 unit 67m 28s hadoop-hdfs in the patch failed. +1 asflicense 0m 39s The patch does not generate ASF License warnings. 160m 15s Reason Tests Failed junit tests hadoop.hdfs.server.datanode.checker.TestThrottledAsyncChecker Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HADOOP-13890 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12842818/HADOOP-13890.01.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 1ac439b2ce6d 3.13.0-103-generic #150-Ubuntu SMP Thu Nov 24 10:34:17 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / f66f618 Default Java 1.8.0_111 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/11248/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11248/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11248/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          xyao Xiaoyu Yao added a comment - - edited

          Thanks Brahma Reddy Battula. I've updated patch to cover both hadoop-common and haddop-hdfs tests. Also update the title and description.

          Show
          xyao Xiaoyu Yao added a comment - - edited Thanks Brahma Reddy Battula . I've updated patch to cover both hadoop-common and haddop-hdfs tests. Also update the title and description.
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          Xiaoyu Yao thanks for updating the patch..TestTrashWithSecureEncryptionZones and TestSecureEncryptionZoneWithKMS also related.can you fix them also..?

          Jenkins URL
          https://builds.apache.org/job/PreCommit-HDFS-Build/17836/testReport/

          Show
          brahmareddy Brahma Reddy Battula added a comment - Xiaoyu Yao thanks for updating the patch.. TestTrashWithSecureEncryptionZones and TestSecureEncryptionZoneWithKMS also related.can you fix them also..? Jenkins URL https://builds.apache.org/job/PreCommit-HDFS-Build/17836/testReport/
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 13s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
          0 mvndep 0m 10s Maven dependency ordering for branch
          +1 mvninstall 6m 51s trunk passed
          +1 compile 9m 28s trunk passed
          +1 checkstyle 0m 34s trunk passed
          +1 mvnsite 1m 25s trunk passed
          +1 mvneclipse 0m 36s trunk passed
          +1 findbugs 1m 50s trunk passed
          +1 javadoc 1m 4s trunk passed
          0 mvndep 0m 7s Maven dependency ordering for patch
          +1 mvninstall 0m 54s the patch passed
          +1 compile 9m 9s the patch passed
          +1 javac 9m 9s the patch passed
          -0 checkstyle 0m 35s hadoop-common-project: The patch generated 2 new + 153 unchanged - 2 fixed = 155 total (was 155)
          +1 mvnsite 1m 24s the patch passed
          +1 mvneclipse 0m 36s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 2m 7s the patch passed
          +1 javadoc 1m 5s the patch passed
          +1 unit 8m 23s hadoop-common in the patch passed.
          +1 unit 2m 13s hadoop-kms in the patch passed.
          +1 asflicense 0m 31s The patch does not generate ASF License warnings.
          53m 46s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HADOOP-13890
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12842762/HADOOP-13890.00.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux af07495fee4f 3.13.0-103-generic #150-Ubuntu SMP Thu Nov 24 10:34:17 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 4c38f11
          Default Java 1.8.0_111
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/11245/artifact/patchprocess/diff-checkstyle-hadoop-common-project.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11245/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms U: hadoop-common-project
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11245/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 13s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. 0 mvndep 0m 10s Maven dependency ordering for branch +1 mvninstall 6m 51s trunk passed +1 compile 9m 28s trunk passed +1 checkstyle 0m 34s trunk passed +1 mvnsite 1m 25s trunk passed +1 mvneclipse 0m 36s trunk passed +1 findbugs 1m 50s trunk passed +1 javadoc 1m 4s trunk passed 0 mvndep 0m 7s Maven dependency ordering for patch +1 mvninstall 0m 54s the patch passed +1 compile 9m 9s the patch passed +1 javac 9m 9s the patch passed -0 checkstyle 0m 35s hadoop-common-project: The patch generated 2 new + 153 unchanged - 2 fixed = 155 total (was 155) +1 mvnsite 1m 24s the patch passed +1 mvneclipse 0m 36s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 2m 7s the patch passed +1 javadoc 1m 5s the patch passed +1 unit 8m 23s hadoop-common in the patch passed. +1 unit 2m 13s hadoop-kms in the patch passed. +1 asflicense 0m 31s The patch does not generate ASF License warnings. 53m 46s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HADOOP-13890 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12842762/HADOOP-13890.00.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux af07495fee4f 3.13.0-103-generic #150-Ubuntu SMP Thu Nov 24 10:34:17 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 4c38f11 Default Java 1.8.0_111 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/11245/artifact/patchprocess/diff-checkstyle-hadoop-common-project.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11245/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms U: hadoop-common-project Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11245/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          xyao Xiaoyu Yao added a comment -

          Sorry I was not able to catch this at the commit time because Jenkins only run hadoop-auth tests but the failed tests are from hadoop-common.

          Show
          xyao Xiaoyu Yao added a comment - Sorry I was not able to catch this at the commit time because Jenkins only run hadoop-auth tests but the failed tests are from hadoop-common.
          Hide
          xyao Xiaoyu Yao added a comment -

          Attach a patch that fixed the unit tests.

          Show
          xyao Xiaoyu Yao added a comment - Attach a patch that fixed the unit tests.
          Show
          jzhuge John Zhuge added a comment - Saw them in pre-commit unit tests: https://builds.apache.org/job/PreCommit-HDFS-Build/17824/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt and https://builds.apache.org/job/PreCommit-HDFS-Build/17824/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt .

            People

            • Assignee:
              xyao Xiaoyu Yao
              Reporter:
              brahmareddy Brahma Reddy Battula
            • Votes:
              1 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development