Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-13255

KMSClientProvider should check and renew tgt when doing delegation token operations.

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 2.7.3, 2.6.5, 3.0.0-alpha1
    • Component/s: kms
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed
    1. HADOOP-13255.01.patch
      2 kB
      Xiao Chen
    2. HADOOP-13255.02.patch
      8 kB
      Xiao Chen
    3. HADOOP-13255.03.patch
      9 kB
      Xiao Chen
    4. HADOOP-13255.04.patch
      10 kB
      Xiao Chen
    5. HADOOP-13255.05.patch
      11 kB
      Xiao Chen
    6. HADOOP-13255.branch-2.patch
      3 kB
      Xiao Chen
    7. HADOOP-13255.test.patch
      10 kB
      Xiao Chen

      Issue Links

        Activity

        Hide
        xiaochen Xiao Chen added a comment -

        HADOOP-12559 is a good fix.
        But for delegation token operations, it doesn't go through createConnection. We should fix these places as well.

        Show
        xiaochen Xiao Chen added a comment - HADOOP-12559 is a good fix. But for delegation token operations, it doesn't go through createConnection . We should fix these places as well.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 18s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 7m 3s trunk passed
        +1 compile 6m 30s trunk passed
        +1 checkstyle 0m 22s trunk passed
        +1 mvnsite 0m 54s trunk passed
        +1 mvneclipse 0m 13s trunk passed
        +1 findbugs 1m 19s trunk passed
        +1 javadoc 0m 53s trunk passed
        +1 mvninstall 0m 36s the patch passed
        +1 compile 7m 44s the patch passed
        +1 javac 7m 44s the patch passed
        +1 checkstyle 0m 24s the patch passed
        +1 mvnsite 0m 59s the patch passed
        +1 mvneclipse 0m 13s the patch passed
        -1 whitespace 0m 0s The patch has 20 line(s) that end in whitespace. Use git apply --whitespace=fix.
        +1 findbugs 1m 43s the patch passed
        +1 javadoc 0m 55s the patch passed
        +1 unit 7m 54s hadoop-common in the patch passed.
        +1 asflicense 0m 22s The patch does not generate ASF License warnings.
        39m 6s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:2c91fd8
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12809308/HADOOP-13255.01.patch
        JIRA Issue HADOOP-13255
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux fa9121e9ae92 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 9581fb7
        Default Java 1.8.0_91
        findbugs v3.0.0
        whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/9712/artifact/patchprocess/whitespace-eol.txt
        Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9712/testReport/
        modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9712/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 18s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 7m 3s trunk passed +1 compile 6m 30s trunk passed +1 checkstyle 0m 22s trunk passed +1 mvnsite 0m 54s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 19s trunk passed +1 javadoc 0m 53s trunk passed +1 mvninstall 0m 36s the patch passed +1 compile 7m 44s the patch passed +1 javac 7m 44s the patch passed +1 checkstyle 0m 24s the patch passed +1 mvnsite 0m 59s the patch passed +1 mvneclipse 0m 13s the patch passed -1 whitespace 0m 0s The patch has 20 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 findbugs 1m 43s the patch passed +1 javadoc 0m 55s the patch passed +1 unit 7m 54s hadoop-common in the patch passed. +1 asflicense 0m 22s The patch does not generate ASF License warnings. 39m 6s Subsystem Report/Notes Docker Image:yetus/hadoop:2c91fd8 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12809308/HADOOP-13255.01.patch JIRA Issue HADOOP-13255 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux fa9121e9ae92 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 9581fb7 Default Java 1.8.0_91 findbugs v3.0.0 whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/9712/artifact/patchprocess/whitespace-eol.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9712/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9712/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        xiaochen Xiao Chen added a comment -

        Whitespace is not related to this patch. Zhe Zhang, may I ask for your review? Thanks a lot!

        Show
        xiaochen Xiao Chen added a comment - Whitespace is not related to this patch. Zhe Zhang , may I ask for your review? Thanks a lot!
        Hide
        zhz Zhe Zhang added a comment -

        Thanks Xiao, good work here.

        I wonder if we should do it at DelegationTokenAuthenticator#doDelegationTokenOperation?

        Show
        zhz Zhe Zhang added a comment - Thanks Xiao, good work here. I wonder if we should do it at DelegationTokenAuthenticator#doDelegationTokenOperation ?
        Hide
        xiaochen Xiao Chen added a comment -

        Thanks Zhe for the inspiration!
        Patch 2 is up for review:

        • I think we can move it further up to a place where the KMSClientProvider calls converge: DelegationTokenAuthenticator#authenticate. That's about the farthest we can go, since KerberosAuthenticator would be in hadoop-auth.
        • Borrowed your test from HADOOP-12559, hope you don't mind.
        • Poked around the newly Kerby-backed minikdc. I was able to hack it so that we can unit test in a relatively short interval.
        • Updated kms test's log4j default to INFO, which can provide enough information by default.

        Please let me know what you think. If a unit test is not necessary, I can back it out and make the patch small.

        Show
        xiaochen Xiao Chen added a comment - Thanks Zhe for the inspiration! Patch 2 is up for review: I think we can move it further up to a place where the KMSClientProvider calls converge: DelegationTokenAuthenticator#authenticate . That's about the farthest we can go, since KerberosAuthenticator would be in hadoop-auth. Borrowed your test from HADOOP-12559 , hope you don't mind. Poked around the newly Kerby-backed minikdc. I was able to hack it so that we can unit test in a relatively short interval. Updated kms test's log4j default to INFO, which can provide enough information by default. Please let me know what you think. If a unit test is not necessary, I can back it out and make the patch small.
        Hide
        xiaochen Xiao Chen added a comment -

        Below are the stack traces, hopefully helps review:
        createConnection:

        	at org.apache.hadoop.crypto.key.kms.KMSClientProvider.createConnection(KMSClientProvider.java:556)
        	at org.apache.hadoop.crypto.key.kms.KMSClientProvider.getKeys(KMSClientProvider.java:661)
        	at org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider$7.call(LoadBalancingKMSClientProvider.java:235)
        	at org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider$7.call(LoadBalancingKMSClientProvider.java:232)
        	at org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider.doOp(LoadBalancingKMSClientProvider.java:94)
        	at org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider.getKeys(LoadBalancingKMSClientProvider.java:232)
        	at org.apache.hadoop.crypto.key.kms.server.TestKMS$17.call(TestKMS.java:2097)
        	at org.apache.hadoop.crypto.key.kms.server.TestKMS$17.call(TestKMS.java:2085)
        	at org.apache.hadoop.crypto.key.kms.server.TestKMS.runServer(TestKMS.java:144)
        	at org.apache.hadoop.crypto.key.kms.server.TestKMS.runServer(TestKMS.java:126)
        	at org.apache.hadoop.crypto.key.kms.server.TestKMS.testTGTRenewal(TestKMS.java:2085)
        	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        	at java.lang.reflect.Method.invoke(Method.java:497)
        	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
        	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
        	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
        	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
        	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
        	at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
        Caused by: org.apache.hadoop.security.authentication.client.AuthenticationException: GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)
        	at org.apache.hadoop.security.authentication.client.KerberosAuthenticator.doSpnegoSequence(KerberosAuthenticator.java:333)
        	at org.apache.hadoop.security.authentication.client.KerberosAuthenticator.authenticate(KerberosAuthenticator.java:203)
        	at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticator.authenticate(DelegationTokenAuthenticator.java:146)
        	at org.apache.hadoop.security.authentication.client.AuthenticatedURL.openConnection(AuthenticatedURL.java:216)
        	at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticatedURL.openConnection(DelegationTokenAuthenticatedURL.java:322)
        	at org.apache.hadoop.crypto.key.kms.KMSClientProvider$1.run(KMSClientProvider.java:547)
        	at org.apache.hadoop.crypto.key.kms.KMSClientProvider$1.run(KMSClientProvider.java:542)
        	at java.security.AccessController.doPrivileged(Native Method)
        	at javax.security.auth.Subject.doAs(Subject.java:422)
        	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1755)
        	at org.apache.hadoop.crypto.key.kms.KMSClientProvider.createConnection(KMSClientProvider.java:542)
        	... 20 more
        Caused by: GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)
        	at sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147)
        	at sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:122)
        	at sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187)
        	at sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:224)
        	at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212)
        	at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
        	at org.apache.hadoop.security.authentication.client.KerberosAuthenticator$1.run(KerberosAuthenticator.java:309)
        	at org.apache.hadoop.security.authentication.client.KerberosAuthenticator$1.run(KerberosAuthenticator.java:285)
        	at java.security.AccessController.doPrivileged(Native Method)
        	at javax.security.auth.Subject.doAs(Subject.java:422)
        	at org.apache.hadoop.security.authentication.client.KerberosAuthenticator.doSpnegoSequence(KerberosAuthenticator.java:285)
        	... 30 more
        

        addDelegationToken:

        java.io.IOException: java.lang.reflect.UndeclaredThrowableException
        	at org.apache.hadoop.crypto.key.kms.KMSClientProvider.addDelegationTokens(KMSClientProvider.java:894)
        	at org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider$1.call(LoadBalancingKMSClientProvider.java:132)
        	at org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider$1.call(LoadBalancingKMSClientProvider.java:129)
        	at org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider.doOp(LoadBalancingKMSClientProvider.java:94)
        	at org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider.addDelegationTokens(LoadBalancingKMSClientProvider.java:129)
        	at org.apache.hadoop.crypto.key.kms.server.TestKMS$15.call(TestKMS.java:1964)
        	at org.apache.hadoop.crypto.key.kms.server.TestKMS$15.call(TestKMS.java:1951)
        	at org.apache.hadoop.crypto.key.kms.server.TestKMS.runServer(TestKMS.java:130)
        	at org.apache.hadoop.crypto.key.kms.server.TestKMS.runServer(TestKMS.java:112)
        	at org.apache.hadoop.crypto.key.kms.server.TestKMS.testTGTRenewal(TestKMS.java:1951)
        	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        	at java.lang.reflect.Method.invoke(Method.java:606)
        	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
        	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
        	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
        	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
        	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
        	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
        	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
        	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
        	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
        	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
        	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
        	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
        	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
        	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
        	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
        	at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
        	at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
        	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:119)
        	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:42)
        	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:234)
        	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:74)
        	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        	at java.lang.reflect.Method.invoke(Method.java:606)
        	at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)
        Caused by: java.lang.reflect.UndeclaredThrowableException
        	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1687)
        	at org.apache.hadoop.crypto.key.kms.KMSClientProvider.addDelegationTokens(KMSClientProvider.java:876)
        	... 39 more
        Caused by: org.apache.hadoop.security.authentication.client.AuthenticationException: GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)
        	at org.apache.hadoop.security.authentication.client.KerberosAuthenticator.doSpnegoSequence(KerberosAuthenticator.java:332)
        	at org.apache.hadoop.security.authentication.client.KerberosAuthenticator.authenticate(KerberosAuthenticator.java:205)
        	at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticator.authenticate(DelegationTokenAuthenticator.java:128)
        	at org.apache.hadoop.security.authentication.client.AuthenticatedURL.openConnection(AuthenticatedURL.java:215)
        	at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticator.doDelegationTokenOperation(DelegationTokenAuthenticator.java:285)
        	at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticator.getDelegationToken(DelegationTokenAuthenticator.java:166)
        	at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticatedURL.getDelegationToken(DelegationTokenAuthenticatedURL.java:371)
        	at org.apache.hadoop.crypto.key.kms.KMSClientProvider$2.run(KMSClientProvider.java:881)
        	at org.apache.hadoop.crypto.key.kms.KMSClientProvider$2.run(KMSClientProvider.java:876)
        	at java.security.AccessController.doPrivileged(Native Method)
        	at javax.security.auth.Subject.doAs(Subject.java:415)
        	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1669)
        	... 40 more
        Caused by: GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)
        	at sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147)
        	at sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:121)
        	at sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187)
        	at sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:223)
        	at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212)
        	at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
        	at org.apache.hadoop.security.authentication.client.KerberosAuthenticator$1.run(KerberosAuthenticator.java:311)
        	at org.apache.hadoop.security.authentication.client.KerberosAuthenticator$1.run(KerberosAuthenticator.java:287)
        	at java.security.AccessController.doPrivileged(Native Method)
        	at javax.security.auth.Subject.doAs(Subject.java:415)
        	at org.apache.hadoop.security.authentication.client.KerberosAuthenticator.doSpnegoSequence(KerberosAuthenticator.java:287)
        	... 51 more
        

        I see Chris and Xiaoyu are on watch as well, please feel free to chime in.

        Show
        xiaochen Xiao Chen added a comment - Below are the stack traces, hopefully helps review: createConnection: at org.apache.hadoop.crypto.key.kms.KMSClientProvider.createConnection(KMSClientProvider.java:556) at org.apache.hadoop.crypto.key.kms.KMSClientProvider.getKeys(KMSClientProvider.java:661) at org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider$7.call(LoadBalancingKMSClientProvider.java:235) at org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider$7.call(LoadBalancingKMSClientProvider.java:232) at org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider.doOp(LoadBalancingKMSClientProvider.java:94) at org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider.getKeys(LoadBalancingKMSClientProvider.java:232) at org.apache.hadoop.crypto.key.kms.server.TestKMS$17.call(TestKMS.java:2097) at org.apache.hadoop.crypto.key.kms.server.TestKMS$17.call(TestKMS.java:2085) at org.apache.hadoop.crypto.key.kms.server.TestKMS.runServer(TestKMS.java:144) at org.apache.hadoop.crypto.key.kms.server.TestKMS.runServer(TestKMS.java:126) at org.apache.hadoop.crypto.key.kms.server.TestKMS.testTGTRenewal(TestKMS.java:2085) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74) Caused by: org.apache.hadoop.security.authentication.client.AuthenticationException: GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt) at org.apache.hadoop.security.authentication.client.KerberosAuthenticator.doSpnegoSequence(KerberosAuthenticator.java:333) at org.apache.hadoop.security.authentication.client.KerberosAuthenticator.authenticate(KerberosAuthenticator.java:203) at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticator.authenticate(DelegationTokenAuthenticator.java:146) at org.apache.hadoop.security.authentication.client.AuthenticatedURL.openConnection(AuthenticatedURL.java:216) at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticatedURL.openConnection(DelegationTokenAuthenticatedURL.java:322) at org.apache.hadoop.crypto.key.kms.KMSClientProvider$1.run(KMSClientProvider.java:547) at org.apache.hadoop.crypto.key.kms.KMSClientProvider$1.run(KMSClientProvider.java:542) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1755) at org.apache.hadoop.crypto.key.kms.KMSClientProvider.createConnection(KMSClientProvider.java:542) ... 20 more Caused by: GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt) at sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147) at sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:122) at sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187) at sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:224) at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212) at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179) at org.apache.hadoop.security.authentication.client.KerberosAuthenticator$1.run(KerberosAuthenticator.java:309) at org.apache.hadoop.security.authentication.client.KerberosAuthenticator$1.run(KerberosAuthenticator.java:285) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.authentication.client.KerberosAuthenticator.doSpnegoSequence(KerberosAuthenticator.java:285) ... 30 more addDelegationToken: java.io.IOException: java.lang.reflect.UndeclaredThrowableException at org.apache.hadoop.crypto.key.kms.KMSClientProvider.addDelegationTokens(KMSClientProvider.java:894) at org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider$1.call(LoadBalancingKMSClientProvider.java:132) at org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider$1.call(LoadBalancingKMSClientProvider.java:129) at org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider.doOp(LoadBalancingKMSClientProvider.java:94) at org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider.addDelegationTokens(LoadBalancingKMSClientProvider.java:129) at org.apache.hadoop.crypto.key.kms.server.TestKMS$15.call(TestKMS.java:1964) at org.apache.hadoop.crypto.key.kms.server.TestKMS$15.call(TestKMS.java:1951) at org.apache.hadoop.crypto.key.kms.server.TestKMS.runServer(TestKMS.java:130) at org.apache.hadoop.crypto.key.kms.server.TestKMS.runServer(TestKMS.java:112) at org.apache.hadoop.crypto.key.kms.server.TestKMS.testTGTRenewal(TestKMS.java:1951) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner.run(ParentRunner.java:309) at org.junit.runner.JUnitCore.run(JUnitCore.java:160) at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:119) at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:42) at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:234) at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:74) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144) Caused by: java.lang.reflect.UndeclaredThrowableException at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1687) at org.apache.hadoop.crypto.key.kms.KMSClientProvider.addDelegationTokens(KMSClientProvider.java:876) ... 39 more Caused by: org.apache.hadoop.security.authentication.client.AuthenticationException: GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt) at org.apache.hadoop.security.authentication.client.KerberosAuthenticator.doSpnegoSequence(KerberosAuthenticator.java:332) at org.apache.hadoop.security.authentication.client.KerberosAuthenticator.authenticate(KerberosAuthenticator.java:205) at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticator.authenticate(DelegationTokenAuthenticator.java:128) at org.apache.hadoop.security.authentication.client.AuthenticatedURL.openConnection(AuthenticatedURL.java:215) at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticator.doDelegationTokenOperation(DelegationTokenAuthenticator.java:285) at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticator.getDelegationToken(DelegationTokenAuthenticator.java:166) at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticatedURL.getDelegationToken(DelegationTokenAuthenticatedURL.java:371) at org.apache.hadoop.crypto.key.kms.KMSClientProvider$2.run(KMSClientProvider.java:881) at org.apache.hadoop.crypto.key.kms.KMSClientProvider$2.run(KMSClientProvider.java:876) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1669) ... 40 more Caused by: GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt) at sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147) at sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:121) at sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187) at sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:223) at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212) at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179) at org.apache.hadoop.security.authentication.client.KerberosAuthenticator$1.run(KerberosAuthenticator.java:311) at org.apache.hadoop.security.authentication.client.KerberosAuthenticator$1.run(KerberosAuthenticator.java:287) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.authentication.client.KerberosAuthenticator.doSpnegoSequence(KerberosAuthenticator.java:287) ... 51 more I see Chris and Xiaoyu are on watch as well, please feel free to chime in.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 23s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        0 mvndep 0m 8s Maven dependency ordering for branch
        +1 mvninstall 6m 48s trunk passed
        +1 compile 6m 59s trunk passed
        +1 checkstyle 0m 29s trunk passed
        +1 mvnsite 1m 35s trunk passed
        +1 mvneclipse 0m 35s trunk passed
        +1 findbugs 2m 3s trunk passed
        +1 javadoc 1m 9s trunk passed
        0 mvndep 0m 7s Maven dependency ordering for patch
        +1 mvninstall 1m 26s the patch passed
        +1 compile 8m 18s the patch passed
        +1 javac 8m 18s the patch passed
        +1 checkstyle 0m 37s the patch passed
        +1 mvnsite 1m 35s the patch passed
        +1 mvneclipse 0m 38s the patch passed
        -1 whitespace 0m 0s The patch has 20 line(s) that end in whitespace. Use git apply --whitespace=fix.
        +1 findbugs 3m 0s the patch passed
        +1 javadoc 1m 11s the patch passed
        +1 unit 0m 21s hadoop-minikdc in the patch passed.
        -1 unit 7m 47s hadoop-common in the patch failed.
        +1 unit 1m 39s hadoop-kms in the patch passed.
        +1 asflicense 0m 21s The patch does not generate ASF License warnings.
        48m 13s



        Reason Tests
        Failed junit tests hadoop.metrics2.impl.TestGangliaMetrics



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:2c91fd8
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12809595/HADOOP-13255.02.patch
        JIRA Issue HADOOP-13255
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux dd6733c170c3 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 8a1dcce
        Default Java 1.8.0_91
        findbugs v3.0.0
        whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/9750/artifact/patchprocess/whitespace-eol.txt
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9750/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
        Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9750/testReport/
        modules C: hadoop-common-project/hadoop-minikdc hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms U: hadoop-common-project
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9750/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 23s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. 0 mvndep 0m 8s Maven dependency ordering for branch +1 mvninstall 6m 48s trunk passed +1 compile 6m 59s trunk passed +1 checkstyle 0m 29s trunk passed +1 mvnsite 1m 35s trunk passed +1 mvneclipse 0m 35s trunk passed +1 findbugs 2m 3s trunk passed +1 javadoc 1m 9s trunk passed 0 mvndep 0m 7s Maven dependency ordering for patch +1 mvninstall 1m 26s the patch passed +1 compile 8m 18s the patch passed +1 javac 8m 18s the patch passed +1 checkstyle 0m 37s the patch passed +1 mvnsite 1m 35s the patch passed +1 mvneclipse 0m 38s the patch passed -1 whitespace 0m 0s The patch has 20 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 findbugs 3m 0s the patch passed +1 javadoc 1m 11s the patch passed +1 unit 0m 21s hadoop-minikdc in the patch passed. -1 unit 7m 47s hadoop-common in the patch failed. +1 unit 1m 39s hadoop-kms in the patch passed. +1 asflicense 0m 21s The patch does not generate ASF License warnings. 48m 13s Reason Tests Failed junit tests hadoop.metrics2.impl.TestGangliaMetrics Subsystem Report/Notes Docker Image:yetus/hadoop:2c91fd8 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12809595/HADOOP-13255.02.patch JIRA Issue HADOOP-13255 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux dd6733c170c3 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 8a1dcce Default Java 1.8.0_91 findbugs v3.0.0 whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/9750/artifact/patchprocess/whitespace-eol.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9750/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9750/testReport/ modules C: hadoop-common-project/hadoop-minikdc hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms U: hadoop-common-project Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9750/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        xyao Xiaoyu Yao added a comment - - edited

        Thanks Xiao Chen for working on this and Zhe Zhang for the review. I would suggest we fix with the approach in v1 patch.

        1. V1 patch is correct and less risky. All the change is localized to KMSCientProvider compared with broader change in DelegationTokenAuthenticator or KerberosAuthenticator.

        2. V2 patch below won't be able to handle the proxy user and token user cases as the currentUGI is not sufficient for these cases. There are a few fixes around KMSClientProvider#actualUGI to make this right. You can refer to how actualUGI is initialized in KMSClientProvider#KMSClientProvider().

              UserGroupInformation.getCurrentUser().checkTGTAndReloginFromKeytab();
        
        Show
        xyao Xiaoyu Yao added a comment - - edited Thanks Xiao Chen for working on this and Zhe Zhang for the review. I would suggest we fix with the approach in v1 patch. 1. V1 patch is correct and less risky. All the change is localized to KMSCientProvider compared with broader change in DelegationTokenAuthenticator or KerberosAuthenticator. 2. V2 patch below won't be able to handle the proxy user and token user cases as the currentUGI is not sufficient for these cases. There are a few fixes around KMSClientProvider#actualUGI to make this right. You can refer to how actualUGI is initialized in KMSClientProvider#KMSClientProvider(). UserGroupInformation.getCurrentUser().checkTGTAndReloginFromKeytab();
        Hide
        xiaochen Xiao Chen added a comment -

        Thanks Xiaoyu Yao for the review and suggestions.

        I'm neural on #1: it definitely touches less code path and hence safer. OTOH, it feels like a good idea to try fix it in a more general way, to save us efforts in finding and fixing all places in the caller.

        For #2, please correct me if I misunderstood. The reason we have actualUgi is that we want to perform the operation under the creator of the KMSCP. UGI#doAs will do this, and once inside the doAs, UGI#getCurrentUser will return the current user considering the doAs stack, which is actualUgi. UGI getCurrentUser also has a comment about this. I have added 1 code block in the unit test to show that proxy user works. Not sure about how to test TOKEN programmatically, but manually verified it to work as well. (Tested via the webhdsf case in HADOOP-12787, nice fix! )

        Show
        xiaochen Xiao Chen added a comment - Thanks Xiaoyu Yao for the review and suggestions. I'm neural on #1: it definitely touches less code path and hence safer. OTOH, it feels like a good idea to try fix it in a more general way, to save us efforts in finding and fixing all places in the caller. For #2, please correct me if I misunderstood. The reason we have actualUgi is that we want to perform the operation under the creator of the KMSCP. UGI#doAs will do this, and once inside the doAs , UGI#getCurrentUser will return the current user considering the doAs stack, which is actualUgi . UGI getCurrentUser also has a comment about this . I have added 1 code block in the unit test to show that proxy user works. Not sure about how to test TOKEN programmatically, but manually verified it to work as well. (Tested via the webhdsf case in HADOOP-12787 , nice fix! )
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 27s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        0 mvndep 0m 12s Maven dependency ordering for branch
        +1 mvninstall 6m 53s trunk passed
        +1 compile 7m 18s trunk passed
        +1 checkstyle 0m 30s trunk passed
        +1 mvnsite 1m 37s trunk passed
        +1 mvneclipse 0m 32s trunk passed
        +1 findbugs 2m 9s trunk passed
        +1 javadoc 1m 6s trunk passed
        0 mvndep 0m 6s Maven dependency ordering for patch
        +1 mvninstall 1m 17s the patch passed
        +1 compile 7m 2s the patch passed
        +1 javac 7m 2s the patch passed
        +1 checkstyle 0m 29s the patch passed
        +1 mvnsite 1m 27s the patch passed
        +1 mvneclipse 0m 41s the patch passed
        -1 whitespace 0m 0s The patch has 20 line(s) that end in whitespace. Use git apply --whitespace=fix.
        +1 findbugs 2m 27s the patch passed
        +1 javadoc 1m 8s the patch passed
        +1 unit 0m 20s hadoop-minikdc in the patch passed.
        -1 unit 7m 54s hadoop-common in the patch failed.
        -1 unit 1m 34s hadoop-kms in the patch failed.
        +1 asflicense 0m 20s The patch does not generate ASF License warnings.
        46m 29s



        Reason Tests
        Failed junit tests hadoop.metrics2.impl.TestGangliaMetrics
          hadoop.crypto.key.kms.server.TestKMS



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:2c91fd8
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12809767/HADOOP-13255.03.patch
        JIRA Issue HADOOP-13255
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 2a89ec8692fa 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 28b66ae
        Default Java 1.8.0_91
        findbugs v3.0.0
        whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/9757/artifact/patchprocess/whitespace-eol.txt
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9757/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9757/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-kms.txt
        Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9757/testReport/
        modules C: hadoop-common-project/hadoop-minikdc hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms U: hadoop-common-project
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9757/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 27s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. 0 mvndep 0m 12s Maven dependency ordering for branch +1 mvninstall 6m 53s trunk passed +1 compile 7m 18s trunk passed +1 checkstyle 0m 30s trunk passed +1 mvnsite 1m 37s trunk passed +1 mvneclipse 0m 32s trunk passed +1 findbugs 2m 9s trunk passed +1 javadoc 1m 6s trunk passed 0 mvndep 0m 6s Maven dependency ordering for patch +1 mvninstall 1m 17s the patch passed +1 compile 7m 2s the patch passed +1 javac 7m 2s the patch passed +1 checkstyle 0m 29s the patch passed +1 mvnsite 1m 27s the patch passed +1 mvneclipse 0m 41s the patch passed -1 whitespace 0m 0s The patch has 20 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 findbugs 2m 27s the patch passed +1 javadoc 1m 8s the patch passed +1 unit 0m 20s hadoop-minikdc in the patch passed. -1 unit 7m 54s hadoop-common in the patch failed. -1 unit 1m 34s hadoop-kms in the patch failed. +1 asflicense 0m 20s The patch does not generate ASF License warnings. 46m 29s Reason Tests Failed junit tests hadoop.metrics2.impl.TestGangliaMetrics   hadoop.crypto.key.kms.server.TestKMS Subsystem Report/Notes Docker Image:yetus/hadoop:2c91fd8 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12809767/HADOOP-13255.03.patch JIRA Issue HADOOP-13255 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 2a89ec8692fa 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 28b66ae Default Java 1.8.0_91 findbugs v3.0.0 whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/9757/artifact/patchprocess/whitespace-eol.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9757/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9757/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-kms.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9757/testReport/ modules C: hadoop-common-project/hadoop-minikdc hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms U: hadoop-common-project Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9757/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        xiaochen Xiao Chen added a comment -

        Test failed due to a doAs, reattaching a patch 3.

        Show
        xiaochen Xiao Chen added a comment - Test failed due to a doAs, reattaching a patch 3.
        Hide
        xiaochen Xiao Chen added a comment -

        Hm, I cannot reproduce the test failure of Cannot get a KDC reply.... But it's weird that client/host was picked up. I don't think this is a bug coming from this patch. Updating patch 4 to use client/host.

        2016-06-13 08:24:35,993 ERROR DefaultKdcHandler - Error occured while processing request:
        org.apache.kerby.kerberos.kerb.KrbException: Integrity check on decrypted field failed
        	at org.apache.kerby.kerberos.kerb.crypto.enc.KeKiEnc.decryptWith(KeKiEnc.java:127)
        	at org.apache.kerby.kerberos.kerb.crypto.enc.AbstractEncTypeHandler.decrypt(AbstractEncTypeHandler.java:150)
        	at org.apache.kerby.kerberos.kerb.crypto.enc.AbstractEncTypeHandler.decrypt(AbstractEncTypeHandler.java:138)
        	at org.apache.kerby.kerberos.kerb.crypto.EncryptionHandler.decrypt(EncryptionHandler.java:228)
        	at org.apache.kerby.kerberos.kerb.common.EncryptionUtil.unseal(EncryptionUtil.java:136)
        	at org.apache.kerby.kerberos.kerb.server.preauth.builtin.EncTsPreauth.verify(EncTsPreauth.java:51)
        	at org.apache.kerby.kerberos.kerb.server.preauth.PreauthHandle.verify(PreauthHandle.java:46)
        	at org.apache.kerby.kerberos.kerb.server.preauth.PreauthHandler.verify(PreauthHandler.java:101)
        	at org.apache.kerby.kerberos.kerb.server.request.KdcRequest.preauth(KdcRequest.java:562)
        	at org.apache.kerby.kerberos.kerb.server.request.KdcRequest.process(KdcRequest.java:181)
        	at org.apache.kerby.kerberos.kerb.server.KdcHandler.handleMessage(KdcHandler.java:115)
        	at org.apache.kerby.kerberos.kerb.server.impl.DefaultKdcHandler.handleMessage(DefaultKdcHandler.java:67)
        	at org.apache.kerby.kerberos.kerb.server.impl.DefaultKdcHandler.run(DefaultKdcHandler.java:52)
        	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        	at java.lang.Thread.run(Thread.java:745)
        2016-06-13 08:24:35,994 WARN  LoadBalancingKMSClientProvider - KMS provider at [http://localhost:40216/kms/v1/] threw an IOException [Login failure for client/host from keytab /testptch/hadoop/hadoop-common-project/hadoop-kms/target/af48f139-dd06-4616-bedd-6b098449b503/keytab: javax.security.auth.login.LoginException: Cannot get a KDC reply]!!
        
        Show
        xiaochen Xiao Chen added a comment - Hm, I cannot reproduce the test failure of Cannot get a KDC reply .... But it's weird that client/host was picked up. I don't think this is a bug coming from this patch. Updating patch 4 to use client/host. 2016-06-13 08:24:35,993 ERROR DefaultKdcHandler - Error occured while processing request: org.apache.kerby.kerberos.kerb.KrbException: Integrity check on decrypted field failed at org.apache.kerby.kerberos.kerb.crypto.enc.KeKiEnc.decryptWith(KeKiEnc.java:127) at org.apache.kerby.kerberos.kerb.crypto.enc.AbstractEncTypeHandler.decrypt(AbstractEncTypeHandler.java:150) at org.apache.kerby.kerberos.kerb.crypto.enc.AbstractEncTypeHandler.decrypt(AbstractEncTypeHandler.java:138) at org.apache.kerby.kerberos.kerb.crypto.EncryptionHandler.decrypt(EncryptionHandler.java:228) at org.apache.kerby.kerberos.kerb.common.EncryptionUtil.unseal(EncryptionUtil.java:136) at org.apache.kerby.kerberos.kerb.server.preauth.builtin.EncTsPreauth.verify(EncTsPreauth.java:51) at org.apache.kerby.kerberos.kerb.server.preauth.PreauthHandle.verify(PreauthHandle.java:46) at org.apache.kerby.kerberos.kerb.server.preauth.PreauthHandler.verify(PreauthHandler.java:101) at org.apache.kerby.kerberos.kerb.server.request.KdcRequest.preauth(KdcRequest.java:562) at org.apache.kerby.kerberos.kerb.server.request.KdcRequest.process(KdcRequest.java:181) at org.apache.kerby.kerberos.kerb.server.KdcHandler.handleMessage(KdcHandler.java:115) at org.apache.kerby.kerberos.kerb.server.impl.DefaultKdcHandler.handleMessage(DefaultKdcHandler.java:67) at org.apache.kerby.kerberos.kerb.server.impl.DefaultKdcHandler.run(DefaultKdcHandler.java:52) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 2016-06-13 08:24:35,994 WARN LoadBalancingKMSClientProvider - KMS provider at [http://localhost:40216/kms/v1/] threw an IOException [Login failure for client/host from keytab /testptch/hadoop/hadoop-common-project/hadoop-kms/target/af48f139-dd06-4616-bedd-6b098449b503/keytab: javax.security.auth.login.LoginException: Cannot get a KDC reply]!!
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 27s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        0 mvndep 0m 12s Maven dependency ordering for branch
        +1 mvninstall 7m 1s trunk passed
        +1 compile 7m 15s trunk passed
        +1 checkstyle 0m 30s trunk passed
        +1 mvnsite 1m 31s trunk passed
        +1 mvneclipse 0m 31s trunk passed
        +1 findbugs 2m 14s trunk passed
        +1 javadoc 1m 9s trunk passed
        0 mvndep 0m 7s Maven dependency ordering for patch
        +1 mvninstall 1m 6s the patch passed
        +1 compile 7m 18s the patch passed
        +1 javac 7m 18s the patch passed
        -1 checkstyle 0m 30s hadoop-common-project: The patch generated 1 new + 234 unchanged - 0 fixed = 235 total (was 234)
        +1 mvnsite 1m 26s the patch passed
        +1 mvneclipse 0m 30s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 2m 37s the patch passed
        +1 javadoc 1m 10s the patch passed
        +1 unit 0m 19s hadoop-minikdc in the patch passed.
        +1 unit 8m 30s hadoop-common in the patch passed.
        -1 unit 1m 32s hadoop-kms in the patch failed.
        +1 asflicense 0m 20s The patch does not generate ASF License warnings.
        48m 7s



        Reason Tests
        Failed junit tests hadoop.crypto.key.kms.server.TestKMS



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:e2f6409
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12810688/HADOOP-13255.04.patch
        JIRA Issue HADOOP-13255
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 1a971f625b06 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / e2f6409
        Default Java 1.8.0_91
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/9777/artifact/patchprocess/diff-checkstyle-hadoop-common-project.txt
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9777/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-kms.txt
        Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9777/testReport/
        modules C: hadoop-common-project/hadoop-minikdc hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms U: hadoop-common-project
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9777/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 27s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. 0 mvndep 0m 12s Maven dependency ordering for branch +1 mvninstall 7m 1s trunk passed +1 compile 7m 15s trunk passed +1 checkstyle 0m 30s trunk passed +1 mvnsite 1m 31s trunk passed +1 mvneclipse 0m 31s trunk passed +1 findbugs 2m 14s trunk passed +1 javadoc 1m 9s trunk passed 0 mvndep 0m 7s Maven dependency ordering for patch +1 mvninstall 1m 6s the patch passed +1 compile 7m 18s the patch passed +1 javac 7m 18s the patch passed -1 checkstyle 0m 30s hadoop-common-project: The patch generated 1 new + 234 unchanged - 0 fixed = 235 total (was 234) +1 mvnsite 1m 26s the patch passed +1 mvneclipse 0m 30s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 2m 37s the patch passed +1 javadoc 1m 10s the patch passed +1 unit 0m 19s hadoop-minikdc in the patch passed. +1 unit 8m 30s hadoop-common in the patch passed. -1 unit 1m 32s hadoop-kms in the patch failed. +1 asflicense 0m 20s The patch does not generate ASF License warnings. 48m 7s Reason Tests Failed junit tests hadoop.crypto.key.kms.server.TestKMS Subsystem Report/Notes Docker Image:yetus/hadoop:e2f6409 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12810688/HADOOP-13255.04.patch JIRA Issue HADOOP-13255 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 1a971f625b06 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / e2f6409 Default Java 1.8.0_91 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/9777/artifact/patchprocess/diff-checkstyle-hadoop-common-project.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9777/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-kms.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9777/testReport/ modules C: hadoop-common-project/hadoop-minikdc hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms U: hadoop-common-project Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9777/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        xiaochen Xiao Chen added a comment -

        The test failure looks kerby to me... I cannot reproduce it locally, so let me put up a test patch to see how it runs. Sorry for the spam.

        Show
        xiaochen Xiao Chen added a comment - The test failure looks kerby to me... I cannot reproduce it locally, so let me put up a test patch to see how it runs. Sorry for the spam.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 31s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        0 mvndep 0m 12s Maven dependency ordering for branch
        +1 mvninstall 8m 18s trunk passed
        +1 compile 8m 20s trunk passed
        +1 checkstyle 0m 29s trunk passed
        +1 mvnsite 1m 33s trunk passed
        +1 mvneclipse 0m 40s trunk passed
        +1 findbugs 2m 9s trunk passed
        +1 javadoc 1m 10s trunk passed
        0 mvndep 0m 7s Maven dependency ordering for patch
        +1 mvninstall 1m 18s the patch passed
        +1 compile 7m 32s the patch passed
        +1 javac 7m 32s the patch passed
        -1 checkstyle 0m 30s hadoop-common-project: The patch generated 1 new + 234 unchanged - 0 fixed = 235 total (was 234)
        +1 mvnsite 1m 30s the patch passed
        +1 mvneclipse 0m 43s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 2m 40s the patch passed
        +1 javadoc 1m 12s the patch passed
        +1 unit 0m 31s hadoop-minikdc in the patch passed.
        -1 unit 9m 53s hadoop-common in the patch failed.
        -1 unit 1m 45s hadoop-kms in the patch failed.
        +1 asflicense 0m 26s The patch does not generate ASF License warnings.
        52m 30s



        Reason Tests
        Failed junit tests hadoop.metrics2.impl.TestGangliaMetrics
          hadoop.crypto.key.kms.server.TestKMS



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:e2f6409
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12810772/HADOOP-13255.test.patch
        JIRA Issue HADOOP-13255
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux c07cb7298536 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 25064fb
        Default Java 1.8.0_91
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/9782/artifact/patchprocess/diff-checkstyle-hadoop-common-project.txt
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9782/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9782/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-kms.txt
        Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9782/testReport/
        modules C: hadoop-common-project/hadoop-minikdc hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms U: hadoop-common-project
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9782/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 31s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. 0 mvndep 0m 12s Maven dependency ordering for branch +1 mvninstall 8m 18s trunk passed +1 compile 8m 20s trunk passed +1 checkstyle 0m 29s trunk passed +1 mvnsite 1m 33s trunk passed +1 mvneclipse 0m 40s trunk passed +1 findbugs 2m 9s trunk passed +1 javadoc 1m 10s trunk passed 0 mvndep 0m 7s Maven dependency ordering for patch +1 mvninstall 1m 18s the patch passed +1 compile 7m 32s the patch passed +1 javac 7m 32s the patch passed -1 checkstyle 0m 30s hadoop-common-project: The patch generated 1 new + 234 unchanged - 0 fixed = 235 total (was 234) +1 mvnsite 1m 30s the patch passed +1 mvneclipse 0m 43s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 2m 40s the patch passed +1 javadoc 1m 12s the patch passed +1 unit 0m 31s hadoop-minikdc in the patch passed. -1 unit 9m 53s hadoop-common in the patch failed. -1 unit 1m 45s hadoop-kms in the patch failed. +1 asflicense 0m 26s The patch does not generate ASF License warnings. 52m 30s Reason Tests Failed junit tests hadoop.metrics2.impl.TestGangliaMetrics   hadoop.crypto.key.kms.server.TestKMS Subsystem Report/Notes Docker Image:yetus/hadoop:e2f6409 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12810772/HADOOP-13255.test.patch JIRA Issue HADOOP-13255 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux c07cb7298536 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 25064fb Default Java 1.8.0_91 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/9782/artifact/patchprocess/diff-checkstyle-hadoop-common-project.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9782/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9782/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-kms.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9782/testReport/ modules C: hadoop-common-project/hadoop-minikdc hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms U: hadoop-common-project Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9782/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        xiaochen Xiao Chen added a comment -

        The test failure was due to cross-testcase clean up. Patch 5 should pass.

        Xiaoyu Yao / Zhe Zhang, could you take a look and share your thoughts? I think this patch is correct, and personally wants to let the fix to be more generic. Thanks again!

        Show
        xiaochen Xiao Chen added a comment - The test failure was due to cross-testcase clean up. Patch 5 should pass. Xiaoyu Yao / Zhe Zhang , could you take a look and share your thoughts? I think this patch is correct, and personally wants to let the fix to be more generic. Thanks again!
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 24s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        0 mvndep 0m 6s Maven dependency ordering for branch
        +1 mvninstall 7m 13s trunk passed
        +1 compile 7m 47s trunk passed
        +1 checkstyle 0m 29s trunk passed
        +1 mvnsite 1m 31s trunk passed
        +1 mvneclipse 0m 41s trunk passed
        +1 findbugs 2m 13s trunk passed
        +1 javadoc 1m 7s trunk passed
        0 mvndep 0m 7s Maven dependency ordering for patch
        +1 mvninstall 1m 25s the patch passed
        +1 compile 7m 40s the patch passed
        +1 javac 7m 40s the patch passed
        +1 checkstyle 0m 33s the patch passed
        +1 mvnsite 1m 40s the patch passed
        +1 mvneclipse 0m 34s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 2m 41s the patch passed
        +1 javadoc 1m 16s the patch passed
        +1 unit 0m 19s hadoop-minikdc in the patch passed.
        -1 unit 17m 11s hadoop-common in the patch failed.
        +1 unit 2m 2s hadoop-kms in the patch passed.
        +1 asflicense 0m 27s The patch does not generate ASF License warnings.
        58m 22s



        Reason Tests
        Timed out junit tests org.apache.hadoop.http.TestHttpServerLifecycle



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:e2f6409
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12810928/HADOOP-13255.05.patch
        JIRA Issue HADOOP-13255
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux dac1aef6864e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 6f0aa75
        Default Java 1.8.0_91
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9789/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
        Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9789/testReport/
        modules C: hadoop-common-project/hadoop-minikdc hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms U: hadoop-common-project
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9789/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 24s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. 0 mvndep 0m 6s Maven dependency ordering for branch +1 mvninstall 7m 13s trunk passed +1 compile 7m 47s trunk passed +1 checkstyle 0m 29s trunk passed +1 mvnsite 1m 31s trunk passed +1 mvneclipse 0m 41s trunk passed +1 findbugs 2m 13s trunk passed +1 javadoc 1m 7s trunk passed 0 mvndep 0m 7s Maven dependency ordering for patch +1 mvninstall 1m 25s the patch passed +1 compile 7m 40s the patch passed +1 javac 7m 40s the patch passed +1 checkstyle 0m 33s the patch passed +1 mvnsite 1m 40s the patch passed +1 mvneclipse 0m 34s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 2m 41s the patch passed +1 javadoc 1m 16s the patch passed +1 unit 0m 19s hadoop-minikdc in the patch passed. -1 unit 17m 11s hadoop-common in the patch failed. +1 unit 2m 2s hadoop-kms in the patch passed. +1 asflicense 0m 27s The patch does not generate ASF License warnings. 58m 22s Reason Tests Timed out junit tests org.apache.hadoop.http.TestHttpServerLifecycle Subsystem Report/Notes Docker Image:yetus/hadoop:e2f6409 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12810928/HADOOP-13255.05.patch JIRA Issue HADOOP-13255 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux dac1aef6864e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 6f0aa75 Default Java 1.8.0_91 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9789/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9789/testReport/ modules C: hadoop-common-project/hadoop-minikdc hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms U: hadoop-common-project Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9789/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        xyao Xiaoyu Yao added a comment -

        Thanks Xiao Chen for the clarification. The change makes sense to me after recheck the code with the stack attached. +1 for the v05 patch.
        I will commit it by EOD today in case Zhe Zhang and others have additional comments.

        Show
        xyao Xiaoyu Yao added a comment - Thanks Xiao Chen for the clarification. The change makes sense to me after recheck the code with the stack attached. +1 for the v05 patch. I will commit it by EOD today in case Zhe Zhang and others have additional comments.
        Hide
        zhz Zhe Zhang added a comment -

        Thanks Xiao for the revs and Xiaoyu for the review. The v05 patch LGTM as well.

        Show
        zhz Zhe Zhang added a comment - Thanks Xiao for the revs and Xiaoyu for the review. The v05 patch LGTM as well.
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-trunk-Commit #9972 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9972/)
        HADOOP-13255. KMSClientProvider should check and renew tgt when doing (xyao: rev b1674caa409ca2c616207acb72aeb2767d28b10c)

        • hadoop-common-project/hadoop-minikdc/src/main/java/org/apache/hadoop/minikdc/MiniKdc.java
        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/kms/KMSClientProvider.java
        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/web/DelegationTokenAuthenticator.java
        • hadoop-common-project/hadoop-kms/src/test/java/org/apache/hadoop/crypto/key/kms/server/TestKMS.java
        • hadoop-common-project/hadoop-kms/src/test/resources/log4j.properties
        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #9972 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9972/ ) HADOOP-13255 . KMSClientProvider should check and renew tgt when doing (xyao: rev b1674caa409ca2c616207acb72aeb2767d28b10c) hadoop-common-project/hadoop-minikdc/src/main/java/org/apache/hadoop/minikdc/MiniKdc.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/kms/KMSClientProvider.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/web/DelegationTokenAuthenticator.java hadoop-common-project/hadoop-kms/src/test/java/org/apache/hadoop/crypto/key/kms/server/TestKMS.java hadoop-common-project/hadoop-kms/src/test/resources/log4j.properties hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java
        Hide
        xiaochen Xiao Chen added a comment -

        Thanks a lot Xiaoyu Yao and Zhe Zhang!

        Show
        xiaochen Xiao Chen added a comment - Thanks a lot Xiaoyu Yao and Zhe Zhang !
        Hide
        xyao Xiaoyu Yao added a comment -

        Thanks Xiao Chen for the contribution and Zhe Zhang for the review. I'v committed the patch to trunk.

        The unit test needs some additional work for branch-2 and branch-2.8/2.7 as kerby is not available in branch-2. if it is not feasible without Kerby, I'm OK to have a separate patch for branch-2 without unit test. Let me know your thoughts Xiao Chen and Zhe Zhang.

        Show
        xyao Xiaoyu Yao added a comment - Thanks Xiao Chen for the contribution and Zhe Zhang for the review. I'v committed the patch to trunk. The unit test needs some additional work for branch-2 and branch-2.8/2.7 as kerby is not available in branch-2. if it is not feasible without Kerby, I'm OK to have a separate patch for branch-2 without unit test. Let me know your thoughts Xiao Chen and Zhe Zhang .
        Hide
        xiaochen Xiao Chen added a comment -

        Thanks Xiaoyu Yao.
        I tried with the directory based minikdc, even if I set the MIN_TICKET_LIFETIME, it ends up with this error if max lifetime is less than 6 mins, which I think is what Zhe met in HADOOP-12559.

        java.io.IOException: org.apache.hadoop.security.authentication.client.AuthenticationException: GSSException: No valid credentials provided (Mechanism level: Requested start time is later than end time (11) - Requested start time is later than end time)
        
        	at org.apache.hadoop.crypto.key.kms.KMSClientProvider.createConnection(KMSClientProvider.java:554)
        	at org.apache.hadoop.crypto.key.kms.KMSClientProvider.getKeys(KMSClientProvider.java:659)
        	at org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider$7.call(LoadBalancingKMSClientProvider.java:235)
        	at org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider$7.call(LoadBalancingKMSClientProvider.java:232)
        	at org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider.doOp(LoadBalancingKMSClientProvider.java:94)
        	at org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider.getKeys(LoadBalancingKMSClientProvider.java:232)
        	at org.apache.hadoop.crypto.key.kms.server.TestKMS$17$1.run(TestKMS.java:2097)
        	at org.apache.hadoop.crypto.key.kms.server.TestKMS$17$1.run(TestKMS.java:2091)
        	at java.security.AccessController.doPrivileged(Native Method)
        	at javax.security.auth.Subject.doAs(Subject.java:415)
        	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1744)
        	at org.apache.hadoop.crypto.key.kms.server.TestKMS$17.call(TestKMS.java:2091)
        	at org.apache.hadoop.crypto.key.kms.server.TestKMS$17.call(TestKMS.java:2081)
        	at org.apache.hadoop.crypto.key.kms.server.TestKMS.runServer(TestKMS.java:141)
        	at org.apache.hadoop.crypto.key.kms.server.TestKMS.runServer(TestKMS.java:123)
        	at org.apache.hadoop.crypto.key.kms.server.TestKMS.testTGTRenewal(TestKMS.java:2081)
        	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        	at java.lang.reflect.Method.invoke(Method.java:606)
        	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
        	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
        	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
        	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
        	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
        	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
        	at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
        Caused by: org.apache.hadoop.security.authentication.client.AuthenticationException: GSSException: No valid credentials provided (Mechanism level: Requested start time is later than end time (11) - Requested start time is later than end time)
        	at org.apache.hadoop.security.authentication.client.KerberosAuthenticator.doSpnegoSequence(KerberosAuthenticator.java:333)
        	at org.apache.hadoop.security.authentication.client.KerberosAuthenticator.authenticate(KerberosAuthenticator.java:203)
        	at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticator.authenticate(DelegationTokenAuthenticator.java:149)
        	at org.apache.hadoop.security.authentication.client.AuthenticatedURL.openConnection(AuthenticatedURL.java:216)
        	at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticatedURL.openConnection(DelegationTokenAuthenticatedURL.java:322)
        	at org.apache.hadoop.crypto.key.kms.KMSClientProvider$1.run(KMSClientProvider.java:545)
        	at org.apache.hadoop.crypto.key.kms.KMSClientProvider$1.run(KMSClientProvider.java:540)
        	at java.security.AccessController.doPrivileged(Native Method)
        	at javax.security.auth.Subject.doAs(Subject.java:415)
        	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1744)
        	at org.apache.hadoop.crypto.key.kms.KMSClientProvider.createConnection(KMSClientProvider.java:540)
        	... 26 more
        Caused by: GSSException: No valid credentials provided (Mechanism level: Requested start time is later than end time (11) - Requested start time is later than end time)
        	at sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:710)
        	at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248)
        	at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
        	at org.apache.hadoop.security.authentication.client.KerberosAuthenticator$1.run(KerberosAuthenticator.java:309)
        	at org.apache.hadoop.security.authentication.client.KerberosAuthenticator$1.run(KerberosAuthenticator.java:285)
        	at java.security.AccessController.doPrivileged(Native Method)
        	at javax.security.auth.Subject.doAs(Subject.java:415)
        	at org.apache.hadoop.security.authentication.client.KerberosAuthenticator.doSpnegoSequence(KerberosAuthenticator.java:285)
        	... 36 more
        Caused by: KrbException: Requested start time is later than end time (11) - Requested start time is later than end time
        	at sun.security.krb5.KrbTgsRep.<init>(KrbTgsRep.java:73)
        	at sun.security.krb5.KrbTgsReq.getReply(KrbTgsReq.java:192)
        	at sun.security.krb5.KrbTgsReq.sendAndGetCreds(KrbTgsReq.java:203)
        	at sun.security.krb5.internal.CredentialsUtil.serviceCreds(CredentialsUtil.java:309)
        	at sun.security.krb5.internal.CredentialsUtil.acquireServiceCreds(CredentialsUtil.java:115)
        	at sun.security.krb5.Credentials.acquireServiceCreds(Credentials.java:454)
        	at sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:641)
        	... 43 more
        Caused by: KrbException: Identifier doesn't match expected value (906)
        	at sun.security.krb5.internal.KDCRep.init(KDCRep.java:143)
        	at sun.security.krb5.internal.TGSRep.init(TGSRep.java:66)
        	at sun.security.krb5.internal.TGSRep.<init>(TGSRep.java:61)
        	at sun.security.krb5.KrbTgsRep.<init>(KrbTgsRep.java:55)
        	... 49 more
        

        So I think we need to go without the test in branch-2. Attached a patch based on latest branch-2.

        Show
        xiaochen Xiao Chen added a comment - Thanks Xiaoyu Yao . I tried with the directory based minikdc, even if I set the MIN_TICKET_LIFETIME , it ends up with this error if max lifetime is less than 6 mins, which I think is what Zhe met in HADOOP-12559 . java.io.IOException: org.apache.hadoop.security.authentication.client.AuthenticationException: GSSException: No valid credentials provided (Mechanism level: Requested start time is later than end time (11) - Requested start time is later than end time) at org.apache.hadoop.crypto.key.kms.KMSClientProvider.createConnection(KMSClientProvider.java:554) at org.apache.hadoop.crypto.key.kms.KMSClientProvider.getKeys(KMSClientProvider.java:659) at org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider$7.call(LoadBalancingKMSClientProvider.java:235) at org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider$7.call(LoadBalancingKMSClientProvider.java:232) at org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider.doOp(LoadBalancingKMSClientProvider.java:94) at org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider.getKeys(LoadBalancingKMSClientProvider.java:232) at org.apache.hadoop.crypto.key.kms.server.TestKMS$17$1.run(TestKMS.java:2097) at org.apache.hadoop.crypto.key.kms.server.TestKMS$17$1.run(TestKMS.java:2091) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1744) at org.apache.hadoop.crypto.key.kms.server.TestKMS$17.call(TestKMS.java:2091) at org.apache.hadoop.crypto.key.kms.server.TestKMS$17.call(TestKMS.java:2081) at org.apache.hadoop.crypto.key.kms.server.TestKMS.runServer(TestKMS.java:141) at org.apache.hadoop.crypto.key.kms.server.TestKMS.runServer(TestKMS.java:123) at org.apache.hadoop.crypto.key.kms.server.TestKMS.testTGTRenewal(TestKMS.java:2081) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74) Caused by: org.apache.hadoop.security.authentication.client.AuthenticationException: GSSException: No valid credentials provided (Mechanism level: Requested start time is later than end time (11) - Requested start time is later than end time) at org.apache.hadoop.security.authentication.client.KerberosAuthenticator.doSpnegoSequence(KerberosAuthenticator.java:333) at org.apache.hadoop.security.authentication.client.KerberosAuthenticator.authenticate(KerberosAuthenticator.java:203) at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticator.authenticate(DelegationTokenAuthenticator.java:149) at org.apache.hadoop.security.authentication.client.AuthenticatedURL.openConnection(AuthenticatedURL.java:216) at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticatedURL.openConnection(DelegationTokenAuthenticatedURL.java:322) at org.apache.hadoop.crypto.key.kms.KMSClientProvider$1.run(KMSClientProvider.java:545) at org.apache.hadoop.crypto.key.kms.KMSClientProvider$1.run(KMSClientProvider.java:540) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1744) at org.apache.hadoop.crypto.key.kms.KMSClientProvider.createConnection(KMSClientProvider.java:540) ... 26 more Caused by: GSSException: No valid credentials provided (Mechanism level: Requested start time is later than end time (11) - Requested start time is later than end time) at sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:710) at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248) at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179) at org.apache.hadoop.security.authentication.client.KerberosAuthenticator$1.run(KerberosAuthenticator.java:309) at org.apache.hadoop.security.authentication.client.KerberosAuthenticator$1.run(KerberosAuthenticator.java:285) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.authentication.client.KerberosAuthenticator.doSpnegoSequence(KerberosAuthenticator.java:285) ... 36 more Caused by: KrbException: Requested start time is later than end time (11) - Requested start time is later than end time at sun.security.krb5.KrbTgsRep.<init>(KrbTgsRep.java:73) at sun.security.krb5.KrbTgsReq.getReply(KrbTgsReq.java:192) at sun.security.krb5.KrbTgsReq.sendAndGetCreds(KrbTgsReq.java:203) at sun.security.krb5.internal.CredentialsUtil.serviceCreds(CredentialsUtil.java:309) at sun.security.krb5.internal.CredentialsUtil.acquireServiceCreds(CredentialsUtil.java:115) at sun.security.krb5.Credentials.acquireServiceCreds(Credentials.java:454) at sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:641) ... 43 more Caused by: KrbException: Identifier doesn't match expected value (906) at sun.security.krb5.internal.KDCRep.init(KDCRep.java:143) at sun.security.krb5.internal.TGSRep.init(TGSRep.java:66) at sun.security.krb5.internal.TGSRep.<init>(TGSRep.java:61) at sun.security.krb5.KrbTgsRep.<init>(KrbTgsRep.java:55) ... 49 more So I think we need to go without the test in branch-2. Attached a patch based on latest branch-2.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 19s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        0 mvndep 0m 15s Maven dependency ordering for branch
        +1 mvninstall 6m 34s branch-2 passed
        +1 compile 5m 33s branch-2 passed with JDK v1.8.0_91
        +1 compile 6m 36s branch-2 passed with JDK v1.7.0_101
        +1 checkstyle 0m 31s branch-2 passed
        +1 mvnsite 1m 23s branch-2 passed
        +1 mvneclipse 0m 30s branch-2 passed
        +1 findbugs 2m 6s branch-2 passed
        +1 javadoc 0m 57s branch-2 passed with JDK v1.8.0_91
        +1 javadoc 1m 9s branch-2 passed with JDK v1.7.0_101
        0 mvndep 0m 9s Maven dependency ordering for patch
        -1 mvninstall 0m 18s hadoop-kms in the patch failed.
        +1 compile 5m 54s the patch passed with JDK v1.8.0_91
        +1 javac 5m 54s the patch passed
        +1 compile 7m 3s the patch passed with JDK v1.7.0_101
        +1 javac 7m 3s the patch passed
        +1 checkstyle 0m 29s the patch passed
        +1 mvnsite 1m 22s the patch passed
        +1 mvneclipse 0m 30s the patch passed
        -1 whitespace 0m 0s The patch has 49 line(s) that end in whitespace. Use git apply --whitespace=fix.
        +1 findbugs 2m 34s the patch passed
        +1 javadoc 0m 57s the patch passed with JDK v1.8.0_91
        +1 javadoc 1m 10s the patch passed with JDK v1.7.0_101
        +1 unit 8m 26s hadoop-common in the patch passed with JDK v1.7.0_101.
        +1 unit 1m 37s hadoop-kms in the patch passed with JDK v1.7.0_101.
        +1 asflicense 0m 21s The patch does not generate ASF License warnings.
        80m 3s



        Reason Tests
        JDK v1.8.0_91 Timed out junit tests org.apache.hadoop.http.TestHttpServerLifecycle



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:d1c475d
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12811245/HADOOP-13255.branch-2.patch
        JIRA Issue HADOOP-13255
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux e9759d5e72d1 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision branch-2 / 49c0d9b
        Default Java 1.7.0_101
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_101
        findbugs v3.0.0
        mvninstall https://builds.apache.org/job/PreCommit-HADOOP-Build/9806/artifact/patchprocess/patch-mvninstall-hadoop-common-project_hadoop-kms.txt
        whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/9806/artifact/patchprocess/whitespace-eol.txt
        JDK v1.7.0_101 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9806/testReport/
        modules C: hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms U: hadoop-common-project
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9806/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 19s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 15s Maven dependency ordering for branch +1 mvninstall 6m 34s branch-2 passed +1 compile 5m 33s branch-2 passed with JDK v1.8.0_91 +1 compile 6m 36s branch-2 passed with JDK v1.7.0_101 +1 checkstyle 0m 31s branch-2 passed +1 mvnsite 1m 23s branch-2 passed +1 mvneclipse 0m 30s branch-2 passed +1 findbugs 2m 6s branch-2 passed +1 javadoc 0m 57s branch-2 passed with JDK v1.8.0_91 +1 javadoc 1m 9s branch-2 passed with JDK v1.7.0_101 0 mvndep 0m 9s Maven dependency ordering for patch -1 mvninstall 0m 18s hadoop-kms in the patch failed. +1 compile 5m 54s the patch passed with JDK v1.8.0_91 +1 javac 5m 54s the patch passed +1 compile 7m 3s the patch passed with JDK v1.7.0_101 +1 javac 7m 3s the patch passed +1 checkstyle 0m 29s the patch passed +1 mvnsite 1m 22s the patch passed +1 mvneclipse 0m 30s the patch passed -1 whitespace 0m 0s The patch has 49 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 findbugs 2m 34s the patch passed +1 javadoc 0m 57s the patch passed with JDK v1.8.0_91 +1 javadoc 1m 10s the patch passed with JDK v1.7.0_101 +1 unit 8m 26s hadoop-common in the patch passed with JDK v1.7.0_101. +1 unit 1m 37s hadoop-kms in the patch passed with JDK v1.7.0_101. +1 asflicense 0m 21s The patch does not generate ASF License warnings. 80m 3s Reason Tests JDK v1.8.0_91 Timed out junit tests org.apache.hadoop.http.TestHttpServerLifecycle Subsystem Report/Notes Docker Image:yetus/hadoop:d1c475d JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12811245/HADOOP-13255.branch-2.patch JIRA Issue HADOOP-13255 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux e9759d5e72d1 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2 / 49c0d9b Default Java 1.7.0_101 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_101 findbugs v3.0.0 mvninstall https://builds.apache.org/job/PreCommit-HADOOP-Build/9806/artifact/patchprocess/patch-mvninstall-hadoop-common-project_hadoop-kms.txt whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/9806/artifact/patchprocess/whitespace-eol.txt JDK v1.7.0_101 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9806/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms U: hadoop-common-project Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9806/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        xyao Xiaoyu Yao added a comment - - edited

        Thanks Xiao Chen for the branch-2 patch. I've commit the patch (with git apply --whitespace=fix) to trunk, branch-2, branch-2.8, branch-2.7.3 and branch-2.6.5.

        Show
        xyao Xiaoyu Yao added a comment - - edited Thanks Xiao Chen for the branch-2 patch. I've commit the patch (with git apply --whitespace=fix) to trunk, branch-2, branch-2.8, branch-2.7.3 and branch-2.6.5.
        Hide
        xiaochen Xiao Chen added a comment -

        Thanks again Xiaoyu for the backports.

        Show
        xiaochen Xiao Chen added a comment - Thanks again Xiaoyu for the backports.
        Hide
        zhz Zhe Zhang added a comment -

        Sorry for chimming in late. Yes I was having a hard time setting the expiry time shorter than 6 mins. I think it is reasonable to backport the patch without the unit test to branch-2 and downward.

        Show
        zhz Zhe Zhang added a comment - Sorry for chimming in late. Yes I was having a hard time setting the expiry time shorter than 6 mins. I think it is reasonable to backport the patch without the unit test to branch-2 and downward.
        Hide
        vinodkv Vinod Kumar Vavilapalli added a comment -

        Closing the JIRA as part of 2.7.3 release.

        Show
        vinodkv Vinod Kumar Vavilapalli added a comment - Closing the JIRA as part of 2.7.3 release.

          People

          • Assignee:
            xiaochen Xiao Chen
            Reporter:
            xiaochen Xiao Chen
          • Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development