Xiaoyu Yao added a comment - 28/Dec/15 21:24 Good catch, thanks for reporting this Wei-Chiu Chuang ! Somehow Jenkins passed with only the client side KMS unit tests run for HADOOP-12559 .

Wei-Chiu Chuang added a comment - 29/Dec/15 02:15 Rev01: create a helper method doAsFromKeytab() which invokes UserGroupInformation.loginUserFromKeytab(), and use this helper method instead of doAs() in doKMSRestart(). The tests in TestKMS passed.

Wei-Chiu Chuang added a comment - 29/Dec/15 02:17 Xiaoyu Yao I have limited knowledge of KMS. Even though the rev01 patch passed TestKMS tests, I am unsure if this is the right approach. Also, I am not sure if other test cases in TestKMS should invoke UserGroupInformation.loginUserFromKeytab() as well.

Wei-Chiu Chuang added a comment - 29/Dec/15 02:57 Xiaoyu Yao We should probably file a YETUS jira to have jenkins test server side KMS as well.

Hadoop QA added a comment - 29/Dec/15 03:20 +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 9m 18s trunk passed +1 compile 10m 47s trunk passed with JDK v1.8.0_66 +1 compile 10m 51s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 11s trunk passed +1 mvnsite 0m 26s trunk passed +1 mvneclipse 0m 17s trunk passed +1 findbugs 0m 32s trunk passed +1 javadoc 0m 15s trunk passed with JDK v1.8.0_66 +1 javadoc 0m 17s trunk passed with JDK v1.7.0_91 +1 mvninstall 0m 21s the patch passed +1 compile 10m 28s the patch passed with JDK v1.8.0_66 +1 javac 10m 28s the patch passed +1 compile 10m 41s the patch passed with JDK v1.7.0_91 +1 javac 10m 41s the patch passed +1 checkstyle 0m 10s the patch passed +1 mvnsite 0m 25s the patch passed +1 mvneclipse 0m 17s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 0m 42s the patch passed +1 javadoc 0m 18s the patch passed with JDK v1.8.0_66 +1 javadoc 0m 19s the patch passed with JDK v1.7.0_91 +1 unit 1m 43s hadoop-kms in the patch passed with JDK v1.8.0_66. +1 unit 1m 42s hadoop-kms in the patch passed with JDK v1.7.0_91. +1 asflicense 0m 25s Patch does not generate ASF License warnings. 61m 54s Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12779737/HADOOP-12682.001.patch JIRA Issue HADOOP-12682 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 290e43182626 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / d0a22ba Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs v3.0.0 JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8319/testReport/ modules C: hadoop-common-project/hadoop-kms U: hadoop-common-project/hadoop-kms Max memory used 76MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8319/console This message was automatically generated.

Wei-Chiu Chuang added a comment - 29/Dec/15 20:43 All precommit tests passed. Xiaoyu Yao may I ask you for a review? Thank you

Xiaoyu Yao added a comment - 29/Dec/15 21:42 Thanks Wei-Chiu Chuang for working on this. The patch looks good to me overall and I think you are on the right direction for the fix. Here are some of my comments: 1. The root cause of this issue is TestKMS#doAs() uses UserGroupInformation.getUGIFromSubject() to retrieve UGI for KMS restart tests. UGI returned from getUGIFromSubject does not contain any loginContext or Keytab file information. When this UGI is later invoked by the newly added code below in KMSClientProvider, it failed at UGI#reloginFromKeytab() because the the login requires loginContext and keytab information. // check and renew TGT to handle potential expiration actualUgi.checkTGTAndReloginFromKeytab(); 2. We should use try/finally to logout the user after ugi.doAs() in the new TestKMS#doAsFromKeytab. Ideally, we should update TestKMS#doAs directly to ensure it use the correct/compete UGI. This will need to add a new public logout method in the UGI class.

Wei-Chiu Chuang added a comment - 29/Dec/15 23:26 Thanks Xiaoyu Yao for the first review. I posted rev02 patch. In this patch, TestKMS#doAs() calls UGI.loginUserFromKeytab() and UGI.logoutUserFromKeytab(). UGI.logoutUserFromKeytab() is a new UserGroupInformation method that logs out a user who previously logs in using keytab (by invoking loginUserFromKeytab)

Hadoop QA added a comment - 30/Dec/15 00:53 -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 8m 10s trunk passed +1 compile 9m 13s trunk passed with JDK v1.8.0_66 +1 compile 9m 31s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 23s trunk passed +1 mvnsite 1m 28s trunk passed +1 mvneclipse 0m 27s trunk passed +1 findbugs 2m 27s trunk passed +1 javadoc 1m 13s trunk passed with JDK v1.8.0_66 +1 javadoc 1m 23s trunk passed with JDK v1.7.0_91 +1 mvninstall 1m 54s the patch passed +1 compile 9m 9s the patch passed with JDK v1.8.0_66 +1 javac 9m 9s the patch passed +1 compile 9m 33s the patch passed with JDK v1.7.0_91 +1 javac 9m 33s the patch passed -1 checkstyle 0m 24s Patch generated 1 new checkstyle issues in hadoop-common-project (total was 108, now 109). +1 mvnsite 1m 27s the patch passed +1 mvneclipse 0m 28s the patch passed +1 whitespace 0m 1s Patch has no whitespace issues. +1 findbugs 2m 42s the patch passed +1 javadoc 1m 11s the patch passed with JDK v1.8.0_66 +1 javadoc 1m 23s the patch passed with JDK v1.7.0_91 +1 unit 8m 22s hadoop-common in the patch passed with JDK v1.8.0_66. +1 unit 1m 35s hadoop-kms in the patch passed with JDK v1.8.0_66. +1 unit 8m 27s hadoop-common in the patch passed with JDK v1.7.0_91. +1 unit 1m 40s hadoop-kms in the patch passed with JDK v1.7.0_91. +1 asflicense 0m 24s Patch does not generate ASF License warnings. 84m 24s Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12779875/HADOOP-12682.002.patch JIRA Issue HADOOP-12682 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux c00ab7a4063d 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 84a8147 Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/8323/artifact/patchprocess/diff-checkstyle-hadoop-common-project.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8323/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms U: hadoop-common-project Max memory used 76MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8323/console This message was automatically generated.

Xiaoyu Yao added a comment - 30/Dec/15 01:20 Thanks Wei-Chiu Chuang for updating the patch. Patch v2 looks good to me. Can you address the checkstyle issue by ensuring {} for the if and remove ": " + le from the IOException message string? } catch (LoginException le) { throw new IOException( "Logout failure for " + user + " from keytab " + keytabFile + ": " + le, le); }

Wei-Chiu Chuang added a comment - 30/Dec/15 01:30 Thanks for the review! Rev03 fixed the checkstyle issue and updated the exception message.

Hadoop QA added a comment - 30/Dec/15 03:05 -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 8m 57s trunk passed +1 compile 10m 19s trunk passed with JDK v1.8.0_66 +1 compile 10m 33s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 25s trunk passed +1 mvnsite 1m 43s trunk passed +1 mvneclipse 0m 29s trunk passed +1 findbugs 2m 36s trunk passed +1 javadoc 1m 10s trunk passed with JDK v1.8.0_66 +1 javadoc 1m 23s trunk passed with JDK v1.7.0_91 +1 mvninstall 1m 58s the patch passed +1 compile 9m 16s the patch passed with JDK v1.8.0_66 +1 javac 9m 16s the patch passed +1 compile 9m 27s the patch passed with JDK v1.7.0_91 +1 javac 9m 27s the patch passed +1 checkstyle 0m 23s the patch passed +1 mvnsite 1m 26s the patch passed +1 mvneclipse 0m 28s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 2m 42s the patch passed +1 javadoc 1m 12s the patch passed with JDK v1.8.0_66 +1 javadoc 1m 21s the patch passed with JDK v1.7.0_91 -1 unit 8m 4s hadoop-common in the patch failed with JDK v1.8.0_66. +1 unit 1m 33s hadoop-kms in the patch passed with JDK v1.8.0_66. +1 unit 8m 18s hadoop-common in the patch passed with JDK v1.7.0_91. +1 unit 1m 38s hadoop-kms in the patch passed with JDK v1.7.0_91. +1 asflicense 0m 23s Patch does not generate ASF License warnings. 87m 17s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.ipc.TestIPC Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12779891/HADOOP-12682.003.patch JIRA Issue HADOOP-12682 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 83b9e5413896 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / ad997fa Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8324/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8324/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8324/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms U: hadoop-common-project Max memory used 76MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8324/console This message was automatically generated.

Wei-Chiu Chuang added a comment - 30/Dec/15 03:24 Test failure is in TestIPC, which looks unrelated.

Xiaoyu Yao added a comment - 30/Dec/15 17:42 Thanks Wei-Chiu Chuang for updating the patch. +1 for the v03 patch and I will commit it shortly.

Xiaoyu Yao added a comment - 30/Dec/15 18:34 Thanks Wei-Chiu Chuang for the contribution. I've commit the fix to trunk, branch-2 and branch-2.8.

Hudson added a comment - 30/Dec/15 18:47 FAILURE: Integrated in Hadoop-trunk-Commit #9039 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9039/ ) HADOOP-12682 . Fix TestKMS#testKMSRestart* failure. Contributed by (xyao: rev ab725cff66e8a047e9437e42ac49ac8685ee7a94) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-kms/src/test/java/org/apache/hadoop/crypto/key/kms/server/TestKMS.java

Wei-Chiu Chuang added a comment - 30/Dec/15 18:53 Thank you Xiaoyu Yao for the commit and review!

Zhe Zhang added a comment - 22/Mar/16 21:05 I just backported the patch to branch-2.7 and branch-2.6 per discussion under HADOOP-12559

Vinod Kumar Vavilapalli added a comment - 25/Aug/16 22:48 Closing the JIRA as part of 2.7.3 release.