John Zhuge added a comment - 02/Aug/16 06:52 Patch 001: Make class Token$PrivateToken private Add method Token#privateClone to create a private clone of a public token Replace all instanceof Token.PrivateToken and typecasting with polymophic methods

Hadoop QA added a comment - 02/Aug/16 08:56 -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 19s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 16s Maven dependency ordering for branch +1 mvninstall 7m 25s trunk passed +1 compile 7m 27s trunk passed +1 checkstyle 1m 33s trunk passed -1 mvnsite 2m 25s hadoop-common in trunk failed. +1 mvneclipse 0m 26s trunk passed +1 findbugs 4m 1s trunk passed +1 javadoc 1m 49s trunk passed 0 mvndep 0m 15s Maven dependency ordering for patch +1 mvninstall 1m 34s the patch passed +1 compile 7m 24s the patch passed +1 javac 7m 24s the patch passed +1 checkstyle 1m 35s root: The patch generated 0 new + 175 unchanged - 2 fixed = 175 total (was 177) +1 mvnsite 1m 56s the patch passed +1 mvneclipse 0m 27s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 4m 2s the patch passed +1 javadoc 1m 50s the patch passed +1 unit 7m 53s hadoop-common in the patch passed. -1 unit 62m 54s hadoop-hdfs in the patch failed. +1 asflicense 0m 29s The patch does not generate ASF License warnings. 117m 55s Reason Tests Failed junit tests hadoop.hdfs.TestFileCorruption Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12821520/HDFS-10683.001.patch JIRA Issue HDFS-10683 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux c07c6e69cdec 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 6890d5b Default Java 1.8.0_101 mvnsite https://builds.apache.org/job/PreCommit-HDFS-Build/16285/artifact/patchprocess/branch-mvnsite-hadoop-common-project_hadoop-common.txt findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/16285/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16285/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16285/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.

John Zhuge added a comment - 02/Aug/16 16:26 TestFileCorruption passes locally. The link for row unit is not valid (error 404).

John Zhuge added a comment - 23/Sep/16 20:45 Patch 002: Rebase

Hadoop QA added a comment - 23/Sep/16 23:43 -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 20s Maven dependency ordering for branch +1 mvninstall 8m 24s trunk passed +1 compile 8m 41s trunk passed +1 checkstyle 1m 36s trunk passed +1 mvnsite 2m 4s trunk passed +1 mvneclipse 0m 28s trunk passed +1 findbugs 3m 30s trunk passed +1 javadoc 1m 52s trunk passed 0 mvndep 0m 14s Maven dependency ordering for patch +1 mvninstall 1m 38s the patch passed +1 compile 7m 21s the patch passed +1 javac 7m 21s the patch passed +1 checkstyle 1m 27s root: The patch generated 0 new + 174 unchanged - 2 fixed = 174 total (was 176) +1 mvnsite 1m 45s the patch passed +1 mvneclipse 0m 25s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 3m 19s the patch passed +1 javadoc 1m 39s the patch passed +1 unit 7m 23s hadoop-common in the patch passed. -1 unit 59m 2s hadoop-hdfs in the patch failed. +1 asflicense 0m 25s The patch does not generate ASF License warnings. 112m 40s Reason Tests Failed junit tests hadoop.hdfs.TestFileChecksum   hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HDFS-10683 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12830110/HDFS-10683.002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 7f17313592d3 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 6eb700e Default Java 1.8.0_101 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/16846/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16846/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16846/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.

Wei-Chiu Chuang added a comment - 29/Sep/16 05:26 +1

Wei-Chiu Chuang added a comment - 29/Sep/16 13:20 Retract my +1. The test failure of TestFileChecksum is related. John Zhuge please take a look again at your patch.

John Zhuge added a comment - 29/Sep/16 16:58 Sure Wei-Chiu Chuang , I will take a look.

John Zhuge added a comment - 29/Sep/16 20:27 Wei-Chiu Chuang From the test output, these 2 test failures seem unrelated to the patch: testCannotUpgradeSecondNameNode(org.apache.hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA) Time elapsed: 0.506 sec <<< ERROR! java.net.BindException: Port in use: localhost:40448 at sun.nio.ch.Net.bind0(Native Method) ... testStripedFileChecksumWithMissedDataBlocksRangeQuery15(org.apache.hadoop.hdfs.TestFileChecksum) Time elapsed: 4.122 sec <<< ERROR! java.net.BindException: Problem binding to [localhost:59525] java.net.BindException: Address already in use; For more details see: http://wiki.apache.org/hadoop/BindException at sun.nio.ch.Net.bind0(Native Method)

Wei-Chiu Chuang added a comment - 30/Sep/16 13:16 Weird. I sawTestFileChecksum has 9 test failures even after reverting the patch java.lang.AssertionError: Checksum mismatches! at org.junit.Assert.fail(Assert.java:88) at org.junit.Assert.assertTrue(Assert.java:41) at org.apache.hadoop.hdfs.TestFileChecksum.testStripedFileChecksumWithMissedDataBlocksRangeQuery(TestFileChecksum.java:227) at org.apache.hadoop.hdfs.TestFileChecksum.testStripedFileChecksumWithMissedDataBlocksRangeQuery10(TestFileChecksum.java:336) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)

Wei-Chiu Chuang added a comment - 30/Sep/16 13:56 The tests failed after HDFS-10460 . Maybe it only fails for Mac. I'll file a jira.

Wei-Chiu Chuang added a comment - 30/Sep/16 15:32 Triggered a rebuild to see if the failed test are reproducible.

John Zhuge added a comment - 04/Oct/16 17:18 Re-submit 002 to trigger test again.

Wei-Chiu Chuang added a comment - 05/Oct/16 21:26 Manually triggered precommit. You can monitor the job here: https://builds.apache.org/job/PreCommit-HDFS-Build/17027/

Hadoop QA added a comment - 05/Oct/16 23:40 +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 24s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 18s Maven dependency ordering for branch +1 mvninstall 8m 36s trunk passed +1 compile 8m 19s trunk passed +1 checkstyle 1m 44s trunk passed +1 mvnsite 2m 8s trunk passed +1 mvneclipse 0m 29s trunk passed +1 findbugs 3m 26s trunk passed +1 javadoc 2m 7s trunk passed 0 mvndep 0m 17s Maven dependency ordering for patch +1 mvninstall 1m 52s the patch passed +1 compile 8m 58s the patch passed +1 javac 8m 58s the patch passed +1 checkstyle 1m 46s root: The patch generated 0 new + 173 unchanged - 2 fixed = 173 total (was 175) +1 mvnsite 2m 16s the patch passed +1 mvneclipse 0m 31s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 3m 54s the patch passed +1 javadoc 1m 38s the patch passed +1 unit 8m 39s hadoop-common in the patch passed. +1 unit 76m 1s hadoop-hdfs in the patch passed. +1 asflicense 0m 29s The patch does not generate ASF License warnings. 134m 51s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HDFS-10683 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12830110/HDFS-10683.002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 918bd5603957 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 00160f7 Default Java 1.8.0_101 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17027/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17027/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.

Wei-Chiu Chuang added a comment - 06/Oct/16 00:07 +1

Wei-Chiu Chuang added a comment - 06/Oct/16 00:52 Committed to trunk and branch-2. Thanks John Zhuge !

Hudson added a comment - 06/Oct/16 01:00 SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10550 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10550/ ) HDFS-10683 . Make class Token$PrivateToken private. Contributed by John (weichiu: rev c5ca2169151a5eec57152775789b6f53664e102c) (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/Credentials.java (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestUserGroupInformation.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/HAUtil.java (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/Token.java

John Zhuge added a comment - 06/Oct/16 02:28 Thanks Wei-Chiu Chuang for the review and commit!

Andrew Wang added a comment - 05/Apr/17 23:41 Backported this to 2.8.1, looks innocuous, to make another backport clean.