Details
Description
Looking at the ServerName's compareTo(),
It converts the return int value by converting long to int like (int)(longValue), which could be incorrect when it overflows, need to replace it with Long.compareTo(a,b).
mbertozzi found some others as well, such as
https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java#L990
Attachments
Attachments
- HBASE-16272-v001.patch
- 6 kB
- Hua Xiang
- HBASE-16272-0.98.patch
- 8 kB
- Andrew Kyle Purtell
- HBASE-16272-0.98-addendum.patch
- 0.9 kB
- Andrew Kyle Purtell
Issue Links
- is depended upon by
-
HBASE-16275 Change ServerManager#onlineServers from ConcurrentHashMap to ConcurrentSkipListMap
- Resolved
Activity
+1 but let's see if there is any unit test relying on that behavior
-1 overall |
Vote | Subsystem | Runtime | Comment |
---|---|---|---|
+1 | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
+1 | @author | 0m 0s | The patch does not contain any @author tags. |
-1 | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
0 | mvndep | 1m 33s | Maven dependency ordering for branch |
+1 | mvninstall | 3m 16s | master passed |
+1 | compile | 1m 10s | master passed with JDK v1.8.0 |
+1 | compile | 0m 55s | master passed with JDK v1.7.0_80 |
+1 | checkstyle | 0m 48s | master passed |
+1 | mvneclipse | 0m 28s | master passed |
+1 | findbugs | 2m 43s | master passed |
+1 | javadoc | 0m 54s | master passed with JDK v1.8.0 |
+1 | javadoc | 0m 50s | master passed with JDK v1.7.0_80 |
0 | mvndep | 0m 10s | Maven dependency ordering for patch |
+1 | mvninstall | 1m 9s | the patch passed |
+1 | compile | 1m 18s | the patch passed with JDK v1.8.0 |
+1 | javac | 1m 18s | the patch passed |
+1 | compile | 0m 50s | the patch passed with JDK v1.7.0_80 |
+1 | javac | 0m 50s | the patch passed |
+1 | checkstyle | 0m 45s | the patch passed |
+1 | mvneclipse | 0m 28s | the patch passed |
+1 | whitespace | 0m 0s | Patch has no whitespace issues. |
+1 | hadoopcheck | 27m 18s | Patch does not cause any errors with Hadoop 2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. |
+1 | findbugs | 3m 9s | the patch passed |
+1 | javadoc | 0m 52s | the patch passed with JDK v1.8.0 |
+1 | javadoc | 0m 49s | the patch passed with JDK v1.7.0_80 |
+1 | unit | 1m 45s | hbase-common in the patch passed. |
-1 | unit | 135m 53s | hbase-server in the patch failed. |
+1 | asflicense | 0m 37s | Patch does not generate ASF License warnings. |
188m 31s |
Reason | Tests |
---|---|
Failed junit tests | hadoop.hbase.regionserver.TestHRegion |
hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures |
Subsystem | Report/Notes |
---|---|
JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12819693/HBASE-16272-v001.patch |
JIRA Issue | |
Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile |
uname | Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
Build tool | maven |
Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh |
git revision | master / 519f87f |
Default Java | 1.7.0_80 |
Multi-JDK versions | /home/jenkins/tools/java/jdk1.8.0:1.8.0 /home/jenkins/jenkins-slave/tools/hudson.model.JDK/JDK_1.7_latest_:1.7.0_80 |
findbugs | v3.0.0 |
unit | https://builds.apache.org/job/PreCommit-HBASE-Build/2736/artifact/patchprocess/patch-unit-hbase-server.txt |
unit test logs | https://builds.apache.org/job/PreCommit-HBASE-Build/2736/artifact/patchprocess/patch-unit-hbase-server.txt |
Test Results | https://builds.apache.org/job/PreCommit-HBASE-Build/2736/testReport/ |
modules | C: hbase-common hbase-server U: . |
Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/2736/console |
Powered by | Apache Yetus 0.2.1 http://yetus.apache.org |
This message was automatically generated.
Manually run mvn clean test -Dtest=TestHRegion,TestMasterFailoverWithProcedure locally passed for me.
yeah, did a run too and a quick code check. seems unrelated.
I'm thinking to port this stuff in 1.x too, let me run tests and commit
FAILURE: Integrated in HBase-1.4 #303 (See https://builds.apache.org/job/HBase-1.4/303/)
HBASE-16272 Overflow in ServerName's compareTo method (Huaxiang Sun) (matteo.bertozzi: rev 24a3d6952ccffdcb340f520926852af788d59ef1)
- hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
- hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/SimpleRpcScheduler.java
- hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java
- hbase-client/src/main/java/org/apache/hadoop/hbase/ServerName.java
- hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
SUCCESS: Integrated in HBase-1.3-IT #763 (See https://builds.apache.org/job/HBase-1.3-IT/763/)
HBASE-16272 Overflow in ServerName's compareTo method (Huaxiang Sun) (matteo.bertozzi: rev ad91bfd0a88f761dcc07924403ddbde491868f4c)
- hbase-client/src/main/java/org/apache/hadoop/hbase/ServerName.java
- hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
- hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/SimpleRpcScheduler.java
- hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java
- hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
FAILURE: Integrated in HBase-1.3 #792 (See https://builds.apache.org/job/HBase-1.3/792/)
HBASE-16272 Overflow in ServerName's compareTo method (Huaxiang Sun) (matteo.bertozzi: rev ad91bfd0a88f761dcc07924403ddbde491868f4c)
- hbase-client/src/main/java/org/apache/hadoop/hbase/ServerName.java
- hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java
- hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/SimpleRpcScheduler.java
- hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
- hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
FAILURE: Integrated in HBase-Trunk_matrix #1278 (See https://builds.apache.org/job/HBase-Trunk_matrix/1278/)
HBASE-16272 Overflow in ServerName's compareTo method (Huaxiang Sun) (matteo.bertozzi: rev cc766df28b80827f8951c2c1364443a8c5a39b35)
- hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/SimpleRpcScheduler.java
- hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java
- hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
- hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
- hbase-common/src/main/java/org/apache/hadoop/hbase/ServerName.java
FAILURE: Integrated in HBase-1.2 #679 (See https://builds.apache.org/job/HBase-1.2/679/)
HBASE-16272 Overflow in ServerName's compareTo method (Huaxiang Sun) (matteo.bertozzi: rev 0ed7d5f80353d3c736838ead73ed8c43fcd6cac4)
- hbase-client/src/main/java/org/apache/hadoop/hbase/ServerName.java
- hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
- hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
- hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java
- hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/SimpleRpcScheduler.java
SUCCESS: Integrated in HBase-1.2-IT #560 (See https://builds.apache.org/job/HBase-1.2-IT/560/)
HBASE-16272 Overflow in ServerName's compareTo method (Huaxiang Sun) (matteo.bertozzi: rev 0ed7d5f80353d3c736838ead73ed8c43fcd6cac4)
- hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/SimpleRpcScheduler.java
- hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
- hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java
- hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
- hbase-client/src/main/java/org/apache/hadoop/hbase/ServerName.java
FAILURE: Integrated in HBase-1.1-JDK8 #1838 (See https://builds.apache.org/job/HBase-1.1-JDK8/1838/)
HBASE-16272 Overflow in ServerName's compareTo method (Huaxiang Sun) (matteo.bertozzi: rev 4f6647b9f0d089341d4aa9e693e421b2c89ac514)
- hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/SimpleRpcScheduler.java
- hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java
- hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
- hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
- hbase-client/src/main/java/org/apache/hadoop/hbase/ServerName.java
FAILURE: Integrated in HBase-1.1-JDK7 #1752 (See https://builds.apache.org/job/HBase-1.1-JDK7/1752/)
HBASE-16272 Overflow in ServerName's compareTo method (Huaxiang Sun) (matteo.bertozzi: rev 4f6647b9f0d089341d4aa9e693e421b2c89ac514)
- hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
- hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/SimpleRpcScheduler.java
- hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java
- hbase-client/src/main/java/org/apache/hadoop/hbase/ServerName.java
- hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
FAILURE: Integrated in HBase-0.98-matrix #376 (See https://builds.apache.org/job/HBase-0.98-matrix/376/)
HBASE-16272 Overflow in ServerName's compareTo method (Huaxiang Sun) (matteo.bertozzi: rev 6e54ac352ae2a7aae9cc804132126aee0c44b971)
- hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
- hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java
- hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
- hbase-client/src/main/java/org/apache/hadoop/hbase/ServerName.java
FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #1248 (See https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/1248/)
HBASE-16272 Overflow in ServerName's compareTo method (Huaxiang Sun) (matteo.bertozzi: rev 6e54ac352ae2a7aae9cc804132126aee0c44b971)
- hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
- hbase-client/src/main/java/org/apache/hadoop/hbase/ServerName.java
- hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java
- hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
The 0.98 commit needs an addendum:
[ERROR] hbase-client/src/main/java/org/apache/hadoop/hbase/ServerName.java:[293,15] cannot find symbol [ERROR] symbol : method compare(long,long) [ERROR] location: class java.lang.Long
and 9 others due to lack of Long.compare() in JDK/JRE 6
Working on it. Will commit addendum shortly.
Sorry, fixing the problem basically reverts the patch. Going to revert from 0.98.
Reverted with 6dfdc0d898b92371b8694ff4b32897ade8f53bcb. You are welcome to try again without use of Long#compare
Let me make a new patch for 0.98 that brings in Long#compare as a new utility method. Coming shortly.
0.98 patch. Committing shortly. New LongUtils class in hbase-common has an equivalent implementation of Long#compare to what's in JDK 7 but is not a cut-and-paste job
Addendum for 0.98 adding audience annotation to LongUtils to fix TestInterfaceAudienceAnnotations
FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #1251 (See https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/1251/)
Revert "HBASE-16272 Overflow in ServerName's compareTo method (Huaxiang (apurtell: rev 6dfdc0d898b92371b8694ff4b32897ade8f53bcb)
- hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
- hbase-client/src/main/java/org/apache/hadoop/hbase/ServerName.java
- hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java
- hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
HBASE-16272Overflow in ServerName's compareTo method (Huaxiang Sun and (apurtell: rev e8a146a7845d3a6f86bba8f6dbee7c23054d8107) - hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
- hbase-common/src/main/java/org/apache/hadoop/hbase/util/LongUtils.java
- hbase-client/src/main/java/org/apache/hadoop/hbase/ServerName.java
- hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java
- hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
AmendHBASE-16272Overflow in ServerName's compareTo method (apurtell: rev 92816613069df045495ae8fc5124224f98b5b390) - hbase-common/src/main/java/org/apache/hadoop/hbase/util/LongUtils.java
FAILURE: Integrated in HBase-0.98-matrix #379 (See https://builds.apache.org/job/HBase-0.98-matrix/379/)
Revert "HBASE-16272 Overflow in ServerName's compareTo method (Huaxiang (apurtell: rev 6dfdc0d898b92371b8694ff4b32897ade8f53bcb)
- hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java
- hbase-client/src/main/java/org/apache/hadoop/hbase/ServerName.java
- hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
- hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
HBASE-16272Overflow in ServerName's compareTo method (Huaxiang Sun and (apurtell: rev e8a146a7845d3a6f86bba8f6dbee7c23054d8107) - hbase-client/src/main/java/org/apache/hadoop/hbase/ServerName.java
- hbase-common/src/main/java/org/apache/hadoop/hbase/util/LongUtils.java
- hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java
- hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
- hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
AmendHBASE-16272Overflow in ServerName's compareTo method (apurtell: rev 92816613069df045495ae8fc5124224f98b5b390) - hbase-common/src/main/java/org/apache/hadoop/hbase/util/LongUtils.java
Fixed some other places and did some cleanups.