Description
This project is to do what apurtell did in the issue "HBASE-18431 Mitigate compatibility concerns between branch-1.3 and branch-1.4" only do it between branch-1 and branch-2.
Attachments
Attachments
- report.1.2_2.0.html.gz
- 321 kB
- Michael Stack
Activity
Here is summary:
API compatibility report for the hbase library between 2f9b9e17d0522e36063bf52ecc58576243d20b3f and fad968d99f4693278edcaefff92738e67ad8e8a0 versions
Test Info
Library Name hbase
Version #1 2f9b9e17d0522e36063bf52ecc58576243d20b3f
Version #2 fad968d99f4693278edcaefff92738e67ad8e8a0
Subject Binary Compatibility
Test Results
Total JARs 13
Total Methods / Classes 142606 / 6306
Compatibility 66.9%
Problem Summary
Severity Count
Added Methods - 78007
Removed Methods High 15834
Problems with
Data Types High 1131
Medium 244
Low 181
Problems with
Methods High 74
Medium 0
Low 79
Other Changes
in Data Types - 1004
Other Changes
in Methods - 160
Excluding protocol, shaded, and archetype and new jars like backup and rsgroup makes the output more manageable:
./dev-support/checkcompatibility.py -e .*hbase-rsgroup.* -e .*hbase-backup.* -e .*hbase-protoco.* -e .*hbase-shaded.* -e .*origina-hbase.* -e .*hbase-archetypes.* -e .*hbase-examples.* 2f9b9e17d0522e36063bf52ecc58576243d20b3f fad968d99f4693278edcaefff92738e67ad8e8a0 Test Info Library Name hbase Version #1 2f9b9e17d0522e36063bf52ecc58576243d20b3f Version #2 fad968d99f4693278edcaefff92738e67ad8e8a0 Subject Binary Compatibility Test Results Total JARs 11 Total Methods / Classes 40212 / 3303 Compatibility 70.5% Problem Summary Severity Count Added Methods - 12218 Removed Methods High 4140 Problems with Data Types High 688 Medium 243 Low 178 Problems with Methods High 74 Medium 0 Low 79 Other Changes in Data Types - 878 Other Changes in Methods - 160 Added Methods 12218
Admin branch-2 has 80 methods more than was in branch-1 Admin.
There are too many adds and removes to review.
Going over the methods flagged problematic, a good portion can be ignored since the class was InterfaceAudience Private in hbase-1.0; e.g. Call or ConnectionUtils.
We have a few methods that differ in signature in Admin. If we say hbase1 can't admin an hbase2 cluster, then we can let below slide:
hbase-client-1.2.6.jar, Admin.class package org.apache.hadoop.hbase.client [−] Admin.createTableAsync ( HTableDescriptor p1, byte[ ][ ] p2 ) [abstract] : void 1 org/apache/hadoop/hbase/client/Admin.createTableAsync:(Lorg/apache/hadoop/hbase/HTableDescriptor;[[B)V Change Effect 1 Return value type has been changed from void to java.util.concurrent.Future<java.lang.Void>. This method has been removed because the return type is part of the method signature. [−] Admin.disableTableAsync ( TableName p1 ) [abstract] : void 1 org/apache/hadoop/hbase/client/Admin.disableTableAsync:(Lorg/apache/hadoop/hbase/TableName;)V Change Effect 1 Return value type has been changed from void to java.util.concurrent.Future<java.lang.Void>. This method has been removed because the return type is part of the method signature. [−] Admin.enableTableAsync ( TableName p1 ) [abstract] : void 1 org/apache/hadoop/hbase/client/Admin.enableTableAsync:(Lorg/apache/hadoop/hbase/TableName;)V Change Effect 1 Return value type has been changed from void to java.util.concurrent.Future<java.lang.Void>. This method has been removed because the return type is part of the method signature.
Filing subtasks for violations found.
Got as far as: hbase-common-1.2.6.jar, CellComparator.class Will be back.
I might be going about this wrong. Rather, I should just review the differences and then write unit tests that have hbase1 client going against an hbase2 cluster exercising problematic areas to see if incompatibilities show up on other side of the wire.
Here is how I generated the report:
$ ./dev-support/checkcompatibility.py --annotation org.apache.hadoop.hbase.classification.InterfaceAudience.Public --annotation org.apache.hadoop.hbase.classification.InterfaceAudience.LimitedPrivate 6c98bff7b719efdb16f71606f3b7d8229445eb81 a0f980b573b5af134002f0cb9163384d694b4388
I'd edited ./dev-support/checkcompatibiilty.py to use JACC 2.3 instead of 2.1 because it had this fix... https://github.com/lvc/japi-compliance-checker/issues/40 (via busbey
Otherwise the reports are massive including privately annotated classes.
Of note, to be complete, I need to not only compare 1.0.0 to branch-2 but 1.1.0 to branch-2 and so on...
List of incompatibilties are in section #6 of the 2.0 doc https://docs.google.com/document/d/1WCsVlnHjJeKUcl7wHwqb4z9iEu_ktczrlKHK8N4SZzs/edit#heading=h.723jjn18p2jr Will tidy it up soon (don't look yet!)
JACC report of 1.2 to 2.0. I've gone through a few times. Have a glaze now and can't look at it anymore. I think I have most captured in writeup at end of this doc in compatibility chapter: https://docs.google.com/document/d/1WCsVlnHjJeKUcl7wHwqb4z9iEu_ktczrlKHK8N4SZzs/edit#heading=h.xjagihujxydb Need to do some cleanup to make it more palatable.
InterfaceAudience.Private is saving our butt all over the place.
HColumnDescriptor went from this in 1.0.0
@InterfaceAudience.Public @InterfaceStability.Evolving public class HColumnDescriptor implements WritableComparable<HColumnDescriptor> {
to this in 2.0
@InterfaceAudience.Public @Deprecated // remove it in 3.0 public class HColumnDescriptor implements ColumnFamilyDescriptor, Comparable<HColumnDescriptor> {
Figuring this ok.
Otherwise, filed subtasks here and updated the incompatibility section in the 'hbase2 book': https://docs.google.com/document/d/1WCsVlnHjJeKUcl7wHwqb4z9iEu_ktczrlKHK8N4SZzs/edit#heading=h.723jjn18p2jr
This is an on-going 2.0.0 project. Best list of incompatibles will only be known on day we make release. Meantime we will keep turning them up whether via JACC examination or via testing.
Reran the report
Changed the reporting tool to use 2.4 instead of 2.1 (released in January).
diff --git a/dev-support/checkcompatibility.py b/dev-support/checkcompatibility.py
index ea9c229344..9f0d797ff3 100755
— a/dev-support/checkcompatibility.py
+++ b/dev-support/checkcompatibility.py
@@ -156,7 +156,7 @@ def checkout_java_acc(force):
logging.info("Downloading Java ACC...")
- url = "https://github.com/lvc/japi-compliance-checker/archive/2.1.tar.gz"
+ url = "https://github.com/lvc/japi-compliance-checker/archive/2.4.tar.gz"
scratch_dir = get_scratch_dir()
path = os.path.join(scratch_dir, os.path.basename(url))
jacc = urllib2.urlopen(url)
@@ -166,7 +166,7 @@ def checkout_java_acc(force):
subprocess.check_call(["tar", "xzf", path],
cwd=scratch_dir)
- shutil.move(os.path.join(scratch_dir, "japi-compliance-checker-2.1"),
+ shutil.move(os.path.join(scratch_dir, "japi-compliance-checker-2.4"),
os.path.join(acc_dir))
$ ./dev-support/checkcompatibility.py --annotation org.apache.hadoop.hbase.classification.InterfaceAudience.Public rel/1.2.6 2.0.0RC0
Do we still have unfinished work for this issue? It is marked as blocker...
We do. Need to write up report on what diffs are. It didn't block 2.0 so can't block 2.1.
Resolving. Subtasks done. We published a report when we released 2.0.0 (though its probably hard to digest – could have done better).
Compared released 1.2.6 – a likely starting point – to branch-2 by doing below:
Report attached.