[HBASE-25473] [create-release] checkcompatibility.py failing with "KeyError: 'binary'"

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.0.0-alpha-1
• Component/s: scripts
• Hadoop Flags: Reviewed
• Release Note: Adds temporary exclude of hbase-shaded-testing-util from checkcompatibility so create-release can work again. Undo exclude when sub-issue is fixed.
• Assignee: Michael Stack
• Reporter: Michael Stack
• Created: 07/Jan/21 06:46
• Updated: 03/Jul/21 13:13
• Resolved: 08/Jan/21 22:48

Description

Below is excerpt from log of RC0 2.3.4 build this afternoon when we try to do the compatibility check (inside docker create-release build). It had failed for me previously on another run of different versions. The failure is unfortunately cryptic.

2021-01-07T01:08:34Z generate_api_report start
INFO:root:Source revision: rel/2.3.3
INFO:root:Destination revision: 2.3.4RC0
INFO:root:Filtering classes using 1 annotation(s):
INFO:root:      org.apache.yetus.audience.InterfaceAudience.Public
INFO:root:Downloading Java ACC...
INFO:root:Java ACC version: 2.4

INFO:root:Removing scratch dir /home/vagrant/hbase-rm/output/hbase/target/compat-check
INFO:root:Creating empty scratch dir /home/vagrant/hbase-rm/output/hbase/target/compat-check
INFO:root:Checking out 5f7fe66efbdfc1c0b3262c0677f17e4055ba94cd in /home/vagrant/hbase-rm/output/hbase/target/compat-check/src
INFO:root:Checking out ec79612b9ac6afd73fa8fc7f7f5c8d3bac277679 in /home/vagrant/hbase-rm/output/hbase/target/compat-check/dst
INFO:root:Building in /home/vagrant/hbase-rm/output/hbase/target/compat-check/src
INFO:root:Building in /home/vagrant/hbase-rm/output/hbase/target/compat-check/dst
INFO:root:Will check compatibility between original jars:
        /home/vagrant/hbase-rm/output/hbase/target/compat-check/src/hbase-shaded/hbase-shaded-client-byo-hadoop/target/original-hbase-shaded-client-byo-hadoop-2.3.3.jar
        /home/vagrant/hbase-rm/output/hbase/target/compat-check/src/hbase-external-blockcache/target/hbase-external-blockcache-2.3.3.jar
        /home/vagrant/hbase-rm/output/hbase/target/compat-check/src/hbase-shaded/hbase-shaded-client/target/original-hbase-shaded-client-2.3.3.jar
        /home/vagrant/hbase-rm/output/hbase/target/compat-check/src/hbase-zookeeper/target/hbase-zookeeper-2.3.3.jar
        /home/vagrant/hbase-rm/output/hbase/target/compat-check/src/hbase-common/target/hbase-common-2.3.3.jar

....
        /home/vagrant/hbase-rm/output/hbase/target/compat-check/dst/hbase-shaded/hbase-shaded-testing-util-tester/target/hbase-shaded-testing-util-tester-2.3.4.jar
        /home/vagrant/hbase-rm/output/hbase/target/compat-check/dst/hbase-archetypes/hbase-shaded-client-project/target/hbase-shaded-client-project-2.3.4.jar
        /home/vagrant/hbase-rm/output/hbase/target/compat-check/dst/hbase-protocol/target/hbase-protocol-2.3.4.jar
INFO:root:Annotations are: ['org.apache.yetus.audience.InterfaceAudience.Public']
INFO:root:Annotations path: /home/vagrant/hbase-rm/output/hbase/target/compat-check/annotations.txt
INFO:root:Results: {}
Traceback (most recent call last):
  File "./hbase/dev-support/checkcompatibility.py", line 530, in <module>
    main()
  File "./hbase/dev-support/checkcompatibility.py", line 526, in main
    args.compare_warnings))
  File "./hbase/dev-support/checkcompatibility.py", line 232, in compare_results
    if tool_results[check][issue_type] > known_count]

Issue Links

• breaks: HBASE-26061 [create-release] The indent of checkcompatibility.py is broken after HBASE-25473
• links to: GitHub Pull Request #2862

Sub-Tasks

• Fix assembly of the hbase-shaded-testing-util*.jar; can't unzip on case-insensitive fs: Open