[PHOENIX-6333] Hbase versions older than 2.4.0 are incompatible with Hadoop 3.1.4

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 5.1.0
• Fix Version/s: 5.1.0
• Component/s: core

Environment

Phoenix 5 is now defaults to building with the following Hadoop versions:
* hbase.profile=2.1 : Hadoop 3.0.3

• hbase.profile=2.2 : Hadoop3.1.3
• hbase.profile=2.3 : Hadoop3.1.3
• hbase.profile=2.4 : Hadoop3.1.4

• Assignee: Istvan Toth
• Reporter: Istvan Toth
• Created: 21/Jan/21 06:01
• Updated: 05/Apr/22 06:51
• Resolved: 21/Jan/21 13:23

Description

In PHOENIX-6326 I've upgraded Hadoop to 3.1.4 to solve a test failure caused by older Jetty versions.
It turns out that only HBase 2.4 is compatible with the new Jetty in 3.1.4, (it uses a shaded jetty), but earlier versions are not.
Unfortunately, this happened right after we swicthed Yetus to to use Hbase 2.4, so the procommit checks didn't catch this.

Technically, neither the Phoenix client or server runtime artifacts are affected by this, as the server doesn't include HBase, and the client doesn' use jetty.

Considering that any pre-2.4.0 HBase version is incompatible with Hadoop 3.1.4, I think that the best option to use Hadoop 3.1.3 in those profiles.
As any pre-2.4.0 HBase is incompatible (at least on ther server side) with the problematic Java releases,
we are not really losing anything by not supporting building releases with this profiles in Phoenix.

Furthermore, it turns out that even minor hadoop releases are not ABI compatible.
Rebuilding HBase 2.1 with hadoop.profile=3.0, which uses Hadoop 3.0.3, and then building Phoenix with Hadoop 3.1.3 will result in the usual Class<->Interface errors.

While building Hbase 2.1 with hadoop.profile=3.0 hadoop.version=3.1.3 seems to resolve this, I think that sticiking with same Hadoop minor version that HBase defaults to in each profile will be simpler, and more likely to be compatible with user environments.

Issue Links

• is caused by: PHOENIX-6326 Phoenix doesn't work with Java version 11.0.9.1 , due to Jetty problem
• is duplicated by: PHOENIX-6332 After Hadoop upgrade, we need to rebuild all Hbase versions
• relates to: HBASE-23834 HBase fails to run on Hadoop 3.3.0/3.2.2/3.1.4 due to jetty version mismatch
• relates to: PHOENIX-6682 Jenkins tests are failing for Java 11.0.14.1
• links to: GitHub Pull Request #1103