[HBASE-26546] hbase-shaded-client missing required thirdparty classes under hadoop 3.3.1

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.4.3, 2.4.4, 2.4.5, 2.4.6, 2.4.7, 2.4.8, 2.4.9
• Fix Version/s: 2.5.0, 3.0.0-alpha-3, 2.4.10
• Component/s: Client, hadoop3, shading
• Assignee: Bryan Beaudreault
• Reporter: Bryan Beaudreault
• Created: 07/Dec/21 17:44
• Updated: 01/Sep/22 10:57
• Resolved: 22/Feb/22 07:15

Description

In HBASE-25792, the shaded thirdparty libraries from hadoop were removed from the hbase-shaded-client fat jar to satisfy invariant checks. Unfortunately this causes users of hbase-shaded-client to fail, because required classes are not available at runtime.

The specific failure I'm seeing is when trying to call new Configuration(), which results in:

 
 

Caused by: java.lang.NoClassDefFoundError: org/apache/hadoop/thirdparty/com/google/common/base/Preconditions   
  at org.apache.hadoop.conf.Configuration$DeprecationDelta.<init>(Configuration.java:430)   
  at org.apache.hadoop.conf.Configuration$DeprecationDelta.<init>(Configuration.java:443)   
  at org.apache.hadoop.conf.Configuration.<clinit>(Configuration.java:525)

 
 
If you take a look at the hbase-shaded-client fat jar, it contains the org.apache.hadoop.conf.Configuration class as you'd expect. If you decompile that class (or look at the 3.3.1 source), you'll see that there is an import for org.apache.hadoop.thirdparty.com.google.common.base.Preconditions but the fat jar does not provide it.
 
One way for clients to get around this is to add an explicit dependency on hadoop-shaded-guava, but this is problematic for a few reasons:
 

• it's best practice to use maven-dependency-plugin to disallow declared, unused dependencies (which this would be)
• it requires users to continually keep the version of hadoop-shaded-guava up-to-date over time.
• it only covers guava, but there is also protobuf and potentially other shaded libraries in the future.
   
  I think we should remove the exclusion of org/apache/hadoop/thirdparty/*/ from the shading config and instead add that pattern to the allowlist so that hbase-shaded-client is all clients need to get started with hbase.

Issue Links

• is broken by: HBASE-25792 Filter out o.a.hadoop.thirdparty building shaded jars
• links to: GitHub Pull Request #3926