[BIGTOP-3961] Fix version mismatch of hbase-client used by hadoop-yarn-server-timelineservice-hbase

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.2.1, 3.3.0
• Assignee: Jialiang Cai
• Reporter: Jialiang Cai
• Created: 21/Jul/23 09:46
• Updated: 02/Aug/23 00:39
• Resolved: 23/Jul/23 05:50

Description

After deploying yarn TimelineService v2, it fails to start because Hadoop 3, by default, includes HBase 1 client in the packaging of hadoop-yarn-server-timelineservice-hbase-client-3.3.4.jar. When the TimelineServiceReader starts and attempts to create an HBase table, it encounters compatibility issues between HBase 1 and 2 APIs, resulting in the failure to find the required method and thus unable to create the table.

Error message:
Caused by: java.lang.NoSuchMethodError:org.apache.hadoop.hbase.client.Admin.createTable

 
 
The Hadoop documentation also acknowledges this issue, and the relevant excerpt is as follows:

https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/TimelineServiceV2.html

The first part is to set up or pick an Apache HBase cluster to use as the storage cluster. The supported versions of Apache HBase are 1.2.6 (default) and 2.0.0-beta1. The 1.0.x versions do not work with Timeline Service v.2. By default, Hadoop releases are built with HBase 1.2.6. To use HBase 2.0.0-beta1, build from source with option -Dhbase.profile=2.0

Issue Links

• links to: GitHub pull request #1138