[HIVE-25101] Remove HBase libraries from Hive distribution

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 4.0.0
• Fix Version/s: 4.0.0-alpha-1
• Component/s: HBase Handler, Hive
• Labels: pull-request-available
• Assignee: Istvan Toth
• Reporter: Istvan Toth
• Created: 11/May/21 10:14
• Updated: 27/Nov/23 18:26
• Resolved: 23/Jun/21 21:13

Description

Hive currently packages HBase libraries into its lib directory.
It also adds the HBase libraries separately to its classpath in the hive startup script.

Having both mechanisms is redundant, and it also causes errors, as the standard HBase libraries packaged into Hive are unshaded, while the libraries added by hbase mapredcp
are shaded, and the two are NOT compatible when custom coprocessors are used, and in some cases the classpaths during local execution and for MR/TEZ jobs are mutually incompatible.

I propose removing all HBase libraries from the distribution, and pulling them via the hbase mapredcp mechanism.

This also solves the old problem of including ancient HBase alpha versions Hive.

Issue Links

• causes: HIVE-26511 Fix NoClassDefFoundError in HMS for HBaseConfiguration
• relates to: PHOENIX-6939 Change phoenix-hive connector shading to work with hbase-shaded-mapreduce
• links to: GitHub Pull Request #2259