[HUDI-515] Resolve API conflict for Hive 2 and Hive 3

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Labels: pull-request-available
• Assignee: Unassigned
• Reporter: Wenning Ding
• Created: 09/Jan/20 19:49
• Updated: 13/Jun/20 11:28
• Resolved: 13/Jun/20 11:27

Description

Currently I am working on supporting HIVE 3. There is an API issue.

In HoodieCombineHiveInputFormat.java, it calls a Hive method: HiveFileFormatUtils.getPartitionDescFromPathRecursively(). But this method is removed in Hive 3 and replaced by HiveFileFormatUtils.getFromPathRecursively().

Ideally, Hudi should support both Hive 2 & Hive 3 so that both

mvn clean install

and

mvn clean install -Dhive.version=3.x

could work.

One solution is to directly copy source code from HiveFileFormatUtils.getFromPathRecursively() and put that code inside HoodieCombineHiveInputFormat.java.

The other way is using java reflection to decide which method to use. (preferred)

Issue Links

• links to: GitHub Pull Request #1638
• links to: GitHub Pull Request #1638