[DRILL-6454] Native MapR DB plugin support for Hive MapR-DB json table

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.13.0
• Fix Version/s: 1.14.0
• Component/s: Storage - Hive, Storage - MapRDB
• Labels: doc-complete ready-to-commit
• Assignee: Vitalii Diravka
• Reporter: Vitalii Diravka
• Reviewer: Gautam Parai
• Created: 30/May/18 11:37
• Updated: 01/Dec/18 13:45
• Resolved: 22/Jun/18 21:25

Description

Hive can create and query MapR-DB tables via maprdb-json-handler:
https://maprdocs.mapr.com/home/Hive/ConnectingToMapR-DB.html

The aim of this Jira to implement Drill native reader for Hive MapR-DB tables (similar to parquet).

Design proposal is:

• to use JsonTableGroupScan instead of HiveScan;
• to add storage planning rule to convert HiveScan to MapRDBGroupScan;
• to add system/session option to enable using of this native reader;
• native reader can be used only for Drill build with mapr profile (there is no reason to leverage it for default profile);

 

For documentation:
two new options were added:
store.hive.parquet.optimize_scan_with_native_reader: false,
store.hive.maprdb_json.optimize_scan_with_native_reader: false,

store.hive.parquet.optimize_scan_with_native_reader is new option used instead of store.hive.optimize_scan_with_native_readers. The latter is deprecated and will be removed in 1.15.
(https://issues.apache.org/jira/browse/DRILL-6527).

 

Issue Links

• is related to: DRILL-6614 Allow usage of MapRDBFormatPlugin for HiveStoragePlugin
• relates to: DRILL-6145 Enable usage of Hive MapR-DB JSON handler
• links to: GitHub Pull Request #1314