[DRILL-7313] Use Hive schema for MaprDB native reader when field was empty and support all text mode

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.16.0
• Fix Version/s: 1.17.0
• Labels: doc-impacting ready-to-commit
• Assignee: Vova Vysotskyi
• Reporter: Vova Vysotskyi
• Reviewer: Arina Ielchiieva
• Created: 03/Jul/19 07:46
• Updated: 05/Jul/19 11:44
• Resolved: 05/Jul/19 11:44

Description

Currently, when an external Hive MaprDB table is queried using hive plugin with enabled store.hive.maprdb_json.optimize_scan_with_native_reader, some queries may fail due to soft schema change, though Hive knows actual data types.

For example, when we have a table with several fields, and one of them has only several non-null values, queries with grouping by such field will fail due to schema change.

The goal of this Jira is to allow using types from Hive when a non-existing field is created, so it will allow avoiding such issues.

In the scope of this Jira is also added new session option store.hive.maprdb_json.all_text_mode to read all data from the maprDB JSON tables as VARCHAR when hive plugin is used and Drill native MaprDB JSON reader usage is enabled.

Issue Links

• links to: GitHub Pull Request #1819