[DRILL-6824] Drill Query on MapRDB JSON table failing on schema SchemaChangeException, the only distinct Values are NULL and Text

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.14.0
• Fix Version/s: 1.15.0
• Component/s: Storage - MapRDB
• Labels: ready-to-commit
• Assignee: Gautam Parai
• Reporter: Gautam Parai
• Reviewer: Aman Sinha
• Created: 31/Oct/18 19:14
• Updated: 02/Nov/18 21:23
• Resolved: 02/Nov/18 21:21

Description

Drill Query on MapR DB JSON Table or it View fails with a SchemaChangeException. The only distinct values is NULL and some Text.

The issue is that when Drill sees a NULL or does not see any values for a column in the first batch it assumes the type as NULLABLE INT. Subsequently, when the data shows up and it is different from NULLABLE INT there is a schema change. Operators e.g. aggregators etc. cannot handle such a Schema Change and throw a SchemaChangeException.

 

One of the short-term solution implemented in this fix: Add a CAST expression which will cast null values to the target type. Hence, we would never see a SchemaChange due to NULLs. However, the MapRDB Reader code was written differently than other reader and was hitting a SchemaChangeException. The code was changed to make it similar to other scans i.e. emit a new batch whenever a schema change is encountered.

Issue Links

• links to: GitHub Pull Request #1518