[DRILL-6035] Specify Drill's JSON behavior

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.13.0
• Assignee: Pritesh Maker
• Reporter: Paul Rogers
• Created: 15/Dec/17 21:10
• Updated: 22/Aug/18 18:11

Description

Drill supports JSON as its native data format. However, experience suggests that Drill may have limitations in the JSON that Drill supports. This ticket asks to clarify Drill's expected behavior on various kinds of JSON.

Topics to be addressed:

• Relational vs. non-relational structures
• JSON structures used in practice and how they map to Drill
• Support for varying data types
• Support for missing values, especially across files

These topics are complex, hence the request to provide a detailed specifications that clarifies what Drill does and does not support (or what is should and should not support.)

As noted below, the "net-net" conclusion for users is to use an ETL tool to convert JSON to Parquet, then allow Drill to query the Parquet.

Issue Links

• is related to: DRILL-6552 Drill Metadata management "Drill Metastore"