[DRILL-1257] Handle schema mutations within a batch

• Type: New Feature
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Fix Version/s: Future
• Assignee: Unassigned
• Reporter: Jason Altekruse
• Created: 05/Aug/14 22:49
• Updated: 19/Jun/17 00:05

Description

Drill is intended to provide flexibility for a wide variety of querying use cases, including those against file formats with changing schema such as JSON. Currently Drill does not have an efficient means of representing these cases, as a constant schema is tied to a record batch, which is best used to represent a few hundred or thousand records. We have have also not designed all of the operators and functions to gracefully handle changes in schema throughout the course of a query.

Issue Links

• blocks: DRILL-1589 count(*) on json fails with exception
• is duplicated by: DRILL-1602 Enhance Drill to support Heterogeneous fields
• is related to: DRILL-4824 Null maps / lists and non-provided state support for JSON fields. Numeric types promotion.

Sub-Tasks

• Embedded Type Vector: Open
• Support in functions for heterogeneous and embedded types: Open
• Promotion of vectors from standard to embedded vectors: Open
• Extend complex reader/writer to handle Embedded Types: Open
• Fix operators to work with embedded types: Open