[DRILL-7086] Enhance row-set scan framework to use external schema

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.15.0
• Fix Version/s: 1.16.0
• Labels: ready-to-commit
• Assignee: Paul Rogers
• Reporter: Paul Rogers
• Reviewer: Arina Ielchiieva
• Created: 10/Mar/19 00:44
• Updated: 19/Mar/19 18:30
• Resolved: 19/Mar/19 18:30

Description

Modify the row-set scan framework to work with an external (partial) schema; inserting "type conversion shims" to convert as needed. The reader provides an "input schema" the data types the reader is prepared to handle. An optional "output schema" describes the types of the value vectors to create. The type conversion "shims" give the reader the "setFoo" method it wants to use, while converting the data to the type needed for the vector. For example, the CSV reader might read only text fields, while the shim converts a column to an INT.

This is just the framework layer, DRILL-7011 will combine this mechanism with the plan-side features to enable use of the feature in the new row-set based CSV reader.

Issue Links

• Dependent: DRILL-6835 Schema Provision using File / Table Function
• links to: GitHub Pull Request #1690