[DRILL-6835] Schema Provision using File / Table Function

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.17.0
• Assignee: Arina Ielchiieva
• Reporter: Arina Ielchiieva
• Created: 08/Nov/18 12:07
• Updated: 10/Sep/19 12:53
• Resolved: 10/Sep/19 12:53

Description

Schema Provision using File / Table Function design document:

https://docs.google.com/document/d/1mp4egSbNs8jFYRbPVbm_l0Y5GjH3HnoqCmOpMTR_g4w/edit?usp=sharing

Phase 1 functional specification - https://docs.google.com/document/d/1ExVgx2FDqxAz5GTqyWt-_1-UqwRSTGLGEYuc8gsESG8/edit?usp=sharing

Issue Links

• Dependent: DRILL-6953 Merge row set-based JSON reader
• Dependent: DRILL-6950 Row set-based scan framework
• Dependent: DRILL-6951 Merge row set based mock data source
• Dependent: DRILL-7011 Allow hybrid model in the Row set-based scan framework
• Dependent: DRILL-7086 Enhance row-set scan framework to use external schema
• Dependent: DRILL-7143 Enforce column-level constraints when using a schema
• Dependent: DRILL-6952 Merge row set based "compliant" text reader
• requires: DRILL-6903 SchemaBuilder improvements
• requires: DRILL-6901 Move SchemaBuilder from test to main for use outside tests
• requires: DRILL-6870 Upgrade to ANTLR4
• links to: Presentation

Sub-Tasks

• Implement CREATE / DROP TABLE SCHEMA commands: Closed
• Adjust table function usage for all storage plugins and implement schema parameter: Resolved
• CREATE SCHEMA command / TupleSchema / ColumnMetadata improvements: Resolved
• Rename map to struct in schema definition: Resolved
• Expose Tuple Metadata to the physical operator: Resolved
• Implement command to describe schema for table: Resolved
• Wrap SchemaParsingException into UserException when creating schema: Resolved
• After renaming MAP to STRUCT typeString method still outputs MAP name: Resolved
• Implement ALTER SCHEMA ADD / REMOVE COLUMN / PROPERTY commands: Resolved