[DRILL-6791] Merge scan projection framework into master

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.15.0
• Fix Version/s: 1.15.0
• Labels: ready-to-commit
• Assignee: Paul Rogers
• Reporter: Paul Rogers
• Reviewer: Arina Ielchiieva
• Created: 12/Oct/18 04:07
• Updated: 10/Dec/18 12:04
• Resolved: 10/Dec/18 12:04

Description

Merge the next set of "result set loader" code into master via a PR. This one covers the "schema projection" mechanism which:

• Handles none (SELECT COUNT(*)), some (SELECT a, b, x) and all (SELECT *) projection.
• Handles null columns (for projection a column "x" that does not exist in the base table.)
• Handles constant columns as used for file metadata (AKA "implicit" columns).
• Handle schema persistence: the need to reuse the same vectors across different scanners
• Provides a framework for consuming externally-supplied metadata
• Since we don't yet have a way to provide "real" metadata, obtains metadata hints from previous batches and from the projection list (a.b implies that "a" is a map, c[0] implies that "c" is an array, etc.)
• Handles merging the set of data source columns and null columns to create the final output batch.

Issue Links

• links to: GitHub Pull Request #1501