Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-7086

Enhance row-set scan framework to use external schema

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.15.0
    • Fix Version/s: 1.16.0
    • Component/s: None
    • Labels:

      Description

      Modify the row-set scan framework to work with an external (partial) schema; inserting "type conversion shims" to convert as needed. The reader provides an "input schema" the data types the reader is prepared to handle. An optional "output schema" describes the types of the value vectors to create. The type conversion "shims" give the reader the "setFoo" method it wants to use, while converting the data to the type needed for the vector. For example, the CSV reader might read only text fields, while the shim converts a column to an INT.

      This is just the framework layer, DRILL-7011 will combine this mechanism with the plan-side features to enable use of the feature in the new row-set based CSV reader.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Paul.Rogers Paul Rogers
                Reporter:
                Paul.Rogers Paul Rogers
                Reviewer:
                Arina Ielchiieva
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: