Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-1613

Drill does not honor nested schema changes

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.7.0
    • Component/s: None
    • Labels:
      None

      Description

      Drill's current definition of schema change is an incremental update to the top-level schema structure. This is inappropriate for complex data types where schema change may occur at any level in the hierarchy of vectors. Current design does not fit into this requirement neatly as only top-level containers are able to mutate the schema. We need a minor to major re-design to handle this situation.

      This issue exacerbates when querying partitions or batches of files that do not necessarily conform to the exact same schema. New fields from upcoming batches will be omitted in either case.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                mehant Mehant Baid
                Reporter:
                hgunes Hanifi Gunes
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: