Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-7398

dfs avro file support to treat non existent fields as null

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • None
    • Storage - Avro
    • None

    Description

      It would be nice to have an option to return null for non existing fields.

      Currently we have a use case of evolving avro file schemas where things get added.

      It would be nice to be able to query old and new data without having to rewrite the old data with a new schema with null/optional/default values.

      A short term fix would be to provide an option to return null with dfs for avro fields that aren't found in a file.

      A nicer fix would be to support a resolving schema that can be given to dfs to read the avro files to present a uniform interface (so aliases, defaults, etc. could be utilized without rewriting each avro file.)

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              dschmitt006 Dan Schmitt
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: