Uploaded image for project: 'Apache Avro'
  1. Apache Avro
  2. AVRO-2258

Resolution Canonical Form

Add voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: spec
    • Labels:
      None

      Description

      The spec defines the Parsing Canonical Form to identify schemas that are the same from the perspective of reading data written against the schemas. Schema Fingerprints can then be computed based on the Parsing Canonical Form.

      For our implementation of an Avro schema registry, we found schema fingerprints based on the Parsing Canonical Form to be insufficient because they exclude default and alias attributes which affect resolution between different schemas.

      For the registry, we wanted new versions to created when defaults or alias are changed.

      We extended to the Parsing Canonical Form to define a Resolution Canonical Form by modifying the order transformation and adding a transformation on aliases:

      • [ORDER] Order the appearance of fields in JSON objects as follows: name, type, fields, symbols, items, values, size, default, aliases
      • [ALIASES] Aliases for named types and fields are converted to their fullname, using applicable namespace, and sorted.

      We have an implementation for this that we have been using in Ruby:

      https://github.com/salsify/avro-resolution_canonical_form

       

      I'm opening this issue to gauge if there is interest and support for adding this to the Avro specification. If there is then I can open a pull request to modify the spec.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              tjwp Tim Perkins

              Dates

              • Created:
                Updated:

                Issue deployment