Avro
  1. Avro
  2. AVRO-368

Reserve avro.* in object container files, and rename existing reserved words.

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 1.3.0
    • Fix Version/s: 1.3.0
    • Component/s: spec
    • Labels:
      None
    • Hadoop Flags:
      Incompatible change, Reviewed

      Description

      We propose to move "codec", "schema", and "sync" into "avro.codec", "avro.schema", and "avro.sync", and furthermore reserve all avro.* names in the object container file specification.

      (Discussion was started in the thread "Reserving more keywords in Avro Object Container Files?" on avro-dev.)

      1. AVRO-368.patch.txt
        8 kB
        Philip Zeyliger

        Activity

        Hide
        Doug Cutting added a comment -

        +1

        Note that, post AVRO-160, the sync marker is no longer in the metadata, but immediately follows the metadata, terminating the header and prefixing the first block, so that all blocks are now bracketed by sync markers. So "avro.schema" and "avro.codec" are currently the only reserved data file metadata keys.

        Show
        Doug Cutting added a comment - +1 Note that, post AVRO-160 , the sync marker is no longer in the metadata, but immediately follows the metadata, terminating the header and prefixing the first block, so that all blocks are now bracketed by sync markers. So "avro.schema" and "avro.codec" are currently the only reserved data file metadata keys.
        Hide
        Philip Zeyliger added a comment -

        I went ahead and implemented this change. The patch below includes: (1) changes to the spec, (2) changes to DataFileConstants, (3) wider use of DataFileConstants, (4) new tests for data file metadata setters and getters, (5) disallowing for setMeta(key) for all keys that use reserved words. The reserved words should be set via the API (this is true for codec and schema, and I suspect would be true for others).

        "ant clean test-java" passes in the java directory. The spec seems to build.

        Show
        Philip Zeyliger added a comment - I went ahead and implemented this change. The patch below includes: (1) changes to the spec, (2) changes to DataFileConstants, (3) wider use of DataFileConstants, (4) new tests for data file metadata setters and getters, (5) disallowing for setMeta(key) for all keys that use reserved words. The reserved words should be set via the API (this is true for codec and schema, and I suspect would be true for others). "ant clean test-java" passes in the java directory. The spec seems to build.
        Hide
        Doug Cutting added a comment -

        +1. Thanks, Philip!

        Show
        Doug Cutting added a comment - +1. Thanks, Philip!
        Hide
        Philip Zeyliger added a comment -

        I've committed this. Thanks for the discussion and review.

        Show
        Philip Zeyliger added a comment - I've committed this. Thanks for the discussion and review.

          People

          • Assignee:
            Philip Zeyliger
            Reporter:
            Philip Zeyliger
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development