Uploaded image for project: 'Avro'
  1. Avro
  2. AVRO-368

Reserve avro.* in object container files, and rename existing reserved words.

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 1.3.0
    • Fix Version/s: 1.3.0
    • Component/s: spec
    • Labels:
      None
    • Hadoop Flags:
      Incompatible change, Reviewed

      Description

      We propose to move "codec", "schema", and "sync" into "avro.codec", "avro.schema", and "avro.sync", and furthermore reserve all avro.* names in the object container file specification.

      (Discussion was started in the thread "Reserving more keywords in Avro Object Container Files?" on avro-dev.)

      1. AVRO-368.patch.txt
        8 kB
        Philip Zeyliger

        Activity

        Hide
        philip Philip Zeyliger added a comment -

        I've committed this. Thanks for the discussion and review.

        Show
        philip Philip Zeyliger added a comment - I've committed this. Thanks for the discussion and review.
        Hide
        cutting Doug Cutting added a comment -

        +1. Thanks, Philip!

        Show
        cutting Doug Cutting added a comment - +1. Thanks, Philip!
        Hide
        philip Philip Zeyliger added a comment -

        I went ahead and implemented this change. The patch below includes: (1) changes to the spec, (2) changes to DataFileConstants, (3) wider use of DataFileConstants, (4) new tests for data file metadata setters and getters, (5) disallowing for setMeta(key) for all keys that use reserved words. The reserved words should be set via the API (this is true for codec and schema, and I suspect would be true for others).

        "ant clean test-java" passes in the java directory. The spec seems to build.

        Show
        philip Philip Zeyliger added a comment - I went ahead and implemented this change. The patch below includes: (1) changes to the spec, (2) changes to DataFileConstants, (3) wider use of DataFileConstants, (4) new tests for data file metadata setters and getters, (5) disallowing for setMeta(key) for all keys that use reserved words. The reserved words should be set via the API (this is true for codec and schema, and I suspect would be true for others). "ant clean test-java" passes in the java directory. The spec seems to build.
        Hide
        cutting Doug Cutting added a comment -

        +1

        Note that, post AVRO-160, the sync marker is no longer in the metadata, but immediately follows the metadata, terminating the header and prefixing the first block, so that all blocks are now bracketed by sync markers. So "avro.schema" and "avro.codec" are currently the only reserved data file metadata keys.

        Show
        cutting Doug Cutting added a comment - +1 Note that, post AVRO-160 , the sync marker is no longer in the metadata, but immediately follows the metadata, terminating the header and prefixing the first block, so that all blocks are now bracketed by sync markers. So "avro.schema" and "avro.codec" are currently the only reserved data file metadata keys.

          People

          • Assignee:
            philip Philip Zeyliger
            Reporter:
            philip Philip Zeyliger
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development