Avro
  1. Avro
  2. AVRO-301

Handle non-reserved properties appropriately in the Python implementation

    Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.6.2
    • Fix Version/s: 1.6.3
    • Component/s: python
    • Labels:
      None
    • Release Note:
      Non-reserved schema properties are now processed and accessible via an "other_props" property in Schema and Field.
    1. AVRO-301-patch-3.patch
      12 kB
      Marcio Silva
    2. AVRO-301-patch-2.patch
      14 kB
      Marcio Silva
    3. AVRO-301-patch-1.patch
      9 kB
      Marcio Silva

      Activity

      Hide
      Miki Tebeka added a comment -

      Patch looks good, going to to commit tomorrow afternoon if there are no objections.

      Show
      Miki Tebeka added a comment - Patch looks good, going to to commit tomorrow afternoon if there are no objections.
      Hide
      Harsh J added a comment -

      +1. The changes look good here.

      (Am not a committer on Avro btw.)

      Show
      Harsh J added a comment - +1. The changes look good here. (Am not a committer on Avro btw.)
      Hide
      Marcio Silva added a comment -

      updated patch, decoupled from AVRO-300 patch.

      Show
      Marcio Silva added a comment - updated patch, decoupled from AVRO-300 patch.
      Hide
      Harsh J added a comment -

      Hi Marcio,

      I think its better if this goes first, and then the docs one cause 'doc' is a reserved name as well, and would be a nicer, complete commit at AVRO-300.

      Show
      Harsh J added a comment - Hi Marcio, I think its better if this goes first, and then the docs one cause 'doc' is a reserved name as well, and would be a nicer, complete commit at AVRO-300 .
      Hide
      Marcio Silva added a comment -

      I'm happy to decouple the patches, but just to be clear did you want this patch to apply to the pre AVRO-300 schema.py, or should I just wait until the docs change is committed and then generate a new patch that's not co-mingled?

      Show
      Marcio Silva added a comment - I'm happy to decouple the patches, but just to be clear did you want this patch to apply to the pre AVRO-300 schema.py, or should I just wait until the docs change is committed and then generate a new patch that's not co-mingled?
      Hide
      Harsh J added a comment -

      (Failed to add): I'll review actual patch once its been decoupled.

      Show
      Harsh J added a comment - (Failed to add): I'll review actual patch once its been decoupled.
      Hide
      Harsh J added a comment -

      -1 on this one (But I have +1'd the stuff on AVRO-300)

      The docs change ought to go with AVRO-300 after this, and must be removed from this patch first. It appears to be coupled. I'd prefer it separate.

      Show
      Harsh J added a comment - -1 on this one (But I have +1'd the stuff on AVRO-300 ) The docs change ought to go with AVRO-300 after this, and must be removed from this patch first. It appears to be coupled. I'd prefer it separate.
      Hide
      Doug Cutting added a comment -

      I'll commit this tomorrow unless there are objections.

      Show
      Doug Cutting added a comment - I'll commit this tomorrow unless there are objections.
      Hide
      Marcio Silva added a comment -

      Updated patch that includes unit test coverage.

      Show
      Marcio Silva added a comment - Updated patch that includes unit test coverage.
      Hide
      Doug Cutting added a comment -

      Can you please add some tests for this?

      Can someone more fluent in Python have a look at this too? It would be a useful feature.

      Show
      Doug Cutting added a comment - Can you please add some tests for this? Can someone more fluent in Python have a look at this too? It would be a useful feature.
      Hide
      Marcio Silva added a comment -

      This patch adds support for non-reserved properties to the Schemas that support them.
      It currently depends on my patch to AVRO-300, but that could be removed if necessary.

      It adds an "other_props" property to Schema and Field which exposes a dictionary of all the unreserved properties.

      Given example_schema below:

      { "name":"foo",
        "type":"record", 
        "custom_prop":[1,2,3]
        "fields":[ 
          {"name":"field1",
           "type":"int",
           "tag":"value"}
        ]
      }
      
      API Usage
      >>> from avro import schema
      >>> s = schema.parse(example_schema)
      >>> s.other_props
      {u'custom_prop': [1, 2, 3]}
      >>> [f.name for f in s.fields if f.get_prop('tag')]
      [u'field1']
      
      Show
      Marcio Silva added a comment - This patch adds support for non-reserved properties to the Schemas that support them. It currently depends on my patch to AVRO-300 , but that could be removed if necessary. It adds an "other_props" property to Schema and Field which exposes a dictionary of all the unreserved properties. Given example_schema below: { "name":"foo", "type":"record", "custom_prop":[1,2,3] "fields":[ {"name":"field1", "type":" int ", "tag":"value"} ] } API Usage >>> from avro import schema >>> s = schema.parse(example_schema) >>> s.other_props {u'custom_prop': [1, 2, 3]} >>> [f.name for f in s.fields if f.get_prop('tag')] [u'field1']
      Hide
      Harsh J added a comment -

      Jeff,

      I'm unclear on what this one is about. Could you elaborate a bit / add a brief description about what you mean by non-reserved properties?

      Show
      Harsh J added a comment - Jeff, I'm unclear on what this one is about. Could you elaborate a bit / add a brief description about what you mean by non-reserved properties?

        People

        • Assignee:
          Marcio Silva
          Reporter:
          Jeff Hammerbacher
        • Votes:
          0 Vote for this issue
          Watchers:
          1 Start watching this issue

          Dates

          • Created:
            Updated:
            Resolved:

            Development