Avro
  1. Avro
  2. AVRO-1265

Python: schema objects should support builder() default-filling behavior

    Details

    • Type: Improvement Improvement
    • Status: Patch Available
    • Priority: Minor Minor
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: 1.8.0
    • Component/s: python
    • Labels:

      Description

      There seems to be no way to easily use the avro libraries in Python (where I feel most qualified to comment) to encode generics with "missing default values" and have them transmitted in well-formed avro binary.

      If you fill in the "missing" default values, the Python libraries will transmit correctly.

      I'd be happy to add methods to the avro.RecordSchema objects (in the Python libraries) that "fill defaults" on missing member fields of a record, recursively (which probably means method extension of other schema classes as well).

      For backwards compatibility (and probably to avoid unnecessary data traversal), clients probably want to explicitly ask the schema to fill in defaults before transmission in the cases where you'd like to set only the non-default values.

      1. avro-1265a-build-defaults.patch
        3 kB
        Jeremy Kahn
      2. avro-1265b-tests.patch
        2 kB
        Jeremy Kahn

        Activity

        Doug Cutting made changes -
        Fix Version/s 1.8.0 [ 12323299 ]
        Fix Version/s 1.7.5 [ 12324096 ]
        Jeremy Kahn made changes -
        Status In Progress [ 3 ] Patch Available [ 10002 ]
        Hide
        Jeremy Kahn added a comment -

        Patches on top of AVRO-1284 provide schema.build_defaults(datum) behavior that expands datum to include default values if they are available.

        Useful for systematically producing valid data for (e.g.) IPC protocols.

        Show
        Jeremy Kahn added a comment - Patches on top of AVRO-1284 provide schema.build_defaults(datum) behavior that expands datum to include default values if they are available. Useful for systematically producing valid data for (e.g.) IPC protocols.
        Jeremy Kahn made changes -
        Attachment avro-1265a-build-defaults.patch [ 12576237 ]
        Attachment avro-1265b-tests.patch [ 12576238 ]
        Hide
        Jeremy Kahn added a comment -

        Implement default-build behavior on schema and update tests to do (rather cursory) testing of this behavior

        Show
        Jeremy Kahn added a comment - Implement default-build behavior on schema and update tests to do (rather cursory) testing of this behavior
        Hide
        Jeremy Kahn added a comment -

        Changes are revised to be at this github branch. Note also that AVRO-1284 is a prerequisite fix for this patch.

        Show
        Jeremy Kahn added a comment - Changes are revised to be at this github branch . Note also that AVRO-1284 is a prerequisite fix for this patch.
        Jeremy Kahn made changes -
        Field Original Value New Value
        Status Open [ 1 ] In Progress [ 3 ]
        Hide
        Jeremy Kahn added a comment -

        Here's where development is happening for this ticket. I need to add tests, and won't propose a patch until I have them. I'm hoping to find time to write the tests tomorrow or early next week.

        Show
        Jeremy Kahn added a comment - Here's where development is happening for this ticket. I need to add tests, and won't propose a patch until I have them. I'm hoping to find time to write the tests tomorrow or early next week.
        Hide
        Jeremy Kahn added a comment -

        see this thread on the mailing list.

        Show
        Jeremy Kahn added a comment - see this thread on the mailing list.
        Jeremy Kahn created issue -

          People

          • Assignee:
            Jeremy Kahn
            Reporter:
            Jeremy Kahn
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:

              Development