Uploaded image for project: 'Samza'
  1. Samza
  2. SAMZA-2347

Samza-sql: Fix AvroTypeFactoryImpl to properly denote a field as optional and nullable.

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • 1.3
    • None
    • None

    Description

      • Nullable fields in avro are of type union with one of the values being null.
      • Fields with default values are optional ONLY while reading. Default values are
      • meant for schema evolution where reader schema is different from writer schema.
      • Please note here that even the fields with default values MUST be specified at
      • the time of serialization.
        *
      • Behavior on the consumption side:
      • While reading, the fields that are not written will be filled with default values
      • that are specified in the reader schema. Please note that this happens only when
      • the field with default value is not present in writer schema but is present in
      • reader schema.
        *
      • Behavior on the producer side:
      • All non-nullable avro fields should have values including fields with default values.
      • Nullable fields need not be explicitly set as they are set at the time of serialization.
        *
      • So, all nullable fields are optional fields on the producer side in Avro.
      • {@link AvroTypeFactoryImpl} reflects the state of fields on the producer side.
        *
      • Note: There could be cases where the producer might embed some fields, in which case
      • a non-nullable field is considered optional.

      Attachments

        Issue Links

          Activity

            People

              atoomula Aditya Toomula
              atoomula Aditya Toomula
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 40m
                  40m