[GOBBLIN-1827] Avro Flattener should order nested optional fields correctly if they have a non-null default value

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: gobblin-core
• Assignee: Abhishek Tiwari
• Reporter: William Lo
• Created: 28/Apr/23 03:10
• Updated: 01/May/23 18:40

Description

• Consider the following description:

• Record R1 {
• fields : {[
• {
• Union : [
• null,
• Record R2 {
• fields : {[
• { * String S1 *. default: String S3 * }

  , {

• String S2
• }
• ]}
• }
• }
• ]}
• }
• will be flattened to:
• Record R1 {
• fields : {[
• { * Union : [ null, String S1] *. default: String S3 * }

  , {

• Union : [ null, String S2]
• }
• ]}
• }
  Due to the addition of the AvroCompatibilityHelper, Avro flattening can run into an issue in the above scenario due to the nested non-null default S3 since the default type needs to be the first type in the field order. So R1 should have field S1 as Union [String S1, null]