[HUDI-7152] Fix default value parsing in JSON schema converter

• Type: Bug
• Status: Open
• Priority: Blocker
• Resolution: Unresolved
• Fix Version/s: 1.1.0
• Assignee: Ethan Guo
• Reporter: Ethan Guo
• Created: 28/Nov/23 17:27
• Updated: 21/Dec/23 15:27

Description

The default value can be in the following format in the JSON schema: (1) "field":{"type":"integer","default":"50"}{}, (2) {{{}"field":{"oneOf":[

{"type":"null"}

,{"type":"integer","default":"60"}]}{}}}. In both cases, while the field type is integer (non-String), the default value is put as a String value. The Avro schema requires strict type match for the default value and correct ordering of types (e.g., "type":["long","null"]).  Right now the Avro schema with the default value generated is not valid, causing the following exception

Caused by: org.apache.avro.AvroTypeException: Invalid default for field field1: "0" not a ["null","long"]
	at org.apache.avro.Schema.validateDefault(Schema.java:1576)
	at org.apache.avro.Schema.access$500(Schema.java:91)
	at org.apache.avro.Schema$Field.<init>(Schema.java:549)
	at org.apache.avro.Schema.parse(Schema.java:1683)
	at org.apache.avro.Schema.parse(Schema.java:1760)
	at org.apache.avro.Schema.parse(Schema.java:1673)
	at org.apache.avro.Schema$Parser.parse(Schema.java:1430)
	at org.apache.avro.Schema$Parser.parse(Schema.java:1418)
	at org.apache.hudi.utilities.schema.SchemaRegistryProvider.parseSchemaFromRegistry(SchemaRegistryProvider.java:79)
	... 14 more