[AVRO-2035] enable validation of default values in schemas by default

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.8.1
• Fix Version/s: 1.9.0
• Component/s: java
• Hadoop Flags: Incompatible change
• Assignee: Doug Cutting
• Reporter: Radai Rosenblatt
• Created: 17/May/17 01:27
• Updated: 19/Feb/21 10:07
• Resolved: 27/Feb/18 22:06

Description

suppose i have the following schema evolution:

{
  "name": "Bob",
  "type": "record",
  "fields": [
    {"name": "f1", "type": "int"}
  ]
}

and then:

{
  "name": "Bob",
  "type": "record",
  "fields": [
    {"name": "f1", "type": "int"},
    {"name": "f2", "type": "boolean", "default": "true"}
  ]
}

the default value for "f2" is specified as the STRING "true" (and not the literal boolean true).

if this default value is ever accessed (when reading a gen1-serialized object as a gen2) we get this:

org.apache.avro.AvroTypeException: Non-boolean default for boolean: "true"

	at org.apache.avro.io.parsing.ResolvingGrammarGenerator.encode(ResolvingGrammarGenerator.java:408)
	at org.apache.avro.io.parsing.ResolvingGrammarGenerator.getBinary(ResolvingGrammarGenerator.java:307)
	at org.apache.avro.io.parsing.ResolvingGrammarGenerator.resolveRecords(ResolvingGrammarGenerator.java:285)
	at org.apache.avro.io.parsing.ResolvingGrammarGenerator.generate(ResolvingGrammarGenerator.java:118)
	at org.apache.avro.io.parsing.ResolvingGrammarGenerator.generate(ResolvingGrammarGenerator.java:50)
	at org.apache.avro.io.ResolvingDecoder.resolve(ResolvingDecoder.java:85)
	at org.apache.avro.io.ResolvingDecoder.<init>(ResolvingDecoder.java:49)
	at org.apache.avro.io.DecoderFactory.resolvingDecoder(DecoderFactory.java:307)
	at org.apache.avro.generic.GenericDatumReader.getResolver(GenericDatumReader.java:127)
	at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:142)

yet Schema.parse() passes for this

Issue Links

• relates to: HIVE-24797 Disable validate default values when parsing Avro schemas
• links to: GitHub Pull Request #288