[AVRO-2910] Random untion(__, null) data doesn't conform to schema

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Not A Problem
• Component/s: tools
• Assignee: Unassigned
• Reporter: Chris Coutinho
• Created: 08/Aug/20 13:20
• Updated: 08/Aug/20 14:29
• Resolved: 08/Aug/20 14:29

Description

The random Avro data generator tool doesn't generate data that conforms to a schema containing nullable fields.

For example, the following test.avdl can be used to define a schema for single `test` record, where some of the fields are nullable based on `union`. There doesn't seem to be a difference between setting null first or last.

 

Using the `idl2schemata` tool, the attached .avdl file generates the attached (.avsc) schema

 

Using the `random` tool and piping the result into `tojson`, you can see that the generated data (attached sample.json) doesn't conform to the schema:

 

$ avro-tools random --count 10 --schema-file test.avsc - | avro-tools tojson --reader-schema-file test.avsc - > sample.json

 

Note how the non-null random values are unnecessarily nested in the following examples, where the generated value is nested within an object with the data type as a key (e.g. "null_num2":{"int":1517937645}` instead of `"null_num2":1517937645)