[AVRO-3683] Rust Writer, Reader can't use Schemas with dependencies in other Schemas. i.e. The output of Schema::parse_list

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.11.1
• Fix Version/s: 1.12.0, 1.11.2
• Component/s: rust
• Labels: pull-request-available

Language

Rust

• Assignee: Martin Tzvetanov Grigorov
• Reporter: Mark Farnan
• Created: 02/Dec/22 18:06
• Updated: 30/May/23 07:06
• Resolved: 13/Mar/23 07:10

Description

Avro permits Schema definitions to reference Types defined in other Schema files. 

Currently these can be parsed with Schema:parse_list, but the output of this can't be used in Writer, Reader,  (or to_avro_datum,  from_avro_datum)

• These functions only take a single 'schema' and can't resolve the References
• Schema:parse_list puts in 'refs' for the external dependencies, but doesn't seem to populate the UnionSchema's, so they can be resolved.

 

Example of dependant schemas:
letschema_TypeA=r#"
{
"type": "record",
"namespace": "base.name.space",
"name": "MultiSchemaTestTypeA",
"fields":
[

{ "name": "b", "type": ["null", "some.name.space.MultiSchemaTestTypeB"] }

,

{ "name": "c", "type":

{ "type": "map", "values": "some.name.space.MultiSchemaTestTypeB" }

, "default": {}
}
]
}
"#;

letschema_TypeB=r#"
{
"type": "record",
"namespace": "some.name.space",
"name": "MultiSchemaTestTypeB",
"fields":
[

{ "name": "d", "type": "string" }

,

{ "name": "e", "type": "int" }

]
}
"#;
 

PR Created with a failing test:  https://github.com/apache/avro/pull/1997

Issue Links

• is duplicated by: AVRO-3763 [Rust] Improve Refs workflow
• links to: GitHub Pull Request #1997
• links to: GitHub Pull Request #2014