[AVRO-4014] [Rust] Sporadic value-schema mismatch with fixed struct

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 1.11.3
• Fix Version/s: 1.12.0, 1.11.4
• Component/s: rust
• Labels: pull-request-available
• Assignee: Martin Tzvetanov Grigorov
• Reporter: Matthew Tanous
• Created: 08/Jul/24 16:03
• Updated: 12/Jul/24 12:13
• Resolved: 11/Jul/24 06:12

Description

We are trying to Avro encode a structure before writing to Kafka, and when we are at high load writing the struct into an Avro writer (we started seeing around 2.6% error rates at 500K messages per minute) we start seeing this error: 

Value does not match schema: Reason: Unsupported value-schema combination

This is surprising as the same logic is used to build the record in each case, and that record is built using the Avro record type with the same schema:

Record::new(&self.schema)

This is the code that is ultimately raising the error, but because it is not specifying which value does not match which part of the schema, it is extremely difficult to debug.

                let mut writer = Writer::new(&self.schema, Vec::new());
                writer
                    .append(record) // This will fail if the message and schema don't match
                    .map_err(|err| Report::msg(err.to_string()))?;

A simple start would be to add logging of the value and the schema that are mismatched to help us debug this issue, as I'm not able to determine if the `apache-avro` library is doing something erroneous or our code is breaking in some unforeseen way.

Issue Links

• links to: GitHub Pull Request #3007