[NIFI-13362] JSONRecordSetWriter does not account for schema changes when writing serialized form

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 1.26.0, 2.0.0-M3
• Fix Version/s: 2.0.0
• Component/s: Core Framework
• Assignee: Unassigned
• Reporter: Sander Bylemans
• Created: 05/Jun/24 07:19
• Updated: 29/Oct/24 20:34
• Resolved: 29/Oct/24 20:34

Description

When using the RemoveRecordField processor with the JsonRecordSetWriter as a writer, I came across an issue where not all fields were removed in the resulting records.

When debugging, I noticed the JsonRecordSetWriter uses the WriteJsonResult, which checks if there is a serialized form of the record. If there is, it just uses that even though the serialized form may contain fields that are not present anymore.

There is a check on the schema of the record, if it is the same as the target schema, but the serialized form does not account for this schema change introduced by the RemoveRecordField processor.

Issue Links

• is duplicated by: NIFI-13843 Unknown fields not dropped by JSON Writer as expected by specified schema