[NIFI-11209] UpdateHive3Table and UpdateDatabaseTable erase newly created columns in output FlowFile

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.0.0-M1, 1.21.0
• Component/s: Extensions
• Assignee: Matt Burgess
• Reporter: Matt Burgess
• Created: 21/Feb/23 20:23
• Updated: 27/Feb/23 13:13
• Resolved: 27/Feb/23 13:09

Description

When using UpdateHive3Table or PutDatabaseTable with the Update Field Names property set to "true", the output will include only those fields that already had existing columns in the target table. This means if an incoming FlowFile has a "new" field, the column will be created and populated in the target database, but the output FlowFile will no longer have the "new" fields in the outgoing records.

This is because the original database columns are used to populate the outgoing record schema when the Update Field Names property is set to true. Instead the database columns should be refreshed/maintained after any DDL is executed in order to get the updated set of database column names.

Issue Links

• links to: GitHub Pull Request #6986