[FLINK-33183] Enable metadata columns in NduAnalyzer with retract if non-virtual

• Type: Sub-task
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Table SQL / Planner
• Assignee: Unassigned
• Reporter: Timo Walther
• Created: 04/Oct/23 14:24
• Updated: 07/May/24 02:32

Description

Currently, the NduAnalyzer is very strict about metadata columns in updating sources. Compared to append and upsert sources (see also FLINK-33182), retract sources are tricky. And the analyzer is actually correct.

However, for retract sources we should expose more functionality to the user and add a warning to the documentation that retract mode could potentially cause NDU problems if not enough attention is paid. We should only throw an error on virtual metadata columns. Persisted metadata columns can be considered “safe“. When a metadata column is persisted, we can assume that an upstream Flink job fills its content thus likely also fills its correct retraction.