[FLINK-36095] KeyedLookupJoinWrapper should shuffle by input upsertKey instead of join key to avoid changelog disordering

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.20.0
• Fix Version/s: 2.0-preview
• Component/s: Table SQL / Planner
• Labels: pull-request-available
• Assignee: lincoln lee
• Reporter: lincoln lee
• Created: 19/Aug/24 10:26
• Updated: 02/Sep/24 09:35
• Resolved: 21/Aug/24 02:43

Description

If user encounter the NDU issue caused by a lookup join and enable the TRY_RESOLVE mode ( https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/concepts/determinism/#33-how-to-eliminate-the-impact-of-non-deterministic-update-in-streaming ), the current KeyedLookupJoinWrapper implementation use the join key as the shuffle key, this may lead to changelog disordering issue. It should be fixed to use input upsertKey(or the complete row if upsertKey is empty) instead of join key.

Issue Links

• links to: GitHub Pull Request #25217