[HUDI-5324] Spark SQL MERGE INTO statement should always do upsert if there's matching update clause

• Type: Improvement
• Status: Open
• Priority: Critical
• Resolution: Unresolved
• Component/s: index, spark-sql
• Assignee: Alexey Kudinkin
• Reporter: Ethan Guo (this is the old account; please use "yihua")
• Created: 03/Dec/22 00:04
• Updated: 10/Mar/23 01:50

Description

UPDATED

Aforementioned issue was actually a result of misconfiguration of the Merge Into statement – MIT was using "insert" operation instead of "upsert".

Real issue though is that MIT implicitly predicates using "upsert" operation onto whether "preCombine" config is set. Instead, it should always specify operation as "upsert", since MIT allows to specify updating semantics w/o requiring presence of the "preCombine" field