[HUDI-5824] COMBINE_BEFORE_UPSERT=false option does not work for upsert

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 0.12.1, 0.12.2, 0.13.0
• Fix Version/s: 0.13.1
• Component/s: spark
• Labels: pull-request-available
• Assignee: kazdy
• Reporter: kazdy
• Created: 21/Feb/23 14:52
• Updated: 24/May/23 18:21
• Resolved: 24/May/23 18:21

Description

hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieSparkSqlWriter.scala

shouldCombine does not take into account the situation where the write operation is UPSERT but COMBINE_BEFORE_UPSERT is false.

Currently, Hudi always combines records on UPSERT, and option COMBINE_BEFORE_UPSERT is not honored.

Issue Links

• relates to: HUDI-5272 Align with Flink to support no_precombine in spark
• links to: GitHub Pull Request #7998