[HUDI-2170] Always choose the latest record for HoodieRecordPayload

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.9.0
• Component/s: Common Core
• Labels: pull-request-available
• Assignee: Danny Chen
• Reporter: Danny Chen
• Created: 13/Jul/21 08:18
• Updated: 11/Aug/21 02:24
• Resolved: 11/Aug/21 02:21

Description

Now in OverwriteWithLatestAvroPayload.preCombine, we still choose the old record when the new record has the same preCombine field with the old one, actually it is more natural to keep the new incoming record instead. The DefaultHoodieRecordPayload.combineAndGetUpdateValue method already did that.

See issue: https://github.com/apache/hudi/issues/3266.

Issue Links

• duplicates: HUDI-1763 DefaultHoodieRecordPayload does not honor ordering value when records within multiple log files are merged
• links to: GitHub Pull Request #3267
• links to: GitHub Pull Request #3401