[HUDI-7489] Row writer clustering collects write statuses on the driver

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 0.15.0, 1.0.0
• Component/s: clustering, spark
• Labels: pull-request-available
• Assignee: Jonathan Vexler
• Reporter: Jonathan Vexler
• Created: 07/Mar/24 19:36
• Updated: 09/Jun/24 23:07
• Resolved: 12/Mar/24 00:26

Description

Row writer clustering collects write statuses on the driver which can be a very large amount of data. We Immediately parallelize the data again, so we can avoid this collect.

Issue Links

• links to: GitHub Pull Request #10836