[HUDI-3661] Flink async compaction is not thread safe when use watermark

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Resolved
• Fix Version/s: 0.12.2
• Labels: pull-request-available
• Assignee: Unassigned
• Reporter: hd zhou
• Created: 18/Mar/22 11:41
• Updated: 16/Feb/23 05:54
• Resolved: 26/Dec/22 09:09

Description

async compaction will start a executor async compaciton and send compaction result message to next flink operator. But collector.collect() is not a threadsafe function. when use watermark or latencyMarker, they both call collector.collect() may cause issue.

we should not let async compaction = false

 

 

 

 

 

 

Issue Links

• links to: GitHub Pull Request #7399
• links to: GitHub Pull Request #7465