[FLINK-27335] Optimize async compaction in MergeTreeWriter

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: table-store-0.1.0
• Component/s: Table Store
• Labels: pull-request-available
• Assignee: Jingsong Lee
• Reporter: Jingsong Lee
• Created: 21/Apr/22 04:11
• Updated: 27/Apr/22 04:18
• Resolved: 27/Apr/22 04:18

Description

Currently Full Compaction may cause the writer to be blocked, which has an impact on LogStore latency.

We need to decouple compact and write, compact completely asynchronous.
But too many files will lead to unstable reads, when too many files, Compaction processing speed can not keep up with Writer, need to back press Writer.

Stop parameter: num-sorted-run.stop-trigger, default 10

Issue Links

• links to: GitHub Pull Request #97