[FLINK-27707] Implement ManagedTableFactory#onCompactTable

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: table-store-0.2.0
• Fix Version/s: table-store-0.2.0
• Component/s: Table Store
• Labels: pull-request-available
• Assignee: Jane Chan
• Reporter: Jane Chan
• Created: 20/May/22 03:20
• Updated: 10/Jun/22 13:58
• Resolved: 10/Jun/22 10:10

Description

As discussed offline, the new implementation has been modified.

• Don't perform a scan during the planning phase. Instead, we put a flag along with part spec to indicate it is ordinary manual trigger compaction.

• Introduce a new PrecommittingSinkWriter impl to perform dedicated compaction tasks. This writer is responsible for scanning and selecting partition and bucket according to the current sub-task id, and then creating a per-bucket compact writer to submit compaction. Since there's no data shuffled between source and sink, so all the compaction is performed when SinkWriterOperator#prepareCommit is invoked.

Issue Links

• links to: GitHub Pull Request #138