[FLINK-29367] Avoid manifest corruption for incorrect checkpoint recovery

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: table-store-0.2.0
• Fix Version/s: table-store-0.3.0, table-store-0.2.1
• Component/s: Table Store
• Labels: pull-request-available
• Assignee: Caizhi Weng
• Reporter: Jingsong Lee
• Created: 21/Sep/22 04:10
• Updated: 22/Sep/22 10:15
• Resolved: 22/Sep/22 10:15

Description

When the job runs to checkpoint N, if the user recovers from an old checkpoint (such as checkpoint N-5), the sink of the current FTS will cause a manifest corruption because duplicate files may be committed.

We should avoid such corruption, and the storage should be robust enough.

Issue Links

• links to: GitHub Pull Request #301