[FLINK-25689] Introduce atomic commit

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: table-store-0.1.0
• Component/s: Table Store
• Labels: pull-request-available
• Assignee: Caizhi Weng
• Reporter: Caizhi Weng
• Created: 18/Jan/22 09:21
• Updated: 14/Feb/22 08:34
• Resolved: 21/Jan/22 05:39

Description

This ticket introduces an atomic commit transaction for file store.

1. Before calling FileStoreCommitImpl#commit, user should first call FileStoreCommitImpl#filterCommitted to make sure this commit is not done before.
2. Before committing, it will first check for conflicts by checking if all files to be removed currently exists.
3. After that it use the external FileStoreCommitImpl#lock (if provided) or the atomic rename of the file system to ensure atomicity.
4. If commit fails due to conflicts or exception it tries its best to clean up and aborts.
5. If atomic rename fails it tries again after reading the latest snapshot from step 2.

Issue Links

• links to: GitHub Pull Request #12