[HUDI-4895] Object Store based lock provider

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Won't Do
• Component/s: multi-writer
• Labels: pull-request-available
• Assignee: Yuwei Xiao
• Reporter: Yuwei Xiao
• Created: 22/Sep/22 01:56
• Updated: 20/Oct/22 06:16
• Resolved: 20/Oct/22 06:16

Description

Currently, we have `FileSystemBasedLockProvier`, which relies on the atomic guarantee of the underlying file system. Specifically, only with filesystem's atomic rename & atomic create capability, the LockProvider can work properly.

 

However, many of hudi users use object store (e.g., S3, OSS). So we wants to implement an object store based lock provider.

Issue Links

• links to: GitHub Pull Request #6738