[HUDI-5675] rollback failed because rollbackFailedWrites schedule rollback on concurrent completed instant

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 0.14.0
• Labels: pull-request-available release-0.14.0-blocker
• Assignee: Qijun Fu
• Reporter: Qijun Fu
• Created: 01/Feb/23 08:56
• Updated: 31/Jan/24 09:04
• Resolved: 18/May/23 07:06

Description

rollbackFailedWrites now will get all inflight instants excluding compaction and clustering 

and rollback expired instants in them. However, for concurrent write, an instant can be included in infight instants and become completed before checking whether its heartbeat is expired. And then the completed instant be rollbacked which will cause data loss (For marker based rollback strategy, rollback will fail because we can't rollback an completed instant using marker). 

 

An pr will be issued to solve the problem

Issue Links

• Blocked: HUDI-7361 Fix a concurrency issue caused by clean
• links to: GitHub Pull Request #7826