[HUDI-5764] Allow lazy rollback for async indexer commit

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Fix Version/s: 0.13.0, 0.13.1, 0.12.3
• Component/s: index, metadata
• Labels: pull-request-available
• Story Points: 4
• Epic Link: Async Metadata Indexing
• Assignee: Ethan Guo
• Reporter: Ethan Guo
• Created: 10/Feb/23 18:24
• Updated: 22/May/23 08:59
• Resolved: 14/Feb/23 00:32

Description

This is to fix HUDI-5733, where async indexer may fail due to eager rollback in metadata table.

Temporary solution for 0.13.0: Little more invovled and not so clean fix. Apply eager rollbacks only for regular delta commits. Deduce delta commits from HoodieIndexer and employ lazy clean policy(based on heartbeat).

Issue Links

• fixes: HUDI-5733 TestHoodieDeltaStreamer.testHoodieIndexer failure
• links to: GitHub Pull Request #7921