[HUDI-1580] Disaster Recovery in case where HBASE index table becomes unrecoverable

• Type: New Feature
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.9.0
• Component/s: index
• Labels: user-support-issues
• Assignee: Unassigned
• Reporter: Ryan Pifer
• Created: 03/Feb/21 20:30
• Updated: 27/Mar/22 14:39

Description

If an HBASE table becomes corrupted for any reason, we should think of a way that we can provide some sort of disaster recovery built in to re-build index. Currently only way to do this without any extra utilities is to rewrite entire dataset.

Can we create a CLI command which rebuilds just index?

Can we add checkpointing to hbase table to reduce disaster recovery time? i.e. only re-build index for records written after last checkpoint time