[HUDI-3755] Change the index plan to capture everything that is needed to create index

• Type: Task
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Fix Version/s: 1.0.0-beta1
• Epic Link: Async Metadata Indexing
• Assignee: Sagar Sumit
• Reporter: Sagar Sumit
• Created: 31/Mar/22 06:16
• Updated: 21/Sep/23 11:58
• Resolved: 21/Sep/23 11:58

Description

Currently, we capture the MetadataPartitionType and base instant upto which there are no holes in the timeline. We can also write other options, e.g. columns that are going to be indexed, catch up start instant, etc. to the plan. This will make it easier to reindex,  just read the plan, map it to configs and start indexing. This also makes the write client APIs cleaner.