[HUDI-7146] Implement secondary index

• Type: Task
• Status: Closed
• Priority: Major
• Resolution: Done
• Fix Version/s: 1.0.0-beta2, 1.0.0
• Labels: hudi-1.0.0-beta2 pull-request-available
• Story Points: 8
• Epic Link: secondary index
• Assignee: Sagar Sumit
• Reporter: Sagar Sumit

Reviewers

Danny Chen

• Created: 27/Nov/23 17:15
• Updated: 31/Oct/24 18:11
• Resolved: 07/Jun/24 06:50

Description

1. Secondary index schema should be flexible enough to accommodate various kinds of secondary index. 
2. Reuse as much as possible the existing framework for indexing.
3. Merge with existing index config and introduce as less configs as possible.

Issue Links

• depends upon: HUDI-7458 Creating multiple functional index fails
• depends upon: HUDI-7472 Creating a functional index implicitly drops metadata RLI partition
• depends upon: HUDI-7474 Functional index creation fails for an existing table as reported by community user
• depends upon: HUDI-7480 initializeFunctionalIndexPartition is called multiple times
• links to: GitHub Pull Request #10211
• links to: GitHub Pull Request #10617
• links to: GitHub Pull Request #10814
• links to: GitHub Pull Request #11129
• links to: GitHub Pull Request #11146
• links to: GitHub Pull Request #11162

Sub-Tasks

• Implement writer path support for secondary index: Closed
• Implement reader path support for secondary index: Closed