[HUDI-8164] Chain multiple indices support in HoodieFileIndex

• Type: Sub-task
• Status: Open
• Priority: Critical
• Resolution: Unresolved
• Fix Version/s: 1.0.1
• Component/s: reader-core
• Story Points: 2
• Assignee: Danny Chen
• Reporter: Danny Chen

Reviewers

Danny Chen

• Created: 04/Sep/24 03:50
• Updated: 12/Dec/24 23:14

Description

In HoodieFileIndex, we have 5 indices support: 

 

• RecordLevelIndexSupport
• BucketIndexSupport
• SecondaryIndexSupport
• FunctionalIndexSupport
• BloomFiltersIndexSupport
• ColumnStatsIndexSupport

 
But the logic would return early when the first index support applied and got pruned files.
A better way is we can chain the multiple indices support all-together: one index support can still be applied based on the pruned files returned by another index support.

Key points in contribution:
1. think through the shared contract/param among these indices support;
2. add a orthoganal test for random index supoprt composition, valid the result set and the indices support are applied effectively.