[HUDI-7007] Integrate functional index using bloom filter on reader side

• Type: Task
• Status: Closed
• Priority: Major
• Resolution: Done
• Fix Version/s: 1.0.0-beta2, 1.0.0
• Labels: hudi-1.0.0-beta2 pull-request-available
• Story Points: 6
• Epic Link: Indexing Functions
• Assignee: Sagar Sumit
• Reporter: Sagar Sumit

Reviewers

Ethan Guo, Jonathan Vexler

• Created: 30/Oct/23 10:07
• Updated: 10/Jun/24 17:56
• Resolved: 01/Jun/24 11:42

Description

Currently, one can create a functional index on a column using bloom filters. However, only the one created using column stats is supported on the reader side (check `FunctionalIndexSupport`). This ticket tracks the support for using bloom filters on functional index in the reader path.

Issue Links

• depends upon: HUDI-7458 Creating multiple functional index fails
• depends upon: HUDI-7472 Creating a functional index implicitly drops metadata RLI partition
• depends upon: HUDI-7480 initializeFunctionalIndexPartition is called multiple times
• depends upon: HUDI-7559 Fix functional index (on column stats): Handle NPE in filterQueriesWithRecordKey(...)
• depends upon: HUDI-7569 Fix wrong result while using RLI for pruning files
• depends upon: HUDI-7579 Functional index (on col stats) creation fails to process all files/partitions
• depends upon: HUDI-7582 Fix NPE in FunctionalIndexSupport::loadFunctionalIndexDataFrame()
• links to: GitHub Pull Request #11043