[HUDI-8533] Bloom Functional Index creation without function fails

• Type: Task
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Fix Version/s: 1.0.0
• Labels: pull-request-available
• Epic Link: secondary index
• Assignee: Lokesh Jain
• Reporter: Vinoth Chandar

Reviewers

Sagar Sumit, Vinoth Chandar

• Created: 16/Nov/24 23:32
• Updated: 13/Dec/24 11:49
• Resolved: 22/Nov/24 14:49

Description

is this expected behavior? We should be able to create a bloom_filters index on any column without having to specify any options, including a conversion function. Also lets allow for users to configure the bloom filter - number of bits, fp ratio etc? 

spark-sql (default)> create index idx_bloom1 on hudi_table using bloom_filters(city);
24/11/16 10:04:44 ERROR SparkSQLDriver: Failed in [create index idx_bloom1 on hudi_table using bloom_filters(city)]
org.apache.hudi.exception.HoodieIndexException: Unknown hoodie index type:bloom_filters
	at org.apache.hudi.index.secondary.SecondaryIndexType.lambda$of$3(SecondaryIndexType.java:55)
	at java.util.Optional.orElseThrow(Optional.java:290)
	at org.apache.hudi.index.secondary.SecondaryIndexType.of(SecondaryIndexType.java:54)
	at org.apache.hudi.index.secondary.HoodieSecondaryIndex$Builder.setIndexType(HoodieSecondaryIndex.java:112)
	at org.apache.hudi.index.secondary.SecondaryIndexManager.create(SecondaryIndexManager.java:110)
	at org.apache.spark.sql.hudi.command.CreateIndexCommand.run(IndexCommands.scala:60)
 

Issue Links

• fixes: HUDI-8546 Fix bloom_filters index failing with and without functions configured
• fixes: HUDI-8743 Bloom filter w/ expression index support
• is a clone of: HUDI-8531 Disallow unsupported secondary index types
• is cloned by: HUDI-8534 Ensure index creation is idempotent in face of failures
• links to: GitHub Pull Request #12290