[HUDI-4373] Consistent bucket index write path for Flink engine

• Type: New Feature
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Fix Version/s: 0.16.0
• Component/s: flink, index
• Labels: pull-request-available
• Epic Link: Consistent Hashing Index
• Assignee: Jing Zhang
• Reporter: Yuwei Xiao
• Created: 08/Jul/22 08:04
• Updated: 11/Jun/24 01:31

Description

Simple bucket index (with fixed bucket number) is ready for Flink engine and has been used widely in the community. 

Since spark now support consistent bucket (dynamic bucket number), we should bridge the gap and bring this feature to Flink too.

Issue Links

• links to: GitHub Pull Request #6737

Sub-Tasks

• Flink write supports consistent bucket index: Closed
• Flink bulk_insert support consistent bucket index: Open
• Flink support generate resize plan for consistent bucket index: Closed
• Introduce UpdateStrategy for Flink to handle conflict between clustering/resize with update: Closed
• Update user document to introduce this feature: Closed