[HUDI-8507] Ensure false positive failures from MDT validations are accounted for

• Type: Improvement
• Status: Open
• Priority: Critical
• Resolution: Unresolved
• Fix Version/s: 1.0.0
• Component/s: metadata
• Story Points: 6
• Assignee: Lin Liu
• Reporter: sivabalan narayanan
• Created: 12/Nov/24 22:27
• Updated: 2 days ago 21:56

Description

We fixed couple of false positive validation failure cases for FILES partition and RLI in 

HoodieMetadataTableValidator. 

Ref https://github.com/apache/hudi/pull/11098 

 

essentially, in b/w the time we poll fs based view and MDT view, there could be additional commits in MDT. so, we need to account for them in our validation. 

 

Lets ensure we account for the same for other partitions like 

col stats, bloom filter, functional index, sec index and partition stats index.