[HUDI-3786] how to deduce what MDT partitions to update on the write path w/ async indeing

• Type: Bug
• Status: Closed
• Priority: Minor
• Resolution: Implemented
• Fix Version/s: 1.0.0-beta1
• Component/s: code-quality, metadata
• Epic Link: Async Metadata Indexing
• Assignee: Sagar Sumit
• Reporter: sivabalan narayanan
• Created: 03/Apr/22 19:37
• Updated: 28/Aug/23 10:40
• Resolved: 28/Aug/23 10:40

Description

w/ async indexing, how do we deduce what are the MDT partitions to update on the regular write path? 

 

private MetadataRecordsGenerationParams getRecordsGenerationParams() {
  return new MetadataRecordsGenerationParams(
      dataMetaClient, enabledPartitionTypes, dataWriteConfig.getBloomFilterType(),
      dataWriteConfig.getBloomIndexParallelism(),
      dataWriteConfig.isMetadataColumnStatsIndexEnabled(),
      dataWriteConfig.getColumnStatsIndexParallelism(),
      StringUtils.toList(dataWriteConfig.getColumnsEnabledForColumnStatsIndex()),
      StringUtils.toList(dataWriteConfig.getColumnsEnabledForBloomFilterIndex()));
} 

As of now, I see above code snippet is what deciding that. But don't we need to decide on tableConfig ?