[HUDI-3975] Checksum can be wrong after table upgrade from version 3 to 4

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Not A Problem
• Fix Version/s: 0.11.0
• Story Points: 1
• Assignee: Ethan Guo
• Reporter: Ethan Guo
• Created: 25/Apr/22 19:39
• Updated: 30/Apr/22 00:04
• Resolved: 30/Apr/22 00:04

Description

In ThreeToFourUpgradeHandler, more table properties can be added after checksum is generated, causing the checksum to be wrong.

@Override
public Map<ConfigProperty, String> upgrade(HoodieWriteConfig config, HoodieEngineContext context, String instantTime, SupportsUpgradeDowngrade upgradeDowngradeHelper) {
  Map<ConfigProperty, String> tablePropsToAdd = new Hashtable<>();
  tablePropsToAdd.put(TABLE_CHECKSUM, String.valueOf(HoodieTableConfig.generateChecksum(config.getProps())));
  // if metadata is enabled and files partition exist then update TABLE_METADATA_INDEX_COMPLETED
  // schema for the files partition is same between the two versions
  if (config.isMetadataTableEnabled() && metadataPartitionExists(config.getBasePath(), context, MetadataPartitionType.FILES)) {
    tablePropsToAdd.put(TABLE_METADATA_PARTITIONS, MetadataPartitionType.FILES.getPartitionPath());
  }
  return tablePropsToAdd;
}