[HUDI-1309] Listing Metadata unreadable in S3 as the log block is deemed corrupted

• Type: Task
• Status: Resolved
• Priority: Blocker
• Resolution: Cannot Reproduce
• Fix Version/s: 0.10.0
• Component/s: writer-core
• Epic Link: Metadata Table for File Listing & Query Planning
• Assignee: Nishith Agarwal
• Reporter: Balaji Varadarajan
• Created: 01/Oct/20 11:43
• Updated: 04/Jan/22 00:09
• Resolved: 01/Sep/21 19:12

Description

When running metadata list-partitions CLI command, I am seeing the below messages and the partition list is empty. Was expecting 10K partitions.

 

 36589 [Spring Shell] INFO org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner - Scanning log file HoodieLogFile{pathStr='s3a://robinhood-encrypted-hudi-data-cove/dev_hudi_tables/balaji_varadarajan/benchmark_1M_10K_partitions/.hoodie/metadata/metadata_partition/.f02585bd-bb02-43f6-8bc8-cec71df87d1e-0_00000000000000.log.1_0-23-206045', fileLen=0}
 36590 [Spring Shell] INFO org.apache.hudi.common.table.log.HoodieLogFileReader - Found corrupted block in file HoodieLogFile{pathStr='s3a://robinhood-encrypted-hudi-data-cove/dev_hudi_tables/balaji_varadarajan/benchmark_1M_10K_partitions/.hoodie/metadata/metadata_partition/.f02585bd-bb02-43f6-8bc8-cec71df87d1e-0_00000000000000.log.1_0-23-206045', fileLen=0} with block size(3723305) running past EOF
 36684 [Spring Shell] INFO org.apache.hudi.common.table.log.HoodieLogFileReader - Log HoodieLogFile{pathStr='s3a://xxxx/dev_hudi_tables/balaji_varadarajan/benchmark_1M_10K_partitions/.hoodie/metadata/metadata_partition/.f02585bd-bb02-43f6-8bc8-cec71df87d1e-0_00000000000000.log.1_0-23-206045', fileLen=0} has a corrupted block at 14
 44515 [Spring Shell] INFO org.apache.hudi.common.table.log.HoodieLogFileReader - Next available block in HoodieLogFile{pathStr='s3a://xxxx/dev_hudi_tables/balaji_varadarajan/benchmark_1M_10K_partitions/.hoodie/metadata/metadata_partition/.f02585bd-bb02-43f6-8bc8-cec71df87d1e-0_00000000000000.log.1_0-23-206045', fileLen=0} starts at 3723319
 44566 [Spring Shell] INFO org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner - Found a corrupt block in s3a://xxxx/dev_hudi_tables/balaji_varadarajan/benchmark_1M_10K_partitions/.hoodie/metadata/metadata_partition/.f02585bd-bb02-43f6-8bc8-cec71df87d1e-0_00000000000000.log.1_0-23-206045
 44567 [Spring Shell] INFO org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner - M