[HUDI-6817] After enable speculation execution of spark compaction job, some broken parquet files might be generated

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: compaction, core, table-service
• Assignee: Unassigned
• Reporter: Jing Zhang
• Created: 05/Sep/23 02:10
• Updated: 28/Mar/24 22:36

Description

After enable speculation execution of spark compaction job, some broken parquet might be generated. It would lead to subsequent jobs (no matter reader jobs, ingestion jobs or compaction jobs) failures.
see more details in https://github.com/apache/hudi/issues/9615.