[SPARK-32124] [SHS] Failed to parse FetchFailed TaskEndReason from event log produce by Spark 2.4

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 3.0.0
• Fix Version/s: 3.0.1, 3.1.0
• Component/s: Spark Core
• Assignee: Zhongwei Zhu
• Reporter: Zhongwei Zhu
• Created: 28/Jun/20 22:46
• Updated: 01/Jul/20 08:22
• Resolved: 29/Jun/20 04:07

Description

When read event log produced by Spark 2.4.4, parsing TaskEndReason failed to due to missing field "Map Index".

 

Issue Links

• is caused by: SPARK-25341 Support rolling back a shuffle map stage and re-generate the shuffle files
• links to: [Github] Pull Request #28941 (warrenzhu25)
• links to: [Github] Pull Request #28965 (xuanyuanking)