[SPARK-33146] Encountering an invalid rolling event log folder prevents loading other applications in SHS

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.1
• Fix Version/s: 3.0.2, 3.1.0
• Component/s: Spark Core
• Assignee: Adam Binford
• Reporter: Adam Binford
• Created: 14/Oct/20 11:15
• Updated: 19/Oct/20 20:10
• Resolved: 15/Oct/20 03:00

Description

A follow-on issue from https://issues.apache.org/jira/browse/SPARK-33133

If an invalid rolling event log folder is encountered by the Spark History Server upon startup, it crashes the whole loading process and prevents any valid applications from loading. We should simply catch the error, log it, and continue loading other applications.

Issue Links

• links to: [Github] Pull Request #30037 (Kimahriman)
• links to: [Github] Pull Request #30048 (dongjoon-hyun)
• links to: [Github] Pull Request #30051 (HeartSaVioR)