[SPARK-3224] FetchFailed stages could show up multiple times in failed stages in web ui

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Fix Version/s: 1.1.0
• Component/s: Web UI
• Target Version/s: 1.1.0
• Assignee: Reynold Xin
• Reporter: Reynold Xin
• Created: 26/Aug/14 05:38
• Updated: 11/Dec/14 22:57
• Resolved: 27/Aug/14 05:13

Description

Today I saw a job in which a reduce stage failed and showed up a lot of times in the failed stages. I think the reason is that the DAGScheduler stage complete (with failure) event multiple times in the case of FetchFailed.

Issue Links

• duplicates: SPARK-2947 DAGScheduler resubmit the stage into an infinite loop
• is duplicated by: SPARK-2947 DAGScheduler resubmit the stage into an infinite loop
• links to: [Github] Pull Request #2127 (rxin)