[SPARK-8606] Exceptions in RDD.getPreferredLocations() and getPartitions() should not be able to crash DAGScheduler

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 1.2.0, 1.3.0, 1.4.0, 1.5.0
• Fix Version/s: 1.3.2, 1.4.1, 1.5.0
• Component/s: Scheduler, Spark Core
• Target Version/s: 1.4.2, 1.5.0
• Assignee: Josh Rosen
• Reporter: Josh Rosen
• Created: 24/Jun/15 22:16
• Updated: 17/May/20 17:46
• Resolved: 27/Jun/15 21:55

Description

RDD.getPreferredLocations() and RDD.getPartitions() may throw exceptions but the DAGScheduler does not guard against this, leaving it vulnerable to crashing and stopping the SparkContext if exceptions occur there.

We should fix this by adding more try blocks around these calls in DAGScheduler.

Issue Links

• links to: [Github] Pull Request #7023 (JoshRosen)