[SPARK-17650] Adding a malformed URL to sc.addJar and/or sc.addFile bricks Executors

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.0.1, 2.1.0
• Component/s: Spark Core
• Assignee: Burak Yavuz
• Reporter: Burak Yavuz
• Created: 23/Sep/16 19:00
• Updated: 28/Sep/16 23:27
• Resolved: 26/Sep/16 05:58

Description

Using a malformed URL in sc.addJar or sc.addFile bricks the executors forever. The executors try to update their dependencies, but because the URL is malformed, they always throw a malformedURL exception. Then your cluster is unusable until you restart it.

Issue Links

• links to: [Github] Pull Request #15224 (brkyvz)