[SPARK-4989] wrong application configuration cause cluster down in standalone mode

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 1.0.0, 1.1.0, 1.2.0
• Fix Version/s: 1.1.2, 1.2.1, 1.3.0
• Component/s: Deploy, Spark Core
• Assignee: Zhang, Liye
• Reporter: Zhang, Liye
• Created: 29/Dec/14 08:04
• Updated: 30/Apr/15 12:51
• Resolved: 30/Apr/15 12:51

Description

when enabling eventlog in standalone mode, if give the wrong configuration, the standalone cluster will down (cause master restart, lose connection with workers).

How to reproduce: just give an invalid value to "spark.eventLog.dir", for example: spark.eventLog.dir=hdfs://tmp/logdir1, hdfs://tmp/logdir2. This will throw illegalArgumentException, which will cause the Master restart. And the whole cluster is not available.

This is not acceptable that cluster is crashed by one application's wrong setting.

Issue Links

• links to: [Github] Pull Request #3824 (liyezhang556520)
• links to: [Github] Pull Request #3969 (liyezhang556520)
• links to: [Github] Pull Request #3970 (liyezhang556520)
• links to: [Github] Pull Request #3971 (liyezhang556520)