Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-4989

wrong application configuration cause cluster down in standalone mode

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 1.0.0, 1.1.0, 1.2.0
    • Fix Version/s: 1.1.2, 1.2.1, 1.3.0
    • Component/s: Deploy, Spark Core
    • Labels:
      None

      Description

      when enabling eventlog in standalone mode, if give the wrong configuration, the standalone cluster will down (cause master restart, lose connection with workers).

      How to reproduce: just give an invalid value to "spark.eventLog.dir", for example: spark.eventLog.dir=hdfs://tmp/logdir1, hdfs://tmp/logdir2. This will throw illegalArgumentException, which will cause the Master restart. And the whole cluster is not available.

      This is not acceptable that cluster is crashed by one application's wrong setting.

        Attachments

          Activity

            People

            • Assignee:
              liyezhang556520 Zhang, Liye
              Reporter:
              liyezhang556520 Zhang, Liye
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: