Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-6735

Provide options to make maximum executor failure count ( which kills the application ) relative to a window duration or disable it.

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.2.0, 1.2.1, 1.3.0
    • 2.0.0
    • Spark Submit, YARN
    • None

    Description

      Currently there is a setting (spark.yarn.max.executor.failures ) which tells maximum number of executor failures, after which Application fails.
      For long running applications, user can require not to kill the application at all or will require such setting relative to a window duration. This improvement is ti provide such options to make maximum executor failure count ( which kills the application ) relative to a window duration or disable it.

      Attachments

        Activity

          People

            jerryshao Saisai Shao
            twinkle Twinkle Sachdeva
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: