Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-16654

UI Should show blacklisted executors & nodes

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.0.0
    • 2.2.0
    • Scheduler, Spark Core, Web UI
    • None

    Description

      SPARK-8425 will add the ability to blacklist entire executors and nodes to deal w/ faulty hardware. However, without displaying it on the UI, it can be hard to realize which executor is bad, and why tasks aren't getting scheduled on certain executors.

      As a first step, we should just show nodes and executors that are blacklisted for the entire application (no need to show blacklisting for tasks & stages).

      This should also ensure that blacklisting events get into the event logs for the history server.

      Attachments

        Issue Links

          blocks

          New Feature - A new feature of the product, which has yet to be developed. SPARK-16554 Spark should kill executors when they are blacklisted

          • Major - Major loss of function.
          • Resolved
          is blocked by

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-8425 Add blacklist mechanism for task scheduling

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          links to

          Pull request #16346 [Github] Pull Request #16346 (jsoltren)

          Activity

            People

              jsoltren Jose Soltren
              irashid Imran Rashid
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: