Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-22426

Spark AM launching containers on node where External spark shuffle service failed to initialize

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Duplicate
    • 1.6.3
    • None
    • Shuffle, Spark Core, YARN
    • None

    Description

      When Spark External Shuffle Service on a NodeManager fails, the remote executors will fail while fetching the data from the executors launched on this Node. Spark ApplicationMaster should not launch containers on this Node or not use external shuffle service.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              prabhujoseph Prabhu Joseph
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: