Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-20624 SPIP: Add better handling for node shutdown
  3. SPARK-20629

Copy shuffle data when nodes are being shut down

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.0
    • Fix Version/s: 3.1.0
    • Component/s: Kubernetes, Spark Core
    • Labels:
      None

      Description

      We decided not to do this for YARN, but for Kubernetes and similar systems nodes may be shut down entirely without the ability to keep an AuxiliaryService around.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                holden Holden Karau
                Reporter:
                holden Holden Karau
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: