Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-36057

Support volcano/alternative schedulers

Attach filesAttach ScreenshotAdd voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 3.2.0
    • Fix Version/s: None
    • Component/s: Kubernetes
    • Labels:
      None

      Description

      This is an umbrella issue for tracking the work for supporting Volcano & Yunikorn on Kubernetes. These schedulers provide more YARN like features (such as queues and minimum resources before scheduling jobs) that many folks want on Kubernetes.

       

      Yunikorn is an ASF project & Volcano is a CNCF project (sig-batch).

       

      They've taken slightly different approaches to solving the same problem, but from Spark's point of view we should be able to share much of the code.

       

      See the initial brainstorming discussion in SPARK-35623.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              holden Holden Karau

              Dates

              • Created:
                Updated:

                Issue deployment