Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-31173

Spark Kubernetes add tolerations and nodeName support

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Open
    • Priority: Trivial
    • Resolution: Unresolved
    • Affects Version/s: 2.4.6, 3.1.0
    • Fix Version/s: None
    • Component/s: Kubernetes, Spark Core
    • Labels:
    • Environment:

      Alibaba Cloud ACK with spark operator(v1beta2-1.1.0-2.4.5) and spark(2.4.5)

    • Flags:
      Patch

      Description

      When you run spark on serverless kubernetes cluster(virtual-kubelet). you need to specific the nodeSelectors,tolerations even nodeName when you want to gain better scheduling performance. Currently spark doesn't support tolerations. If you want to use this feature, You must use admission controller webhook to decorate the pod. But the performance is extremely bad. Here is the benchmark. 

      With webhook 

      Batch Size: 500 Pod creation: about 7 Pods/s   All Pods running: 5min

      Without webhook 

      Batch Size: 500 Pod creation: more than 500 Pods/s All Pods running: 45s

      Adding tolerations and nodeName in spark will bring great help when you want to run a large scale job on serverless kubernetes cluster.

       

       

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              Ringtail zhongwei liu
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:

                Time Tracking

                Estimated:
                Original Estimate - 72h
                72h
                Remaining:
                Remaining Estimate - 72h
                72h
                Logged:
                Time Spent - Not Specified
                Not Specified