Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-31509

Recommend user to disable delay scheduling for barrier taskset

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 2.4.0
    • Fix Version/s: None
    • Component/s: Spark Core
    • Labels:
      None

      Description

      Currently, barrier taskset can only be scheduled when all tasks are launched at the same time. As a result, delay scheduling becomes the biggest obstacle for the barrier taskset  to get scheduled. And the application can be easily fail with barrier taskset when delay scheduling is on. So, probably, we should recommend user to disable delay scheduling while using barrier execution.

       

      BTW, we could still support delay scheduling with barrier taskset if we find a way to resolve SPARK-24818

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              Ngone51 wuyi
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: