Uploaded image for project: 'Apache Tez'
  1. Apache Tez
  2. TEZ-3770

DAG-aware YARN task scheduler

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.9.2
    • None
    • None
    • Reviewed

    Description

      There are cases where priority alone does not convey the relationship between tasks, and this can cause problems when scheduling or preempting tasks. If the YARN task scheduler was aware of the relationship between tasks then it could make smarter decisions when trying to assign tasks to containers or preempt running tasks to schedule pending tasks.

      Attachments

        1. TEZ-3770.001.patch
          169 kB
          Jason Darrell Lowe
        2. TEZ-3770.002.patch
          172 kB
          Jason Darrell Lowe

        Issue Links

          is related to

          Sub-task - The sub-task of the issue TEZ-394 Better scheduling for uneven DAGs

          • Major - Major loss of function.
          • Patch Available

          Activity

            People

              jlowe Jason Darrell Lowe
              jlowe Jason Darrell Lowe
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: