Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-1290

Let continuous scheduling achieve more balanced task assignment

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.3.0
    • Component/s: None
    • Labels:
      None

      Description

      Currently, in continuous scheduling (YARN-1010), in each round, the thread iterates over pre-ordered nodes and assigns tasks. This mechanism may overload the first several nodes, while the latter nodes have no tasks.

      We should sort all nodes according to available resource. In each round, always assign tasks to nodes with larger capacity, which can balance the load distribution among all nodes.

        Attachments

        1. main.pdf
          108 kB
          Wei Yan
        2. YARN-1290.patch
          7 kB
          Wei Yan
        3. YARN-1290.patch
          7 kB
          Wei Yan
        4. YARN-1290.patch
          8 kB
          Wei Yan
        5. YARN-1290.patch
          6 kB
          Wei Yan

          Activity

            People

            • Assignee:
              ywskycn Wei Yan
              Reporter:
              ywskycn Wei Yan
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: