Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-1010

FairScheduler: decouple container scheduling from nodemanager heartbeats

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 2.1.0-beta
    • Fix Version/s: 2.3.0
    • Component/s: scheduler
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Currently scheduling for a node is done when a node heartbeats.

      For large cluster where the heartbeat interval is set to several seconds this delays scheduling of incoming allocations significantly.

      We could have a continuous loop scanning all nodes and doing scheduling. If there is availability AMs will get the allocation in the next heartbeat after the one that placed the request.

        Attachments

        1. YARN-1010.patch
          23 kB
          Wei Yan
        2. YARN-1010.patch
          23 kB
          Wei Yan

          Issue Links

            Activity

              People

              • Assignee:
                ywskycn Wei Yan
                Reporter:
                tucu00 Alejandro Abdelnur
              • Votes:
                0 Vote for this issue
                Watchers:
                15 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: