Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-4963

capacity scheduler: Make number of OFF_SWITCH assignments per heartbeat configurable

    Details

    • Hadoop Flags:
      Reviewed

      Description

      Currently the capacity scheduler will allow exactly 1 OFF_SWITCH assignment per heartbeat. With more and more non MapReduce workloads coming along, the degree of locality is declining, causing scheduling to be significantly slower. It's still important to limit the number of OFF_SWITCH assignments to avoid densely packing OFF_SWITCH containers onto nodes.

      Proposal is to add a simple config that makes the number of OFF_SWITCH assignments configurable.

      Will upload candidate patch shortly.

        Attachments

        1. YARN-4963.001.patch
          12 kB
          Nathan Roberts
        2. YARN-4963.002.patch
          12 kB
          Nathan Roberts
        3. YARN-4963.003.patch
          12 kB
          Nathan Roberts
        4. YARN-4963.004.patch
          12 kB
          Nathan Roberts

          Issue Links

            Activity

              People

              • Assignee:
                nroberts Nathan Roberts
                Reporter:
                nroberts Nathan Roberts
              • Votes:
                0 Vote for this issue
                Watchers:
                11 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: