Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-7317

Fix overallocation resulted from ceiling in LocalityMulticastAMRMProxyPolicy

    XMLWordPrintableJSON

    Details

    • Type: Task
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.9.0, 3.0.0
    • Component/s: None
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      When LocalityMulticastAMRMProxyPolicy is splitting up the ANY requests into different subclusters, we are doing Ceil(N * weight), leading to overallocation overall. It is better to do Floor(N * weight) for each subcluster and then assign the residue randomly according to the weights. So that the total number of containers we ask from all subclusters sum up to be N.

        Attachments

        1. YARN-7317.v1.patch
          22 kB
          Botong Huang
        2. YARN-7317.v2.patch
          24 kB
          Botong Huang
        3. YARN-7317.v3.patch
          25 kB
          Botong Huang
        4. YARN-7317.v4.patch
          25 kB
          Botong Huang

          Activity

            People

            • Assignee:
              botong Botong Huang
              Reporter:
              botong Botong Huang
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: