Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-7317

Fix overallocation resulted from ceiling in LocalityMulticastAMRMProxyPolicy

    XMLWordPrintableJSON

Details

    • Task
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • None
    • 2.9.0, 3.0.0
    • None
    • None
    • Reviewed

    Description

      When LocalityMulticastAMRMProxyPolicy is splitting up the ANY requests into different subclusters, we are doing Ceil(N * weight), leading to overallocation overall. It is better to do Floor(N * weight) for each subcluster and then assign the residue randomly according to the weights. So that the total number of containers we ask from all subclusters sum up to be N.

      Attachments

        1. YARN-7317.v1.patch
          22 kB
          Botong Huang
        2. YARN-7317.v2.patch
          24 kB
          Botong Huang
        3. YARN-7317.v3.patch
          25 kB
          Botong Huang
        4. YARN-7317.v4.patch
          25 kB
          Botong Huang

        Activity

          People

            botong Botong Huang
            botong Botong Huang
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: