Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-4979

FSAppAttempt demand calculation considers demands at multiple locality levels different

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.8.0, 2.7.2
    • Fix Version/s: 2.9.0, 3.0.0-alpha1
    • Component/s: fairscheduler
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      FSAppAttempt adds duplicate ResourceRequest to demand in updateDemand. We should only count ResourceRequest for ResourceRequest.ANY when calculate demand.
      Because hasContainerForNode will return false if no container request for ResourceRequest.ANY and both allocateNodeLocal and allocateRackLocal will also decrease the number of containers for ResourceRequest.ANY.
      This issue may cause current memory demand overflow(integer) because duplicate requests can be on multiple nodes.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                zxu zhihai xu
                Reporter:
                zxu zhihai xu
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: