Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-1489 [Umbrella] Work-preserving ApplicationMaster restart
  3. YARN-2354

DistributedShell may allocate more containers than client specified after it restarts

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.6.0
    • Component/s: None
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      To reproduce, run distributed shell with -num_containers option,
      In ApplicationMaster.java, the following code has some issue.

        int numTotalContainersToRequest =
              numTotalContainers - previousAMRunningContainers.size();
          for (int i = 0; i < numTotalContainersToRequest; ++i) {
            ContainerRequest containerAsk = setupContainerAskForRM();
            amRMClient.addContainerRequest(containerAsk);
          }
          numRequestedContainers.set(numTotalContainersToRequest);
      

      numRequestedContainers doesn't account for previous AM's requested containers. so numRequestedContainers should be set to numTotalContainers

        Attachments

        1. YARN-2354-072914.patch
          4 kB
          Li Lu
        2. YARN-2354-072814.patch
          4 kB
          Li Lu
        3. YARN-2354-072514.patch
          3 kB
          Li Lu

          Activity

            People

            • Assignee:
              gtcarrera9 Li Lu
              Reporter:
              jianhe Jian He
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: