Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-1011 [Umbrella] Schedule containers based on utilization of currently allocated containers
  3. YARN-8807

FairScheduler crashes RM with oversubscription turned on if an application is killed.

    XMLWordPrintableJSON

    Details

    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      When an application, that has got opportunistic containers allocated, is killed, its containers are not released immediately.

      Fair scheduler would therefore continue to try to promote such orphaned containers, which results in NPE.

      java.lang.NullPointerException
          at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.attemptToAssignReservedResourcesOrPromoteOpportunisticContainers(FairScheduler.java:1158)
          at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.attemptScheduling(FairScheduler.java:1129)
          at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.nodeUpdate(FairScheduler.java:1001)
          at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java:1275)
          at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler.testKillingApplicationWithOpportunisticContainersAssigned(TestFairScheduler.java:4019)

        Attachments

        1. YARN-8807-YARN-1011.01.patch
          9 kB
          Haibo Chen
        2. YARN-8807-YARN-1011.00.patch
          9 kB
          Haibo Chen

          Activity

            People

            • Assignee:
              haibochen Haibo Chen
              Reporter:
              haibochen Haibo Chen
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: