2016-08-11 21:36:39,102 INFO [main] capacity.TestCapacityScheduler (TestCapacityScheduler.java:setupQueueConfiguration(419)) - Setup top-level queues a and b 2016-08-11 21:36:39,200 INFO [main] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2434)) - found resource core-site.xml at file:/opt/hadooptrunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/target/test-classes/core-site.xml 2016-08-11 21:36:39,319 INFO [main] security.Groups (Groups.java:refresh(400)) - clearing userToGroupsMap cache 2016-08-11 21:36:39,417 INFO [main] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2434)) - found resource yarn-site.xml at file:/opt/hadooptrunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/target/test-classes/yarn-site.xml 2016-08-11 21:36:39,638 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(212)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.RMFatalEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMFatalEventDispatcher 2016-08-11 21:36:39,819 INFO [main] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:(75)) - NMTokenKeyRollingInterval: 86400000ms and NMTokenKeyActivationDelay: 900000ms 2016-08-11 21:36:39,837 INFO [main] security.RMContainerTokenSecretManager (RMContainerTokenSecretManager.java:(78)) - ContainerTokenKeyRollingInterval: 86400000ms and ContainerTokenKeyActivationDelay: 900000ms 2016-08-11 21:36:39,851 INFO [main] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:(94)) - AMRMTokenKeyRollingInterval: 86400000ms and AMRMTokenKeyActivationDelay: 900000 ms 2016-08-11 21:36:39,912 INFO [main] security.YarnAuthorizationProvider (YarnAuthorizationProvider.java:getInstance(57)) - org.apache.hadoop.yarn.security.ConfiguredYarnAuthorizer is instiantiated. 2016-08-11 21:36:39,939 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(212)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStoreEventType for class org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$ForwardingEventHandler 2016-08-11 21:36:39,942 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(212)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.NodesListManagerEventType for class org.apache.hadoop.yarn.server.resourcemanager.NodesListManager 2016-08-11 21:36:39,943 INFO [main] resourcemanager.ResourceManager (ResourceManager.java:createScheduler(396)) - Using Scheduler: org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler 2016-08-11 21:36:39,972 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(212)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.SchedulerEventType for class org.apache.hadoop.yarn.event.EventDispatcher 2016-08-11 21:36:39,973 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(212)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationEventDispatcher 2016-08-11 21:36:39,973 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(212)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationAttemptEventDispatcher 2016-08-11 21:36:39,974 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(212)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$NodeEventDispatcher 2016-08-11 21:36:40,014 WARN [main] impl.MetricsConfig (MetricsConfig.java:loadFirst(127)) - Cannot locate configuration: tried hadoop-metrics2-resourcemanager.properties,hadoop-metrics2.properties 2016-08-11 21:36:40,083 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:startTimer(375)) - Scheduled snapshot period at 10 second(s). 2016-08-11 21:36:40,084 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:start(192)) - ResourceManager metrics system started 2016-08-11 21:36:40,110 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(212)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.RMAppManagerEventType for class org.apache.hadoop.yarn.server.resourcemanager.RMAppManager 2016-08-11 21:36:40,118 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(212)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncherEventType for class org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher 2016-08-11 21:36:40,119 INFO [main] resourcemanager.RMNMInfo (RMNMInfo.java:(63)) - Registered RMNMInfo MBean 2016-08-11 21:36:40,121 INFO [main] util.HostsFileReader (HostsFileReader.java:refresh(160)) - Refreshing hosts (include/exclude) list 2016-08-11 21:36:40,138 INFO [main] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2434)) - found resource capacity-scheduler.xml at file:/opt/hadooptrunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/target/test-classes/capacity-scheduler.xml 2016-08-11 21:36:40,193 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(692)) - max alloc mb per queue for root is undefined 2016-08-11 21:36:40,193 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(696)) - max alloc vcore per queue for root is undefined 2016-08-11 21:36:40,203 INFO [main] capacity.ParentQueue (ParentQueue.java:setupQueueConfigs(126)) - root, capacity=1.0, asboluteCapacity=1.0, maxCapacity=1.0, asboluteMaxCapacity=1.0, state=RUNNING, acls=SUBMIT_APP:*ADMINISTER_QUEUE:*, labels=*, , reservationsContinueLooking=true 2016-08-11 21:36:40,204 INFO [main] capacity.ParentQueue (ParentQueue.java:(105)) - Initialized parent-queue root name=root, fullname=root 2016-08-11 21:36:40,206 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(692)) - max alloc mb per queue for root.a is undefined 2016-08-11 21:36:40,206 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(696)) - max alloc vcore per queue for root.a is undefined 2016-08-11 21:36:40,207 INFO [main] capacity.ParentQueue (ParentQueue.java:setupQueueConfigs(126)) - a, capacity=0.105, asboluteCapacity=0.105, maxCapacity=1.0, asboluteMaxCapacity=1.0, state=RUNNING, acls=SUBMIT_APP: ADMINISTER_QUEUE: , labels=*, , reservationsContinueLooking=true 2016-08-11 21:36:40,207 INFO [main] capacity.ParentQueue (ParentQueue.java:(105)) - Initialized parent-queue a name=a, fullname=root.a 2016-08-11 21:36:40,216 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(692)) - max alloc mb per queue for root.a.a1 is undefined 2016-08-11 21:36:40,219 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(696)) - max alloc vcore per queue for root.a.a1 is undefined 2016-08-11 21:36:40,222 INFO [main] capacity.LeafQueue (LeafQueue.java:setupQueueConfigs(219)) - Initializing a1 capacity = 0.3 [= (float) configuredCapacity / 100 ] asboluteCapacity = 0.0315 [= parentAbsoluteCapacity * capacity ] maxCapacity = 1.0 [= configuredMaxCapacity ] absoluteMaxCapacity = 1.0 [= 1.0 maximumCapacity undefined, (parentAbsoluteMaxCapacity * maximumCapacity) / 100 otherwise ] userLimit = 100 [= configuredUserLimit ] userLimitFactor = 100.0 [= configuredUserLimitFactor ] maxApplications = 315 [= configuredMaximumSystemApplicationsPerQueue or (int)(configuredMaximumSystemApplications * absoluteCapacity)] maxApplicationsPerUser = 315 [= (int)(maxApplications * (userLimit / 100.0f) * userLimitFactor) ] usedCapacity = 0.0 [= usedResourcesMemory / (clusterResourceMemory * absoluteCapacity)] absoluteUsedCapacity = 0.0 [= usedResourcesMemory / clusterResourceMemory] maxAMResourcePerQueuePercent = 0.1 [= configuredMaximumAMResourcePercent ] minimumAllocationFactor = 0.875 [= (float)(maximumAllocationMemory - minimumAllocationMemory) / maximumAllocationMemory ] maximumAllocation = [= configuredMaxAllocation ] numContainers = 0 [= currentNumContainers ] state = RUNNING [= configuredState ] acls = SUBMIT_APP: ADMINISTER_QUEUE: [= configuredAcls ] nodeLocalityDelay = 40 labels=*, reservationsContinueLooking = true preemptionDisabled = true defaultAppPriorityPerQueue = 0 2016-08-11 21:36:40,222 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:parseQueue(684)) - Initialized queue: a1: capacity=0.3, absoluteCapacity=0.0315, usedResources=, usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=0, numContainers=0 2016-08-11 21:36:40,225 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(692)) - max alloc mb per queue for root.a.a2 is undefined 2016-08-11 21:36:40,226 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(696)) - max alloc vcore per queue for root.a.a2 is undefined 2016-08-11 21:36:40,228 INFO [main] capacity.LeafQueue (LeafQueue.java:setupQueueConfigs(219)) - Initializing a2 capacity = 0.7 [= (float) configuredCapacity / 100 ] asboluteCapacity = 0.0735 [= parentAbsoluteCapacity * capacity ] maxCapacity = 1.0 [= configuredMaxCapacity ] absoluteMaxCapacity = 1.0 [= 1.0 maximumCapacity undefined, (parentAbsoluteMaxCapacity * maximumCapacity) / 100 otherwise ] userLimit = 100 [= configuredUserLimit ] userLimitFactor = 100.0 [= configuredUserLimitFactor ] maxApplications = 735 [= configuredMaximumSystemApplicationsPerQueue or (int)(configuredMaximumSystemApplications * absoluteCapacity)] maxApplicationsPerUser = 735 [= (int)(maxApplications * (userLimit / 100.0f) * userLimitFactor) ] usedCapacity = 0.0 [= usedResourcesMemory / (clusterResourceMemory * absoluteCapacity)] absoluteUsedCapacity = 0.0 [= usedResourcesMemory / clusterResourceMemory] maxAMResourcePerQueuePercent = 0.1 [= configuredMaximumAMResourcePercent ] minimumAllocationFactor = 0.875 [= (float)(maximumAllocationMemory - minimumAllocationMemory) / maximumAllocationMemory ] maximumAllocation = [= configuredMaxAllocation ] numContainers = 0 [= currentNumContainers ] state = RUNNING [= configuredState ] acls = SUBMIT_APP: ADMINISTER_QUEUE: [= configuredAcls ] nodeLocalityDelay = 40 labels=*, reservationsContinueLooking = true preemptionDisabled = true defaultAppPriorityPerQueue = 0 2016-08-11 21:36:40,228 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:parseQueue(684)) - Initialized queue: a2: capacity=0.7, absoluteCapacity=0.0735, usedResources=, usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=0, numContainers=0 2016-08-11 21:36:40,229 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:parseQueue(684)) - Initialized queue: a: numChildQueue= 2, capacity=0.105, absoluteCapacity=0.105, usedResources=usedCapacity=0.0, numApps=0, numContainers=0 2016-08-11 21:36:40,232 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(692)) - max alloc mb per queue for root.b is undefined 2016-08-11 21:36:40,233 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(696)) - max alloc vcore per queue for root.b is undefined 2016-08-11 21:36:40,233 INFO [main] capacity.ParentQueue (ParentQueue.java:setupQueueConfigs(126)) - b, capacity=0.895, asboluteCapacity=0.895, maxCapacity=1.0, asboluteMaxCapacity=1.0, state=RUNNING, acls=SUBMIT_APP: ADMINISTER_QUEUE: , labels=*, , reservationsContinueLooking=true 2016-08-11 21:36:40,234 INFO [main] capacity.ParentQueue (ParentQueue.java:(105)) - Initialized parent-queue b name=b, fullname=root.b 2016-08-11 21:36:40,237 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(692)) - max alloc mb per queue for root.b.b1 is undefined 2016-08-11 21:36:40,237 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(696)) - max alloc vcore per queue for root.b.b1 is undefined 2016-08-11 21:36:40,239 INFO [main] capacity.LeafQueue (LeafQueue.java:setupQueueConfigs(219)) - Initializing b1 capacity = 0.792 [= (float) configuredCapacity / 100 ] asboluteCapacity = 0.70883995 [= parentAbsoluteCapacity * capacity ] maxCapacity = 1.0 [= configuredMaxCapacity ] absoluteMaxCapacity = 1.0 [= 1.0 maximumCapacity undefined, (parentAbsoluteMaxCapacity * maximumCapacity) / 100 otherwise ] userLimit = 100 [= configuredUserLimit ] userLimitFactor = 100.0 [= configuredUserLimitFactor ] maxApplications = 7088 [= configuredMaximumSystemApplicationsPerQueue or (int)(configuredMaximumSystemApplications * absoluteCapacity)] maxApplicationsPerUser = 7088 [= (int)(maxApplications * (userLimit / 100.0f) * userLimitFactor) ] usedCapacity = 0.0 [= usedResourcesMemory / (clusterResourceMemory * absoluteCapacity)] absoluteUsedCapacity = 0.0 [= usedResourcesMemory / clusterResourceMemory] maxAMResourcePerQueuePercent = 0.1 [= configuredMaximumAMResourcePercent ] minimumAllocationFactor = 0.875 [= (float)(maximumAllocationMemory - minimumAllocationMemory) / maximumAllocationMemory ] maximumAllocation = [= configuredMaxAllocation ] numContainers = 0 [= currentNumContainers ] state = RUNNING [= configuredState ] acls = SUBMIT_APP: ADMINISTER_QUEUE: [= configuredAcls ] nodeLocalityDelay = 40 labels=*, reservationsContinueLooking = true preemptionDisabled = true defaultAppPriorityPerQueue = 0 2016-08-11 21:36:40,239 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:parseQueue(684)) - Initialized queue: b1: capacity=0.792, absoluteCapacity=0.70883995, usedResources=, usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=0, numContainers=0 2016-08-11 21:36:40,242 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(692)) - max alloc mb per queue for root.b.b2 is undefined 2016-08-11 21:36:40,242 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(696)) - max alloc vcore per queue for root.b.b2 is undefined 2016-08-11 21:36:40,243 INFO [main] capacity.LeafQueue (LeafQueue.java:setupQueueConfigs(219)) - Initializing b2 capacity = 0.008 [= (float) configuredCapacity / 100 ] asboluteCapacity = 0.00716 [= parentAbsoluteCapacity * capacity ] maxCapacity = 1.0 [= configuredMaxCapacity ] absoluteMaxCapacity = 1.0 [= 1.0 maximumCapacity undefined, (parentAbsoluteMaxCapacity * maximumCapacity) / 100 otherwise ] userLimit = 100 [= configuredUserLimit ] userLimitFactor = 100.0 [= configuredUserLimitFactor ] maxApplications = 71 [= configuredMaximumSystemApplicationsPerQueue or (int)(configuredMaximumSystemApplications * absoluteCapacity)] maxApplicationsPerUser = 71 [= (int)(maxApplications * (userLimit / 100.0f) * userLimitFactor) ] usedCapacity = 0.0 [= usedResourcesMemory / (clusterResourceMemory * absoluteCapacity)] absoluteUsedCapacity = 0.0 [= usedResourcesMemory / clusterResourceMemory] maxAMResourcePerQueuePercent = 0.1 [= configuredMaximumAMResourcePercent ] minimumAllocationFactor = 0.875 [= (float)(maximumAllocationMemory - minimumAllocationMemory) / maximumAllocationMemory ] maximumAllocation = [= configuredMaxAllocation ] numContainers = 0 [= currentNumContainers ] state = RUNNING [= configuredState ] acls = SUBMIT_APP: ADMINISTER_QUEUE: [= configuredAcls ] nodeLocalityDelay = 40 labels=*, reservationsContinueLooking = true preemptionDisabled = true defaultAppPriorityPerQueue = 0 2016-08-11 21:36:40,243 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:parseQueue(684)) - Initialized queue: b2: capacity=0.008, absoluteCapacity=0.00716, usedResources=, usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=0, numContainers=0 2016-08-11 21:36:40,246 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(692)) - max alloc mb per queue for root.b.b3 is undefined 2016-08-11 21:36:40,246 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(696)) - max alloc vcore per queue for root.b.b3 is undefined 2016-08-11 21:36:40,247 INFO [main] capacity.LeafQueue (LeafQueue.java:setupQueueConfigs(219)) - Initializing b3 capacity = 0.2 [= (float) configuredCapacity / 100 ] asboluteCapacity = 0.179 [= parentAbsoluteCapacity * capacity ] maxCapacity = 1.0 [= configuredMaxCapacity ] absoluteMaxCapacity = 1.0 [= 1.0 maximumCapacity undefined, (parentAbsoluteMaxCapacity * maximumCapacity) / 100 otherwise ] userLimit = 100 [= configuredUserLimit ] userLimitFactor = 100.0 [= configuredUserLimitFactor ] maxApplications = 1790 [= configuredMaximumSystemApplicationsPerQueue or (int)(configuredMaximumSystemApplications * absoluteCapacity)] maxApplicationsPerUser = 1790 [= (int)(maxApplications * (userLimit / 100.0f) * userLimitFactor) ] usedCapacity = 0.0 [= usedResourcesMemory / (clusterResourceMemory * absoluteCapacity)] absoluteUsedCapacity = 0.0 [= usedResourcesMemory / clusterResourceMemory] maxAMResourcePerQueuePercent = 0.1 [= configuredMaximumAMResourcePercent ] minimumAllocationFactor = 0.875 [= (float)(maximumAllocationMemory - minimumAllocationMemory) / maximumAllocationMemory ] maximumAllocation = [= configuredMaxAllocation ] numContainers = 0 [= currentNumContainers ] state = RUNNING [= configuredState ] acls = SUBMIT_APP: ADMINISTER_QUEUE: [= configuredAcls ] nodeLocalityDelay = 40 labels=*, reservationsContinueLooking = true preemptionDisabled = true defaultAppPriorityPerQueue = 0 2016-08-11 21:36:40,247 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:parseQueue(684)) - Initialized queue: b3: capacity=0.2, absoluteCapacity=0.179, usedResources=, usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=0, numContainers=0 2016-08-11 21:36:40,248 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:parseQueue(684)) - Initialized queue: b: numChildQueue= 3, capacity=0.895, absoluteCapacity=0.895, usedResources=usedCapacity=0.0, numApps=0, numContainers=0 2016-08-11 21:36:40,248 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:parseQueue(684)) - Initialized queue: root: numChildQueue= 2, capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.0, numApps=0, numContainers=0 2016-08-11 21:36:40,249 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:initializeQueues(519)) - Initialized root queue root: numChildQueue= 2, capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.0, numApps=0, numContainers=0 2016-08-11 21:36:40,250 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:getUserGroupMappingPlacementRule(467)) - Initialized queue mappings, override: false 2016-08-11 21:36:40,252 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:initScheduler(328)) - Initialized CapacityScheduler with calculator=class org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator, minimumAllocation=<>, maximumAllocation=<>, asynchronousScheduling=false, asyncScheduleInterval=5ms 2016-08-11 21:36:40,259 INFO [main] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2431)) - dynamic-resources.xml not found 2016-08-11 21:36:40,290 INFO [main] resourcemanager.ResourceManager (ResourceManager.java:createSystemMetricsPublisher(486)) - TimelineServicePublisher is not configured 2016-08-11 21:36:40,294 INFO [main] security.RMContainerTokenSecretManager (RMContainerTokenSecretManager.java:rollMasterKey(106)) - Rolling master-key for container-tokens 2016-08-11 21:36:40,318 INFO [main] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:rollMasterKey(95)) - Rolling master-key for nm-tokens 2016-08-11 21:36:40,447 INFO [main] capacity.TestCapacityScheduler (TestCapacityScheduler.java:setupBlockedQueueConfiguration(436)) - Setup top-level queues a and b 2016-08-11 21:36:40,477 INFO [main] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2434)) - found resource core-site.xml at file:/opt/hadooptrunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/target/test-classes/core-site.xml 2016-08-11 21:36:40,499 INFO [main] security.Groups (Groups.java:refresh(400)) - clearing userToGroupsMap cache 2016-08-11 21:36:40,500 INFO [main] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2434)) - found resource yarn-site.xml at file:/opt/hadooptrunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/target/test-classes/yarn-site.xml 2016-08-11 21:36:40,539 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(212)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.RMFatalEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMFatalEventDispatcher 2016-08-11 21:36:40,540 INFO [main] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:(75)) - NMTokenKeyRollingInterval: 86400000ms and NMTokenKeyActivationDelay: 900000ms 2016-08-11 21:36:40,541 INFO [main] security.RMContainerTokenSecretManager (RMContainerTokenSecretManager.java:(78)) - ContainerTokenKeyRollingInterval: 86400000ms and ContainerTokenKeyActivationDelay: 900000ms 2016-08-11 21:36:40,541 INFO [main] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:(94)) - AMRMTokenKeyRollingInterval: 86400000ms and AMRMTokenKeyActivationDelay: 900000 ms 2016-08-11 21:36:40,543 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(212)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStoreEventType for class org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$ForwardingEventHandler 2016-08-11 21:36:40,544 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(212)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.NodesListManagerEventType for class org.apache.hadoop.yarn.server.resourcemanager.NodesListManager 2016-08-11 21:36:40,544 INFO [main] resourcemanager.ResourceManager (ResourceManager.java:createScheduler(396)) - Using Scheduler: org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler 2016-08-11 21:36:40,544 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(212)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.SchedulerEventType for class org.apache.hadoop.yarn.event.EventDispatcher 2016-08-11 21:36:40,545 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(212)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationEventDispatcher 2016-08-11 21:36:40,545 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(212)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationAttemptEventDispatcher 2016-08-11 21:36:40,545 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(212)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$NodeEventDispatcher 2016-08-11 21:36:40,545 INFO [main] security.RMContainerTokenSecretManager (RMContainerTokenSecretManager.java:rollMasterKey(106)) - Rolling master-key for container-tokens 2016-08-11 21:36:40,546 INFO [main] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:rollMasterKey(95)) - Rolling master-key for nm-tokens 2016-08-11 21:36:40,546 WARN [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:init(152)) - ResourceManager metrics system already initialized! 2016-08-11 21:36:40,547 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(212)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.RMAppManagerEventType for class org.apache.hadoop.yarn.server.resourcemanager.RMAppManager 2016-08-11 21:36:40,547 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(212)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncherEventType for class org.apache.hadoop.yarn.server.resourcemanager.MockRM$6 2016-08-11 21:36:40,548 WARN [main] util.MBeans (MBeans.java:getMBeanName(125)) - Error creating MBean object name: Hadoop:service=ResourceManager,name=RMNMInfo org.apache.hadoop.metrics2.MetricsException: org.apache.hadoop.metrics2.MetricsException: Hadoop:service=ResourceManager,name=RMNMInfo already exists! at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.newObjectName(DefaultMetricsSystem.java:135) at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.newMBeanName(DefaultMetricsSystem.java:110) at org.apache.hadoop.metrics2.util.MBeans.getMBeanName(MBeans.java:123) at org.apache.hadoop.metrics2.util.MBeans.register(MBeans.java:64) at org.apache.hadoop.yarn.server.resourcemanager.RMNMInfo.(RMNMInfo.java:59) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceInit(ResourceManager.java:692) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAndInitActiveServices(ResourceManager.java:1037) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:294) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.yarn.server.resourcemanager.MockRM.(MockRM.java:135) at org.apache.hadoop.yarn.server.resourcemanager.MockRM.(MockRM.java:128) at org.apache.hadoop.yarn.server.resourcemanager.MockRM.(MockRM.java:124) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler.testCSQueueBlocked(TestCapacityScheduler.java:3594) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229) at org.junit.runners.ParentRunner.run(ParentRunner.java:309) at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:254) at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:149) at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:124) at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:200) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:153) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103) Caused by: org.apache.hadoop.metrics2.MetricsException: Hadoop:service=ResourceManager,name=RMNMInfo already exists! at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.newObjectName(DefaultMetricsSystem.java:131) ... 38 more 2016-08-11 21:36:40,552 INFO [main] resourcemanager.RMNMInfo (RMNMInfo.java:(63)) - Registered RMNMInfo MBean 2016-08-11 21:36:40,552 INFO [main] util.HostsFileReader (HostsFileReader.java:refresh(160)) - Refreshing hosts (include/exclude) list 2016-08-11 21:36:40,554 INFO [main] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2434)) - found resource capacity-scheduler.xml at file:/opt/hadooptrunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/target/test-classes/capacity-scheduler.xml 2016-08-11 21:36:40,581 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(692)) - max alloc mb per queue for root is undefined 2016-08-11 21:36:40,581 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(696)) - max alloc vcore per queue for root is undefined 2016-08-11 21:36:40,582 INFO [main] capacity.ParentQueue (ParentQueue.java:setupQueueConfigs(126)) - root, capacity=1.0, asboluteCapacity=1.0, maxCapacity=1.0, asboluteMaxCapacity=1.0, state=RUNNING, acls=SUBMIT_APP:*ADMINISTER_QUEUE:*, labels=*, , reservationsContinueLooking=true 2016-08-11 21:36:40,582 INFO [main] capacity.ParentQueue (ParentQueue.java:(105)) - Initialized parent-queue root name=root, fullname=root 2016-08-11 21:36:40,584 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(692)) - max alloc mb per queue for root.a is undefined 2016-08-11 21:36:40,584 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(696)) - max alloc vcore per queue for root.a is undefined 2016-08-11 21:36:40,586 INFO [main] capacity.LeafQueue (LeafQueue.java:setupQueueConfigs(219)) - Initializing a capacity = 0.8 [= (float) configuredCapacity / 100 ] asboluteCapacity = 0.8 [= parentAbsoluteCapacity * capacity ] maxCapacity = 1.0 [= configuredMaxCapacity ] absoluteMaxCapacity = 1.0 [= 1.0 maximumCapacity undefined, (parentAbsoluteMaxCapacity * maximumCapacity) / 100 otherwise ] userLimit = 100 [= configuredUserLimit ] userLimitFactor = 100.0 [= configuredUserLimitFactor ] maxApplications = 8000 [= configuredMaximumSystemApplicationsPerQueue or (int)(configuredMaximumSystemApplications * absoluteCapacity)] maxApplicationsPerUser = 8000 [= (int)(maxApplications * (userLimit / 100.0f) * userLimitFactor) ] usedCapacity = 0.0 [= usedResourcesMemory / (clusterResourceMemory * absoluteCapacity)] absoluteUsedCapacity = 0.0 [= usedResourcesMemory / clusterResourceMemory] maxAMResourcePerQueuePercent = 0.1 [= configuredMaximumAMResourcePercent ] minimumAllocationFactor = 0.875 [= (float)(maximumAllocationMemory - minimumAllocationMemory) / maximumAllocationMemory ] maximumAllocation = [= configuredMaxAllocation ] numContainers = 0 [= currentNumContainers ] state = RUNNING [= configuredState ] acls = SUBMIT_APP: ADMINISTER_QUEUE: [= configuredAcls ] nodeLocalityDelay = 40 labels=*, reservationsContinueLooking = true preemptionDisabled = true defaultAppPriorityPerQueue = 0 2016-08-11 21:36:40,586 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:parseQueue(684)) - Initialized queue: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=0, numContainers=0 2016-08-11 21:36:40,588 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(692)) - max alloc mb per queue for root.b is undefined 2016-08-11 21:36:40,588 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(696)) - max alloc vcore per queue for root.b is undefined 2016-08-11 21:36:40,590 INFO [main] capacity.LeafQueue (LeafQueue.java:setupQueueConfigs(219)) - Initializing b capacity = 0.2 [= (float) configuredCapacity / 100 ] asboluteCapacity = 0.2 [= parentAbsoluteCapacity * capacity ] maxCapacity = 1.0 [= configuredMaxCapacity ] absoluteMaxCapacity = 1.0 [= 1.0 maximumCapacity undefined, (parentAbsoluteMaxCapacity * maximumCapacity) / 100 otherwise ] userLimit = 100 [= configuredUserLimit ] userLimitFactor = 100.0 [= configuredUserLimitFactor ] maxApplications = 2000 [= configuredMaximumSystemApplicationsPerQueue or (int)(configuredMaximumSystemApplications * absoluteCapacity)] maxApplicationsPerUser = 2000 [= (int)(maxApplications * (userLimit / 100.0f) * userLimitFactor) ] usedCapacity = 0.0 [= usedResourcesMemory / (clusterResourceMemory * absoluteCapacity)] absoluteUsedCapacity = 0.0 [= usedResourcesMemory / clusterResourceMemory] maxAMResourcePerQueuePercent = 0.1 [= configuredMaximumAMResourcePercent ] minimumAllocationFactor = 0.875 [= (float)(maximumAllocationMemory - minimumAllocationMemory) / maximumAllocationMemory ] maximumAllocation = [= configuredMaxAllocation ] numContainers = 0 [= currentNumContainers ] state = RUNNING [= configuredState ] acls = SUBMIT_APP: ADMINISTER_QUEUE: [= configuredAcls ] nodeLocalityDelay = 40 labels=*, reservationsContinueLooking = true preemptionDisabled = true defaultAppPriorityPerQueue = 0 2016-08-11 21:36:40,590 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:parseQueue(684)) - Initialized queue: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=0, numContainers=0 2016-08-11 21:36:40,590 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:parseQueue(684)) - Initialized queue: root: numChildQueue= 2, capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.0, numApps=0, numContainers=0 2016-08-11 21:36:40,591 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:initializeQueues(519)) - Initialized root queue root: numChildQueue= 2, capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.0, numApps=0, numContainers=0 2016-08-11 21:36:40,591 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:getUserGroupMappingPlacementRule(467)) - Initialized queue mappings, override: false 2016-08-11 21:36:40,592 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:initScheduler(328)) - Initialized CapacityScheduler with calculator=class org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator, minimumAllocation=<>, maximumAllocation=<>, asynchronousScheduling=false, asyncScheduleInterval=5ms 2016-08-11 21:36:40,593 INFO [main] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2431)) - dynamic-resources.xml not found 2016-08-11 21:36:40,594 INFO [main] resourcemanager.ResourceManager (ResourceManager.java:createSystemMetricsPublisher(486)) - TimelineServicePublisher is not configured 2016-08-11 21:36:40,595 INFO [main] resourcemanager.ResourceManager (ResourceManager.java:transitionToActive(1081)) - Transitioning to active state 2016-08-11 21:36:40,596 DEBUG [main] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1817)) - PrivilegedAction as:root (auth:SIMPLE) from:org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.transitionToActive(ResourceManager.java:1083) 2016-08-11 21:36:40,601 DEBUG [main] service.AbstractService (AbstractService.java:start(197)) - Service Dispatcher is started 2016-08-11 21:36:40,602 DEBUG [main] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore is started 2016-08-11 21:36:40,602 DEBUG [main] service.CompositeService (CompositeService.java:serviceStart(115)) - RMActiveServices: starting services, size=14 2016-08-11 21:36:40,608 DEBUG [main] recovery.RMStateStore (RMStateStore.java:handleStoreEvent(1070)) - Processing event of type UPDATE_AMRM_TOKEN 2016-08-11 21:36:40,608 INFO [main] recovery.RMStateStore (RMStateStore.java:transition(487)) - Updating AMRMToken 2016-08-11 21:36:40,609 INFO [main] security.RMContainerTokenSecretManager (RMContainerTokenSecretManager.java:rollMasterKey(106)) - Rolling master-key for container-tokens 2016-08-11 21:36:40,609 INFO [main] security.RMContainerTokenSecretManager (RMContainerTokenSecretManager.java:rollMasterKey(111)) - Going to activate master-key with key-id -1668377322 in 900000ms 2016-08-11 21:36:40,609 INFO [main] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:rollMasterKey(95)) - Rolling master-key for nm-tokens 2016-08-11 21:36:40,610 INFO [main] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:rollMasterKey(100)) - Going to activate master-key with key-id 1657106355 in 900000ms 2016-08-11 21:36:40,610 INFO [main] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2016-08-11 21:36:40,623 INFO [main] security.RMDelegationTokenSecretManager (RMDelegationTokenSecretManager.java:storeNewMasterKey(87)) - storing master key with keyID 1 2016-08-11 21:36:40,624 DEBUG [main] recovery.RMStateStore (RMStateStore.java:handleStoreEvent(1070)) - Processing event of type STORE_MASTERKEY 2016-08-11 21:36:40,624 INFO [main] recovery.RMStateStore (RMStateStore.java:transition(438)) - Storing RMDTMasterKey. 2016-08-11 21:36:40,626 DEBUG [main] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.RMSecretManagerService is started 2016-08-11 21:36:40,627 DEBUG [main] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.ContainerAllocationExpirer is started 2016-08-11 21:36:40,627 INFO [Thread[Thread-18,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2016-08-11 21:36:40,628 INFO [Thread[Thread-18,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2016-08-11 21:36:40,628 INFO [Thread[Thread-18,5,main]] security.RMDelegationTokenSecretManager (RMDelegationTokenSecretManager.java:storeNewMasterKey(87)) - storing master key with keyID 2 2016-08-11 21:36:40,629 DEBUG [Thread[Thread-18,5,main]] recovery.RMStateStore (RMStateStore.java:handleStoreEvent(1070)) - Processing event of type STORE_MASTERKEY 2016-08-11 21:36:40,631 DEBUG [main] service.AbstractService (AbstractService.java:start(197)) - Service AMLivelinessMonitor is started 2016-08-11 21:36:40,631 INFO [Thread[Thread-18,5,main]] recovery.RMStateStore (RMStateStore.java:transition(438)) - Storing RMDTMasterKey. 2016-08-11 21:36:40,631 DEBUG [main] service.AbstractService (AbstractService.java:start(197)) - Service AMLivelinessMonitor is started 2016-08-11 21:36:40,632 DEBUG [main] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.nodelabels.CommonNodeLabelsManager is started 2016-08-11 21:36:40,632 DEBUG [main] service.CompositeService (CompositeService.java:serviceStart(115)) - org.apache.hadoop.yarn.server.resourcemanager.NodesListManager: starting services, size=0 2016-08-11 21:36:40,632 DEBUG [main] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.NodesListManager is started 2016-08-11 21:36:40,633 DEBUG [main] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.scheduler.activities.ActivitiesManager is started 2016-08-11 21:36:40,633 DEBUG [main] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler is started 2016-08-11 21:36:40,633 DEBUG [main] service.AbstractService (AbstractService.java:start(197)) - Service SchedulerEventDispatcher is started 2016-08-11 21:36:40,634 DEBUG [main] service.AbstractService (AbstractService.java:start(197)) - Service NMLivelinessMonitor is started 2016-08-11 21:36:40,634 DEBUG [main] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.ResourceTrackerService is started 2016-08-11 21:36:40,635 DEBUG [main] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.util.JvmPauseMonitor is started 2016-08-11 21:36:40,635 DEBUG [main] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService is started 2016-08-11 21:36:40,635 DEBUG [main] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.ClientRMService is started 2016-08-11 21:36:40,636 DEBUG [main] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher is started 2016-08-11 21:36:40,636 DEBUG [main] service.AbstractService (AbstractService.java:start(197)) - Service RMActiveServices is started 2016-08-11 21:36:40,636 INFO [main] resourcemanager.ResourceManager (ResourceManager.java:transitionToActive(1097)) - Transitioned to active state 2016-08-11 21:36:40,636 DEBUG [main] service.CompositeService (CompositeService.java:serviceStart(115)) - ResourceManager: starting services, size=3 2016-08-11 21:36:40,640 INFO [org.apache.hadoop.util.JvmPauseMonitor$Monitor@6bc407fd] util.JvmPauseMonitor (JvmPauseMonitor.java:run(188)) - Starting JVM pause monitor 2016-08-11 21:36:40,640 DEBUG [main] service.AbstractService (AbstractService.java:start(197)) - Service Dispatcher is started 2016-08-11 21:36:40,643 DEBUG [main] service.CompositeService (CompositeService.java:serviceStart(115)) - org.apache.hadoop.yarn.server.resourcemanager.AdminService: starting services, size=0 2016-08-11 21:36:40,644 DEBUG [main] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.AdminService is started 2016-08-11 21:36:40,644 DEBUG [main] service.CompositeService (CompositeService.java:serviceStart(115)) - org.apache.hadoop.yarn.server.resourcemanager.ahs.RMApplicationHistoryWriter: starting services, size=0 2016-08-11 21:36:40,644 DEBUG [main] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.ahs.RMApplicationHistoryWriter is started 2016-08-11 21:36:40,644 DEBUG [main] service.AbstractService (AbstractService.java:start(197)) - Service ResourceManager is started 2016-08-11 21:36:40,691 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeLabelsUpdateSchedulerEvent.EventType: NODE_LABELS_UPDATE 2016-08-11 21:36:40,693 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:addNode(1515)) - Added node 127.0.0.1:1 clusterResource: 2016-08-11 21:36:40,693 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeLabelsUpdateSchedulerEvent.EventType: NODE_LABELS_UPDATE 2016-08-11 21:36:40,694 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:addNode(1515)) - Added node test:2 clusterResource: 2016-08-11 21:36:40,842 INFO [main] capacity.ParentQueue (ParentQueue.java:addApplication(350)) - Application added - appId: application_100_0001 user: user1 leaf-queue of parent: root #applications: 1 2016-08-11 21:36:40,842 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:addApplication(792)) - Accepted application application_100_0001 from user: user1, in queue: a 2016-08-11 21:36:40,843 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEvent.EventType: APP_ACCEPTED 2016-08-11 21:36:40,876 DEBUG [main] capacity.LeafQueue (LeafQueue.java:activateApplications(648)) - application application_100_0001 AMResource maxAMResourcePerQueuePercent 0.1 amLimit lastClusterResource amIfStarted AM node-partition name 2016-08-11 21:36:40,878 INFO [main] capacity.LeafQueue (LeafQueue.java:activateApplications(722)) - Application application_100_0001 from user: user1 activated in queue: a 2016-08-11 21:36:40,878 INFO [main] capacity.LeafQueue (LeafQueue.java:addApplicationAttempt(737)) - Application added - appId: application_100_0001 user: user1, leaf-queue: a #user-pending-applications: 0 #user-active-applications: 1 #queue-pending-applications: 0 #queue-active-applications: 1 2016-08-11 21:36:40,878 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:addApplicationAttempt(832)) - Added Application Attempt appattempt_100_0001_000001 to scheduler from user user1 in queue a 2016-08-11 21:36:40,879 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: ATTEMPT_ADDED 2016-08-11 21:36:40,881 INFO [main] capacity.ParentQueue (ParentQueue.java:addApplication(350)) - Application added - appId: application_100_0002 user: user2 leaf-queue of parent: root #applications: 2 2016-08-11 21:36:40,881 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:addApplication(792)) - Accepted application application_100_0002 from user: user2, in queue: b 2016-08-11 21:36:40,881 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEvent.EventType: APP_ACCEPTED 2016-08-11 21:36:40,883 DEBUG [main] capacity.LeafQueue (LeafQueue.java:activateApplications(648)) - application application_100_0002 AMResource maxAMResourcePerQueuePercent 0.1 amLimit lastClusterResource amIfStarted AM node-partition name 2016-08-11 21:36:40,884 INFO [main] capacity.LeafQueue (LeafQueue.java:activateApplications(722)) - Application application_100_0002 from user: user2 activated in queue: b 2016-08-11 21:36:40,884 INFO [main] capacity.LeafQueue (LeafQueue.java:addApplicationAttempt(737)) - Application added - appId: application_100_0002 user: user2, leaf-queue: b #user-pending-applications: 0 #user-active-applications: 1 #queue-pending-applications: 0 #queue-active-applications: 1 2016-08-11 21:36:40,884 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:addApplicationAttempt(832)) - Added Application Attempt appattempt_100_0002_000002 to scheduler from user user2 in queue b 2016-08-11 21:36:40,884 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: ATTEMPT_ADDED 2016-08-11 21:36:40,920 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(991)) - allocate: pre-update appattempt_100_0001_000001 ask size =1 2016-08-11 21:36:40,921 DEBUG [main] scheduler.ActiveUsersManager (ActiveUsersManager.java:activateApplication(68)) - User user1 added to activeUsers, currently: 1 2016-08-11 21:36:40,922 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(1003)) - allocate: post-update 2016-08-11 21:36:40,922 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0001 headRoom= currentConsumption=0 2016-08-11 21:36:40,922 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0001 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:36:40,929 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1297)) - Trying to schedule on node: 127.0.0.1, available: 2016-08-11 21:36:40,930 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(437)) - Trying to assign containers to child-queue of root 2016-08-11 21:36:40,931 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.0), label=(*)root.busedCapacity=(0.0), label=(*) 2016-08-11 21:36:40,932 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=1, numContainers=0 2016-08-11 21:36:40,932 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=127.0.0.1 #applications=1 2016-08-11 21:36:40,933 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user1 in queue a userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:36:40,933 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimitAndSetHeadroom(1120)) - Headroom calculation for user user1: userLimit= queueMaxAvailRes= consumed= headroom= 2016-08-11 21:36:40,934 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:assignContainers(511)) - pre-assignContainers for application application_100_0001 2016-08-11 21:36:40,934 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user1 in queue a userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:36:40,934 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0001 headRoom= currentConsumption=0 2016-08-11 21:36:40,935 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0001 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:36:40,935 DEBUG [main] allocator.IncreaseContainerAllocator (IncreaseContainerAllocator.java:assignContainers(187)) - Skip allocating increase request since we don't have any increase request on this node=127.0.0.1:123 2016-08-11 21:36:40,937 DEBUG [main] allocator.RegularContainerAllocator (RegularContainerAllocator.java:assignContainer(473)) - assignContainers: node=127.0.0.1 application=application_100_0001 priority=1 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } type=OFF_SWITCH 2016-08-11 21:36:40,974 DEBUG [main] scheduler.ActiveUsersManager (ActiveUsersManager.java:deactivateApplication(94)) - User user1 removed from activeUsers, currently: 0 2016-08-11 21:36:40,975 DEBUG [main] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:allocate(631)) - allocate: applicationId=application_100_0001 container=container_100_0001_01_000001 host=127.0.0.1:123 user=user1 resource= type=OFF_SWITCH 2016-08-11 21:36:40,975 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0001_01_000001 of type START 2016-08-11 21:36:40,976 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: CONTAINER_ALLOCATED 2016-08-11 21:36:40,976 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0001_01_000001 Container Transitioned from NEW to ALLOCATED 2016-08-11 21:36:40,976 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:allocate(237)) - allocate: applicationAttemptId=appattempt_100_0001_000001 container=container_100_0001_01_000001 host=127.0.0.1 type=OFF_SWITCH 2016-08-11 21:36:40,986 INFO [main] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(159)) - USER=user1 OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_100_0001 CONTAINERID=container_100_0001_01_000001 RESOURCE= 2016-08-11 21:36:40,986 DEBUG [main] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(157)) - Assigned container container_100_0001_01_000001 of capacity on host 127.0.0.1:1, which has 1 containers, used and available after allocation 2016-08-11 21:36:40,987 INFO [main] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(120)) - assignedContainer application attempt=appattempt_100_0001_000001 container=container_100_0001_01_000001 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@22356acd clusterResource= type=OFF_SWITCH 2016-08-11 21:36:40,989 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(962)) - post-assignContainers for application application_100_0001 2016-08-11 21:36:40,990 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user1 in queue a userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:36:40,990 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0001 headRoom= currentConsumption=2048 2016-08-11 21:36:40,990 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0001 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 0, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:36:40,991 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user1 in queue a userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:36:40,991 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user1 in queue a userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:36:40,992 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user1 in queue a userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:36:40,992 DEBUG [main] capacity.LeafQueue (LeafQueue.java:allocateResource(1443)) - a user=user1 used= numContainers=1 headroom = user-resources= 2016-08-11 21:36:40,993 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 --> , OFF_SWITCH 2016-08-11 21:36:40,993 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(670)) - Re-sorting assigned queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 2016-08-11 21:36:40,994 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.busedCapacity=(0.0), label=(*)root.ausedCapacity=(0.15625238), label=(*) 2016-08-11 21:36:40,995 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainers(522)) - assignedContainer queue=root usedCapacity=0.125 absoluteUsedCapacity=0.125 used= cluster= 2016-08-11 21:36:40,996 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(544)) - ParentQ=root assignedSoFarInThisIteration= usedCapacity=0.125 absoluteUsedCapacity=0.125 2016-08-11 21:36:40,996 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(555)) - Not assigning more than one off-switch container, assignments so far: resource:; type:OFF_SWITCH; excessReservation:null; applicationid:null; skipped:NONE; fulfilled reservation:false; allocations(count/resource):1/; reservations(count/resource):0/ 2016-08-11 21:36:40,996 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1297)) - Trying to schedule on node: test, available: 2016-08-11 21:36:40,996 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(416)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 21:36:40,997 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1297)) - Trying to schedule on node: 127.0.0.1, available: 2016-08-11 21:36:40,997 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(416)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 21:36:40,997 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1297)) - Trying to schedule on node: test, available: 2016-08-11 21:36:40,997 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(416)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 21:36:41,003 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(991)) - allocate: pre-update appattempt_100_0002_000002 ask size =1 2016-08-11 21:36:41,003 DEBUG [main] scheduler.ActiveUsersManager (ActiveUsersManager.java:activateApplication(68)) - User user2 added to activeUsers, currently: 1 2016-08-11 21:36:41,003 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(1003)) - allocate: post-update 2016-08-11 21:36:41,004 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=0 2016-08-11 21:36:41,004 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:36:41,005 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1297)) - Trying to schedule on node: 127.0.0.1, available: 2016-08-11 21:36:41,005 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(437)) - Trying to assign containers to child-queue of root 2016-08-11 21:36:41,005 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.busedCapacity=(0.0), label=(*)root.ausedCapacity=(0.15625238), label=(*) 2016-08-11 21:36:41,005 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=1, numContainers=0 2016-08-11 21:36:41,006 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=127.0.0.1 #applications=1 2016-08-11 21:36:41,006 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:36:41,006 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimitAndSetHeadroom(1120)) - Headroom calculation for user user2: userLimit= queueMaxAvailRes= consumed= headroom= 2016-08-11 21:36:41,006 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:assignContainers(511)) - pre-assignContainers for application application_100_0002 2016-08-11 21:36:41,007 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:36:41,007 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=0 2016-08-11 21:36:41,007 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:36:41,007 DEBUG [main] allocator.IncreaseContainerAllocator (IncreaseContainerAllocator.java:assignContainers(187)) - Skip allocating increase request since we don't have any increase request on this node=127.0.0.1:123 2016-08-11 21:36:41,008 DEBUG [main] allocator.RegularContainerAllocator (RegularContainerAllocator.java:assignContainer(473)) - assignContainers: node=127.0.0.1 application=application_100_0002 priority=1 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } type=OFF_SWITCH 2016-08-11 21:36:41,008 DEBUG [main] scheduler.ActiveUsersManager (ActiveUsersManager.java:deactivateApplication(94)) - User user2 removed from activeUsers, currently: 0 2016-08-11 21:36:41,008 DEBUG [main] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:allocate(631)) - allocate: applicationId=application_100_0002 container=container_100_0002_02_000001 host=127.0.0.1:123 user=user2 resource= type=OFF_SWITCH 2016-08-11 21:36:41,009 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0002_02_000001 of type START 2016-08-11 21:36:41,009 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: CONTAINER_ALLOCATED 2016-08-11 21:36:41,009 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0002_02_000001 Container Transitioned from NEW to ALLOCATED 2016-08-11 21:36:41,009 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:allocate(237)) - allocate: applicationAttemptId=appattempt_100_0002_000002 container=container_100_0002_02_000001 host=127.0.0.1 type=OFF_SWITCH 2016-08-11 21:36:41,010 INFO [main] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(159)) - USER=user2 OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_100_0002 CONTAINERID=container_100_0002_02_000001 RESOURCE= 2016-08-11 21:36:41,010 DEBUG [main] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(157)) - Assigned container container_100_0002_02_000001 of capacity on host 127.0.0.1:1, which has 2 containers, used and available after allocation 2016-08-11 21:36:41,011 INFO [main] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(120)) - assignedContainer application attempt=appattempt_100_0002_000002 container=container_100_0002_02_000001 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@386f0da3 clusterResource= type=OFF_SWITCH 2016-08-11 21:36:41,011 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(962)) - post-assignContainers for application application_100_0002 2016-08-11 21:36:41,011 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:36:41,011 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=1024 2016-08-11 21:36:41,012 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 0, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:36:41,012 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:36:41,012 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:36:41,012 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:36:41,013 DEBUG [main] capacity.LeafQueue (LeafQueue.java:allocateResource(1443)) - b user=user2 used= numContainers=1 headroom = user-resources= 2016-08-11 21:36:41,013 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=0.31257632, absoluteUsedCapacity=0.0625, numApps=1, numContainers=1 --> , OFF_SWITCH 2016-08-11 21:36:41,013 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(670)) - Re-sorting assigned queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=0.31257632, absoluteUsedCapacity=0.0625, numApps=1, numContainers=1 2016-08-11 21:36:41,013 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.15625238), label=(*)root.busedCapacity=(0.31257632), label=(*) 2016-08-11 21:36:41,014 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainers(522)) - assignedContainer queue=root usedCapacity=0.1875 absoluteUsedCapacity=0.1875 used= cluster= 2016-08-11 21:36:41,014 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(544)) - ParentQ=root assignedSoFarInThisIteration= usedCapacity=0.1875 absoluteUsedCapacity=0.1875 2016-08-11 21:36:41,014 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(555)) - Not assigning more than one off-switch container, assignments so far: resource:; type:OFF_SWITCH; excessReservation:null; applicationid:null; skipped:NONE; fulfilled reservation:false; allocations(count/resource):1/; reservations(count/resource):0/ 2016-08-11 21:36:41,014 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1297)) - Trying to schedule on node: test, available: 2016-08-11 21:36:41,015 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(416)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 21:36:41,015 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1297)) - Trying to schedule on node: 127.0.0.1, available: 2016-08-11 21:36:41,015 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(416)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 21:36:41,015 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1297)) - Trying to schedule on node: test, available: 2016-08-11 21:36:41,015 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(416)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 21:36:41,021 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(991)) - allocate: pre-update appattempt_100_0002_000002 ask size =1 2016-08-11 21:36:41,021 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:36:41,021 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=1024 2016-08-11 21:36:41,022 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 0, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:36:41,022 DEBUG [main] scheduler.ActiveUsersManager (ActiveUsersManager.java:activateApplication(68)) - User user2 added to activeUsers, currently: 1 2016-08-11 21:36:41,022 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(1003)) - allocate: post-update 2016-08-11 21:36:41,022 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:36:41,023 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=1024 2016-08-11 21:36:41,023 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:36:41,036 DEBUG [main] security.BaseContainerTokenSecretManager (BaseContainerTokenSecretManager.java:createPassword(90)) - Creating password for container_100_0002_02_000001 for user container_100_0002_02_000001 (auth:SIMPLE) to be run on NM 127.0.0.1:123 2016-08-11 21:36:41,173 DEBUG [main] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 2 cluster_timestamp: 100 } attemptId: 2 } id: 1 } nmHostAddr: "127.0.0.1:123" appSubmitter: "user2" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1470932201024 masterKeyId: -1668377323 rmIdentifier: 1470931600597 priority { priority: 1 } creationTime: 1470931601008 nodeLabelExpression: "" containerType: TASK executionType: GUARANTEED 2016-08-11 21:36:41,178 DEBUG [main] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 2 cluster_timestamp: 100 } attemptId: 2 } id: 1 } nmHostAddr: "127.0.0.1:123" appSubmitter: "user2" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1470932201024 masterKeyId: -1668377323 rmIdentifier: 1470931600597 priority { priority: 1 } creationTime: 1470931601008 nodeLabelExpression: "" containerType: TASK executionType: GUARANTEED 2016-08-11 21:36:41,180 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0002_02_000001 of type ACQUIRED 2016-08-11 21:36:41,181 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppRunningOnNodeEvent.EventType: APP_RUNNING_ON_NODE 2016-08-11 21:36:41,181 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0002_02_000001 Container Transitioned from ALLOCATED to ACQUIRED 2016-08-11 21:36:41,181 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:36:41,182 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1297)) - Trying to schedule on node: 127.0.0.1, available: 2016-08-11 21:36:41,182 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(437)) - Trying to assign containers to child-queue of root 2016-08-11 21:36:41,182 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.15625238), label=(*)root.busedCapacity=(0.31257632), label=(*) 2016-08-11 21:36:41,182 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 2016-08-11 21:36:41,182 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=127.0.0.1 #applications=1 2016-08-11 21:36:41,183 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(909)) - Skip this queue=root.a, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 21:36:41,183 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 --> , NODE_LOCAL 2016-08-11 21:36:41,183 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=0.31257632, absoluteUsedCapacity=0.0625, numApps=1, numContainers=1 2016-08-11 21:36:41,183 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=127.0.0.1 #applications=1 2016-08-11 21:36:41,183 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:36:41,183 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimitAndSetHeadroom(1120)) - Headroom calculation for user user2: userLimit= queueMaxAvailRes= consumed= headroom= 2016-08-11 21:36:41,183 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:assignContainers(511)) - pre-assignContainers for application application_100_0002 2016-08-11 21:36:41,184 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:36:41,184 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=1024 2016-08-11 21:36:41,184 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:36:41,184 DEBUG [main] allocator.IncreaseContainerAllocator (IncreaseContainerAllocator.java:assignContainers(187)) - Skip allocating increase request since we don't have any increase request on this node=127.0.0.1:123 2016-08-11 21:36:41,185 DEBUG [main] allocator.RegularContainerAllocator (RegularContainerAllocator.java:assignContainer(473)) - assignContainers: node=127.0.0.1 application=application_100_0002 priority=1 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } type=OFF_SWITCH 2016-08-11 21:36:41,185 DEBUG [main] scheduler.ActiveUsersManager (ActiveUsersManager.java:deactivateApplication(94)) - User user2 removed from activeUsers, currently: 0 2016-08-11 21:36:41,185 DEBUG [main] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:allocate(631)) - allocate: applicationId=application_100_0002 container=container_100_0002_02_000002 host=127.0.0.1:123 user=user2 resource= type=OFF_SWITCH 2016-08-11 21:36:41,185 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0002_02_000002 of type START 2016-08-11 21:36:41,186 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: CONTAINER_ALLOCATED 2016-08-11 21:36:41,186 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0002_02_000002 Container Transitioned from NEW to ALLOCATED 2016-08-11 21:36:41,186 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:allocate(237)) - allocate: applicationAttemptId=appattempt_100_0002_000002 container=container_100_0002_02_000002 host=127.0.0.1 type=OFF_SWITCH 2016-08-11 21:36:41,186 INFO [main] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(159)) - USER=user2 OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_100_0002 CONTAINERID=container_100_0002_02_000002 RESOURCE= 2016-08-11 21:36:41,187 DEBUG [main] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(157)) - Assigned container container_100_0002_02_000002 of capacity on host 127.0.0.1:1, which has 3 containers, used and available after allocation 2016-08-11 21:36:41,187 INFO [main] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(120)) - assignedContainer application attempt=appattempt_100_0002_000002 container=container_100_0002_02_000002 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@386f0da3 clusterResource= type=OFF_SWITCH 2016-08-11 21:36:41,187 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(962)) - post-assignContainers for application application_100_0002 2016-08-11 21:36:41,188 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:36:41,188 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=2048 2016-08-11 21:36:41,188 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 0, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:36:41,188 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:36:41,189 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:36:41,189 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:36:41,189 DEBUG [main] capacity.LeafQueue (LeafQueue.java:allocateResource(1443)) - b user=user2 used= numContainers=2 headroom = user-resources= 2016-08-11 21:36:41,189 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=0.62515265, absoluteUsedCapacity=0.125, numApps=1, numContainers=2 --> , OFF_SWITCH 2016-08-11 21:36:41,190 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(670)) - Re-sorting assigned queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=0.62515265, absoluteUsedCapacity=0.125, numApps=1, numContainers=2 2016-08-11 21:36:41,190 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.15625238), label=(*)root.busedCapacity=(0.62515265), label=(*) 2016-08-11 21:36:41,190 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainers(522)) - assignedContainer queue=root usedCapacity=0.25 absoluteUsedCapacity=0.25 used= cluster= 2016-08-11 21:36:41,190 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(544)) - ParentQ=root assignedSoFarInThisIteration= usedCapacity=0.25 absoluteUsedCapacity=0.25 2016-08-11 21:36:41,191 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(555)) - Not assigning more than one off-switch container, assignments so far: resource:; type:OFF_SWITCH; excessReservation:null; applicationid:null; skipped:NONE; fulfilled reservation:false; allocations(count/resource):1/; reservations(count/resource):0/ 2016-08-11 21:36:41,191 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1297)) - Trying to schedule on node: test, available: 2016-08-11 21:36:41,191 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(416)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 21:36:41,191 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1297)) - Trying to schedule on node: 127.0.0.1, available: 2016-08-11 21:36:41,191 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(416)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 21:36:41,191 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1297)) - Trying to schedule on node: test, available: 2016-08-11 21:36:41,191 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(416)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 21:36:41,197 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(991)) - allocate: pre-update appattempt_100_0002_000002 ask size =1 2016-08-11 21:36:41,197 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:36:41,197 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=2048 2016-08-11 21:36:41,197 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 0, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:36:41,197 DEBUG [main] scheduler.ActiveUsersManager (ActiveUsersManager.java:activateApplication(68)) - User user2 added to activeUsers, currently: 1 2016-08-11 21:36:41,198 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(1003)) - allocate: post-update 2016-08-11 21:36:41,198 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:36:41,198 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=2048 2016-08-11 21:36:41,198 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:36:41,198 DEBUG [main] security.BaseContainerTokenSecretManager (BaseContainerTokenSecretManager.java:createPassword(90)) - Creating password for container_100_0002_02_000002 for user container_100_0002_02_000002 (auth:SIMPLE) to be run on NM 127.0.0.1:123 2016-08-11 21:36:41,199 DEBUG [main] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 2 cluster_timestamp: 100 } attemptId: 2 } id: 2 } nmHostAddr: "127.0.0.1:123" appSubmitter: "user2" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1470932201198 masterKeyId: -1668377323 rmIdentifier: 1470931600597 priority { priority: 1 } creationTime: 1470931601185 nodeLabelExpression: "" containerType: TASK executionType: GUARANTEED 2016-08-11 21:36:41,200 DEBUG [main] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 2 cluster_timestamp: 100 } attemptId: 2 } id: 2 } nmHostAddr: "127.0.0.1:123" appSubmitter: "user2" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1470932201198 masterKeyId: -1668377323 rmIdentifier: 1470931600597 priority { priority: 1 } creationTime: 1470931601185 nodeLabelExpression: "" containerType: TASK executionType: GUARANTEED 2016-08-11 21:36:41,201 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0002_02_000002 of type ACQUIRED 2016-08-11 21:36:41,201 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppRunningOnNodeEvent.EventType: APP_RUNNING_ON_NODE 2016-08-11 21:36:41,201 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0002_02_000002 Container Transitioned from ALLOCATED to ACQUIRED 2016-08-11 21:36:41,202 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:36:41,202 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1297)) - Trying to schedule on node: test, available: 2016-08-11 21:36:41,202 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(437)) - Trying to assign containers to child-queue of root 2016-08-11 21:36:41,202 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.15625238), label=(*)root.busedCapacity=(0.62515265), label=(*) 2016-08-11 21:36:41,203 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 2016-08-11 21:36:41,203 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=test #applications=1 2016-08-11 21:36:41,203 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(909)) - Skip this queue=root.a, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 21:36:41,203 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 --> , NODE_LOCAL 2016-08-11 21:36:41,203 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=0.62515265, absoluteUsedCapacity=0.125, numApps=1, numContainers=2 2016-08-11 21:36:41,203 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=test #applications=1 2016-08-11 21:36:41,204 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:36:41,204 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimitAndSetHeadroom(1120)) - Headroom calculation for user user2: userLimit= queueMaxAvailRes= consumed= headroom= 2016-08-11 21:36:41,204 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:assignContainers(511)) - pre-assignContainers for application application_100_0002 2016-08-11 21:36:41,204 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:36:41,204 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=2048 2016-08-11 21:36:41,205 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:36:41,205 DEBUG [main] allocator.IncreaseContainerAllocator (IncreaseContainerAllocator.java:assignContainers(187)) - Skip allocating increase request since we don't have any increase request on this node=test:123 2016-08-11 21:36:41,205 DEBUG [main] allocator.RegularContainerAllocator (RegularContainerAllocator.java:assignContainer(473)) - assignContainers: node=test application=application_100_0002 priority=1 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } type=OFF_SWITCH 2016-08-11 21:36:41,205 DEBUG [main] scheduler.ActiveUsersManager (ActiveUsersManager.java:deactivateApplication(94)) - User user2 removed from activeUsers, currently: 0 2016-08-11 21:36:41,206 DEBUG [main] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:allocate(631)) - allocate: applicationId=application_100_0002 container=container_100_0002_02_000003 host=test:123 user=user2 resource= type=OFF_SWITCH 2016-08-11 21:36:41,206 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0002_02_000003 of type START 2016-08-11 21:36:41,206 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: CONTAINER_ALLOCATED 2016-08-11 21:36:41,207 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0002_02_000003 Container Transitioned from NEW to ALLOCATED 2016-08-11 21:36:41,207 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:allocate(237)) - allocate: applicationAttemptId=appattempt_100_0002_000002 container=container_100_0002_02_000003 host=test type=OFF_SWITCH 2016-08-11 21:36:41,208 INFO [main] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(159)) - USER=user2 OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_100_0002 CONTAINERID=container_100_0002_02_000003 RESOURCE= 2016-08-11 21:36:41,208 DEBUG [main] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(157)) - Assigned container container_100_0002_02_000003 of capacity on host test:2, which has 1 containers, used and available after allocation 2016-08-11 21:36:41,208 INFO [main] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(120)) - assignedContainer application attempt=appattempt_100_0002_000002 container=container_100_0002_02_000003 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@386f0da3 clusterResource= type=OFF_SWITCH 2016-08-11 21:36:41,208 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(962)) - post-assignContainers for application application_100_0002 2016-08-11 21:36:41,209 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:36:41,209 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=3072 2016-08-11 21:36:41,209 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 0, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:36:41,210 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:36:41,210 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:36:41,210 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:36:41,210 DEBUG [main] capacity.LeafQueue (LeafQueue.java:allocateResource(1443)) - b user=user2 used= numContainers=3 headroom = user-resources= 2016-08-11 21:36:41,210 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=0.93772894, absoluteUsedCapacity=0.1875, numApps=1, numContainers=3 --> , OFF_SWITCH 2016-08-11 21:36:41,211 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(670)) - Re-sorting assigned queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=0.93772894, absoluteUsedCapacity=0.1875, numApps=1, numContainers=3 2016-08-11 21:36:41,211 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.15625238), label=(*)root.busedCapacity=(0.93772894), label=(*) 2016-08-11 21:36:41,211 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainers(522)) - assignedContainer queue=root usedCapacity=0.3125 absoluteUsedCapacity=0.3125 used= cluster= 2016-08-11 21:36:41,211 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(544)) - ParentQ=root assignedSoFarInThisIteration= usedCapacity=0.3125 absoluteUsedCapacity=0.3125 2016-08-11 21:36:41,211 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(555)) - Not assigning more than one off-switch container, assignments so far: resource:; type:OFF_SWITCH; excessReservation:null; applicationid:null; skipped:NONE; fulfilled reservation:false; allocations(count/resource):1/; reservations(count/resource):0/ 2016-08-11 21:36:41,212 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1297)) - Trying to schedule on node: 127.0.0.1, available: 2016-08-11 21:36:41,212 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(416)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 21:36:41,212 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1297)) - Trying to schedule on node: test, available: 2016-08-11 21:36:41,212 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(416)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 21:36:41,218 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(991)) - allocate: pre-update appattempt_100_0002_000002 ask size =1 2016-08-11 21:36:41,218 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:36:41,218 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=3072 2016-08-11 21:36:41,218 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 0, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:36:41,218 DEBUG [main] scheduler.ActiveUsersManager (ActiveUsersManager.java:activateApplication(68)) - User user2 added to activeUsers, currently: 1 2016-08-11 21:36:41,219 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(1003)) - allocate: post-update 2016-08-11 21:36:41,219 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:36:41,219 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=3072 2016-08-11 21:36:41,219 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:36:41,219 DEBUG [main] security.BaseContainerTokenSecretManager (BaseContainerTokenSecretManager.java:createPassword(90)) - Creating password for container_100_0002_02_000003 for user container_100_0002_02_000003 (auth:SIMPLE) to be run on NM test:123 2016-08-11 21:36:41,220 DEBUG [main] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 2 cluster_timestamp: 100 } attemptId: 2 } id: 3 } nmHostAddr: "test:123" appSubmitter: "user2" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1470932201219 masterKeyId: -1668377323 rmIdentifier: 1470931600597 priority { priority: 1 } creationTime: 1470931601205 nodeLabelExpression: "" containerType: TASK executionType: GUARANTEED 2016-08-11 21:37:01,247 DEBUG [main] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 2 cluster_timestamp: 100 } attemptId: 2 } id: 3 } nmHostAddr: "test:123" appSubmitter: "user2" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1470932201219 masterKeyId: -1668377323 rmIdentifier: 1470931600597 priority { priority: 1 } creationTime: 1470931601205 nodeLabelExpression: "" containerType: TASK executionType: GUARANTEED 2016-08-11 21:37:01,248 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0002_02_000003 of type ACQUIRED 2016-08-11 21:37:01,249 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppRunningOnNodeEvent.EventType: APP_RUNNING_ON_NODE 2016-08-11 21:37:01,249 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0002_02_000003 Container Transitioned from ALLOCATED to ACQUIRED 2016-08-11 21:37:01,250 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:37:01,252 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1297)) - Trying to schedule on node: 127.0.0.1, available: 2016-08-11 21:37:01,252 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(437)) - Trying to assign containers to child-queue of root 2016-08-11 21:37:01,252 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.15625238), label=(*)root.busedCapacity=(0.93772894), label=(*) 2016-08-11 21:37:01,253 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 2016-08-11 21:37:01,253 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=127.0.0.1 #applications=1 2016-08-11 21:37:01,254 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(909)) - Skip this queue=root.a, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 21:37:01,254 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 --> , NODE_LOCAL 2016-08-11 21:37:01,255 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=0.93772894, absoluteUsedCapacity=0.1875, numApps=1, numContainers=3 2016-08-11 21:37:01,255 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=127.0.0.1 #applications=1 2016-08-11 21:37:01,255 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:37:01,256 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimitAndSetHeadroom(1120)) - Headroom calculation for user user2: userLimit= queueMaxAvailRes= consumed= headroom= 2016-08-11 21:37:01,256 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:assignContainers(511)) - pre-assignContainers for application application_100_0002 2016-08-11 21:37:01,257 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:37:01,257 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=3072 2016-08-11 21:37:01,257 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:37:01,258 DEBUG [main] allocator.IncreaseContainerAllocator (IncreaseContainerAllocator.java:assignContainers(187)) - Skip allocating increase request since we don't have any increase request on this node=127.0.0.1:123 2016-08-11 21:37:01,258 DEBUG [main] allocator.RegularContainerAllocator (RegularContainerAllocator.java:assignContainer(473)) - assignContainers: node=127.0.0.1 application=application_100_0002 priority=1 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } type=OFF_SWITCH 2016-08-11 21:37:01,259 DEBUG [main] scheduler.ActiveUsersManager (ActiveUsersManager.java:deactivateApplication(94)) - User user2 removed from activeUsers, currently: 0 2016-08-11 21:37:01,260 DEBUG [main] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:allocate(631)) - allocate: applicationId=application_100_0002 container=container_100_0002_02_000004 host=127.0.0.1:123 user=user2 resource= type=OFF_SWITCH 2016-08-11 21:37:01,261 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0002_02_000004 of type START 2016-08-11 21:37:01,261 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0002_02_000004 Container Transitioned from NEW to ALLOCATED 2016-08-11 21:37:01,261 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: CONTAINER_ALLOCATED 2016-08-11 21:37:01,262 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:allocate(237)) - allocate: applicationAttemptId=appattempt_100_0002_000002 container=container_100_0002_02_000004 host=127.0.0.1 type=OFF_SWITCH 2016-08-11 21:37:01,262 INFO [main] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(159)) - USER=user2 OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_100_0002 CONTAINERID=container_100_0002_02_000004 RESOURCE= 2016-08-11 21:37:01,263 DEBUG [main] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(157)) - Assigned container container_100_0002_02_000004 of capacity on host 127.0.0.1:1, which has 4 containers, used and available after allocation 2016-08-11 21:37:01,264 INFO [main] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(120)) - assignedContainer application attempt=appattempt_100_0002_000002 container=container_100_0002_02_000004 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@386f0da3 clusterResource= type=OFF_SWITCH 2016-08-11 21:37:01,264 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(962)) - post-assignContainers for application application_100_0002 2016-08-11 21:37:01,265 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:37:01,265 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=4096 2016-08-11 21:37:01,265 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 0, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:37:01,266 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:37:01,266 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:37:01,267 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:37:01,267 DEBUG [main] capacity.LeafQueue (LeafQueue.java:allocateResource(1443)) - b user=user2 used= numContainers=4 headroom = user-resources= 2016-08-11 21:37:01,268 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=1.2503053, absoluteUsedCapacity=0.25, numApps=1, numContainers=4 --> , OFF_SWITCH 2016-08-11 21:37:01,268 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(670)) - Re-sorting assigned queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=1.2503053, absoluteUsedCapacity=0.25, numApps=1, numContainers=4 2016-08-11 21:37:01,268 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.15625238), label=(*)root.busedCapacity=(1.2503053), label=(*) 2016-08-11 21:37:01,269 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainers(522)) - assignedContainer queue=root usedCapacity=0.375 absoluteUsedCapacity=0.375 used= cluster= 2016-08-11 21:37:01,269 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(544)) - ParentQ=root assignedSoFarInThisIteration= usedCapacity=0.375 absoluteUsedCapacity=0.375 2016-08-11 21:37:01,269 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(555)) - Not assigning more than one off-switch container, assignments so far: resource:; type:OFF_SWITCH; excessReservation:null; applicationid:null; skipped:NONE; fulfilled reservation:false; allocations(count/resource):1/; reservations(count/resource):0/ 2016-08-11 21:37:01,270 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1297)) - Trying to schedule on node: test, available: 2016-08-11 21:37:01,270 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(416)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 21:37:01,270 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1297)) - Trying to schedule on node: 127.0.0.1, available: 2016-08-11 21:37:01,271 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(416)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 21:37:01,271 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1297)) - Trying to schedule on node: test, available: 2016-08-11 21:37:01,271 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(416)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 21:37:01,278 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(991)) - allocate: pre-update appattempt_100_0002_000002 ask size =1 2016-08-11 21:37:01,278 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:37:01,279 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=4096 2016-08-11 21:37:01,279 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 0, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:37:01,279 DEBUG [main] scheduler.ActiveUsersManager (ActiveUsersManager.java:activateApplication(68)) - User user2 added to activeUsers, currently: 1 2016-08-11 21:37:01,280 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(1003)) - allocate: post-update 2016-08-11 21:37:01,280 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:37:01,280 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=4096 2016-08-11 21:37:01,281 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:37:01,281 DEBUG [main] security.BaseContainerTokenSecretManager (BaseContainerTokenSecretManager.java:createPassword(90)) - Creating password for container_100_0002_02_000004 for user container_100_0002_02_000004 (auth:SIMPLE) to be run on NM 127.0.0.1:123 2016-08-11 21:37:01,283 DEBUG [main] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 2 cluster_timestamp: 100 } attemptId: 2 } id: 4 } nmHostAddr: "127.0.0.1:123" appSubmitter: "user2" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1470932221281 masterKeyId: -1668377323 rmIdentifier: 1470931600597 priority { priority: 1 } creationTime: 1470931621259 nodeLabelExpression: "" containerType: TASK executionType: GUARANTEED 2016-08-11 21:37:01,284 DEBUG [main] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 2 cluster_timestamp: 100 } attemptId: 2 } id: 4 } nmHostAddr: "127.0.0.1:123" appSubmitter: "user2" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1470932221281 masterKeyId: -1668377323 rmIdentifier: 1470931600597 priority { priority: 1 } creationTime: 1470931621259 nodeLabelExpression: "" containerType: TASK executionType: GUARANTEED 2016-08-11 21:37:01,284 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0002_02_000004 of type ACQUIRED 2016-08-11 21:37:01,285 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppRunningOnNodeEvent.EventType: APP_RUNNING_ON_NODE 2016-08-11 21:37:01,285 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0002_02_000004 Container Transitioned from ALLOCATED to ACQUIRED 2016-08-11 21:37:01,286 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:37:01,288 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1297)) - Trying to schedule on node: 127.0.0.1, available: 2016-08-11 21:37:01,288 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(437)) - Trying to assign containers to child-queue of root 2016-08-11 21:37:01,288 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.15625238), label=(*)root.busedCapacity=(1.2503053), label=(*) 2016-08-11 21:37:01,289 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 2016-08-11 21:37:01,289 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=127.0.0.1 #applications=1 2016-08-11 21:37:01,289 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(909)) - Skip this queue=root.a, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 21:37:01,290 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 --> , NODE_LOCAL 2016-08-11 21:37:01,290 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=1.2503053, absoluteUsedCapacity=0.25, numApps=1, numContainers=4 2016-08-11 21:37:01,291 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=127.0.0.1 #applications=1 2016-08-11 21:37:01,291 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:37:01,292 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimitAndSetHeadroom(1120)) - Headroom calculation for user user2: userLimit= queueMaxAvailRes= consumed= headroom= 2016-08-11 21:37:01,292 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:assignContainers(511)) - pre-assignContainers for application application_100_0002 2016-08-11 21:37:01,292 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:37:01,293 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=4096 2016-08-11 21:37:01,293 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:37:01,294 DEBUG [main] allocator.IncreaseContainerAllocator (IncreaseContainerAllocator.java:assignContainers(187)) - Skip allocating increase request since we don't have any increase request on this node=127.0.0.1:123 2016-08-11 21:37:01,294 DEBUG [main] allocator.RegularContainerAllocator (RegularContainerAllocator.java:assignContainer(473)) - assignContainers: node=127.0.0.1 application=application_100_0002 priority=1 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } type=OFF_SWITCH 2016-08-11 21:37:01,295 DEBUG [main] scheduler.ActiveUsersManager (ActiveUsersManager.java:deactivateApplication(94)) - User user2 removed from activeUsers, currently: 0 2016-08-11 21:37:01,295 DEBUG [main] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:allocate(631)) - allocate: applicationId=application_100_0002 container=container_100_0002_02_000005 host=127.0.0.1:123 user=user2 resource= type=OFF_SWITCH 2016-08-11 21:37:01,296 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0002_02_000005 of type START 2016-08-11 21:37:01,296 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: CONTAINER_ALLOCATED 2016-08-11 21:37:01,296 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0002_02_000005 Container Transitioned from NEW to ALLOCATED 2016-08-11 21:37:01,297 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:allocate(237)) - allocate: applicationAttemptId=appattempt_100_0002_000002 container=container_100_0002_02_000005 host=127.0.0.1 type=OFF_SWITCH 2016-08-11 21:37:01,297 INFO [main] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(159)) - USER=user2 OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_100_0002 CONTAINERID=container_100_0002_02_000005 RESOURCE= 2016-08-11 21:37:01,298 DEBUG [main] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(157)) - Assigned container container_100_0002_02_000005 of capacity on host 127.0.0.1:1, which has 5 containers, used and available after allocation 2016-08-11 21:37:01,300 INFO [main] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(120)) - assignedContainer application attempt=appattempt_100_0002_000002 container=container_100_0002_02_000005 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@386f0da3 clusterResource= type=OFF_SWITCH 2016-08-11 21:37:01,300 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(962)) - post-assignContainers for application application_100_0002 2016-08-11 21:37:01,300 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:37:01,301 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=5120 2016-08-11 21:37:01,301 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 0, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:37:01,302 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:37:01,302 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:37:01,303 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:37:01,303 DEBUG [main] capacity.LeafQueue (LeafQueue.java:allocateResource(1443)) - b user=user2 used= numContainers=5 headroom = user-resources= 2016-08-11 21:37:01,304 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=1.5628816, absoluteUsedCapacity=0.3125, numApps=1, numContainers=5 --> , OFF_SWITCH 2016-08-11 21:37:01,304 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(670)) - Re-sorting assigned queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=1.5628816, absoluteUsedCapacity=0.3125, numApps=1, numContainers=5 2016-08-11 21:37:01,305 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.15625238), label=(*)root.busedCapacity=(1.5628816), label=(*) 2016-08-11 21:37:01,306 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainers(522)) - assignedContainer queue=root usedCapacity=0.4375 absoluteUsedCapacity=0.4375 used= cluster= 2016-08-11 21:37:01,307 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(544)) - ParentQ=root assignedSoFarInThisIteration= usedCapacity=0.4375 absoluteUsedCapacity=0.4375 2016-08-11 21:37:01,308 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(555)) - Not assigning more than one off-switch container, assignments so far: resource:; type:OFF_SWITCH; excessReservation:null; applicationid:null; skipped:NONE; fulfilled reservation:false; allocations(count/resource):1/; reservations(count/resource):0/ 2016-08-11 21:37:01,308 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1297)) - Trying to schedule on node: test, available: 2016-08-11 21:37:01,308 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(416)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 21:37:01,308 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1297)) - Trying to schedule on node: 127.0.0.1, available: 2016-08-11 21:37:01,309 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(416)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 21:37:01,309 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1297)) - Trying to schedule on node: test, available: 2016-08-11 21:37:01,309 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(416)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 21:37:01,315 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(991)) - allocate: pre-update appattempt_100_0002_000002 ask size =1 2016-08-11 21:37:01,315 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:37:01,315 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=5120 2016-08-11 21:37:01,316 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 0, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:37:01,316 DEBUG [main] scheduler.ActiveUsersManager (ActiveUsersManager.java:activateApplication(68)) - User user2 added to activeUsers, currently: 1 2016-08-11 21:37:01,316 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(1003)) - allocate: post-update 2016-08-11 21:37:01,316 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:37:01,317 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=5120 2016-08-11 21:37:01,317 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:37:01,317 DEBUG [main] security.BaseContainerTokenSecretManager (BaseContainerTokenSecretManager.java:createPassword(90)) - Creating password for container_100_0002_02_000005 for user container_100_0002_02_000005 (auth:SIMPLE) to be run on NM 127.0.0.1:123 2016-08-11 21:37:01,318 DEBUG [main] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 2 cluster_timestamp: 100 } attemptId: 2 } id: 5 } nmHostAddr: "127.0.0.1:123" appSubmitter: "user2" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1470932221317 masterKeyId: -1668377323 rmIdentifier: 1470931600597 priority { priority: 1 } creationTime: 1470931621295 nodeLabelExpression: "" containerType: TASK executionType: GUARANTEED 2016-08-11 21:37:01,319 DEBUG [main] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 2 cluster_timestamp: 100 } attemptId: 2 } id: 5 } nmHostAddr: "127.0.0.1:123" appSubmitter: "user2" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1470932221317 masterKeyId: -1668377323 rmIdentifier: 1470931600597 priority { priority: 1 } creationTime: 1470931621295 nodeLabelExpression: "" containerType: TASK executionType: GUARANTEED 2016-08-11 21:37:01,320 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0002_02_000005 of type ACQUIRED 2016-08-11 21:37:01,320 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppRunningOnNodeEvent.EventType: APP_RUNNING_ON_NODE 2016-08-11 21:37:01,320 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0002_02_000005 Container Transitioned from ALLOCATED to ACQUIRED 2016-08-11 21:37:01,321 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:37:01,321 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1297)) - Trying to schedule on node: test, available: 2016-08-11 21:37:01,322 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(437)) - Trying to assign containers to child-queue of root 2016-08-11 21:37:01,322 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.15625238), label=(*)root.busedCapacity=(1.5628816), label=(*) 2016-08-11 21:37:01,322 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 2016-08-11 21:37:01,322 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=test #applications=1 2016-08-11 21:37:01,322 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(909)) - Skip this queue=root.a, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 21:37:01,323 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 --> , NODE_LOCAL 2016-08-11 21:37:01,323 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=1.5628816, absoluteUsedCapacity=0.3125, numApps=1, numContainers=5 2016-08-11 21:37:01,323 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=test #applications=1 2016-08-11 21:37:01,323 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:37:01,324 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimitAndSetHeadroom(1120)) - Headroom calculation for user user2: userLimit= queueMaxAvailRes= consumed= headroom= 2016-08-11 21:37:01,324 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:assignContainers(511)) - pre-assignContainers for application application_100_0002 2016-08-11 21:37:01,324 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:37:01,324 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=5120 2016-08-11 21:37:01,324 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:37:01,324 DEBUG [main] allocator.IncreaseContainerAllocator (IncreaseContainerAllocator.java:assignContainers(187)) - Skip allocating increase request since we don't have any increase request on this node=test:123 2016-08-11 21:37:01,324 DEBUG [main] allocator.RegularContainerAllocator (RegularContainerAllocator.java:assignContainer(473)) - assignContainers: node=test application=application_100_0002 priority=1 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } type=OFF_SWITCH 2016-08-11 21:37:01,325 DEBUG [main] scheduler.ActiveUsersManager (ActiveUsersManager.java:deactivateApplication(94)) - User user2 removed from activeUsers, currently: 0 2016-08-11 21:37:01,325 DEBUG [main] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:allocate(631)) - allocate: applicationId=application_100_0002 container=container_100_0002_02_000006 host=test:123 user=user2 resource= type=OFF_SWITCH 2016-08-11 21:37:01,325 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0002_02_000006 of type START 2016-08-11 21:37:01,325 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0002_02_000006 Container Transitioned from NEW to ALLOCATED 2016-08-11 21:37:01,325 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: CONTAINER_ALLOCATED 2016-08-11 21:37:01,325 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:allocate(237)) - allocate: applicationAttemptId=appattempt_100_0002_000002 container=container_100_0002_02_000006 host=test type=OFF_SWITCH 2016-08-11 21:37:01,326 INFO [main] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(159)) - USER=user2 OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_100_0002 CONTAINERID=container_100_0002_02_000006 RESOURCE= 2016-08-11 21:37:01,326 DEBUG [main] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(157)) - Assigned container container_100_0002_02_000006 of capacity on host test:2, which has 2 containers, used and available after allocation 2016-08-11 21:37:01,327 INFO [main] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(120)) - assignedContainer application attempt=appattempt_100_0002_000002 container=container_100_0002_02_000006 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@386f0da3 clusterResource= type=OFF_SWITCH 2016-08-11 21:37:01,327 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(962)) - post-assignContainers for application application_100_0002 2016-08-11 21:37:01,327 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:37:01,327 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=6144 2016-08-11 21:37:01,327 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 0, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:37:01,328 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:37:01,328 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:37:01,328 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:37:01,328 DEBUG [main] capacity.LeafQueue (LeafQueue.java:allocateResource(1443)) - b user=user2 used= numContainers=6 headroom = user-resources= 2016-08-11 21:37:01,329 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=1.8754579, absoluteUsedCapacity=0.375, numApps=1, numContainers=6 --> , OFF_SWITCH 2016-08-11 21:37:01,329 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(670)) - Re-sorting assigned queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=1.8754579, absoluteUsedCapacity=0.375, numApps=1, numContainers=6 2016-08-11 21:37:01,329 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.15625238), label=(*)root.busedCapacity=(1.8754579), label=(*) 2016-08-11 21:37:01,330 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainers(522)) - assignedContainer queue=root usedCapacity=0.5 absoluteUsedCapacity=0.5 used= cluster= 2016-08-11 21:37:01,330 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(544)) - ParentQ=root assignedSoFarInThisIteration= usedCapacity=0.5 absoluteUsedCapacity=0.5 2016-08-11 21:37:01,330 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(555)) - Not assigning more than one off-switch container, assignments so far: resource:; type:OFF_SWITCH; excessReservation:null; applicationid:null; skipped:NONE; fulfilled reservation:false; allocations(count/resource):1/; reservations(count/resource):0/ 2016-08-11 21:37:01,330 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1297)) - Trying to schedule on node: 127.0.0.1, available: 2016-08-11 21:37:01,330 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(416)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 21:37:01,330 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1297)) - Trying to schedule on node: test, available: 2016-08-11 21:37:01,331 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(416)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 21:37:01,336 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(991)) - allocate: pre-update appattempt_100_0002_000002 ask size =1 2016-08-11 21:37:01,336 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:37:01,337 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=6144 2016-08-11 21:37:01,337 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 0, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:37:01,337 DEBUG [main] scheduler.ActiveUsersManager (ActiveUsersManager.java:activateApplication(68)) - User user2 added to activeUsers, currently: 1 2016-08-11 21:37:01,337 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(1003)) - allocate: post-update 2016-08-11 21:37:01,337 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:37:01,338 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=6144 2016-08-11 21:37:01,338 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:37:01,338 DEBUG [main] security.BaseContainerTokenSecretManager (BaseContainerTokenSecretManager.java:createPassword(90)) - Creating password for container_100_0002_02_000006 for user container_100_0002_02_000006 (auth:SIMPLE) to be run on NM test:123 2016-08-11 21:37:01,339 DEBUG [main] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 2 cluster_timestamp: 100 } attemptId: 2 } id: 6 } nmHostAddr: "test:123" appSubmitter: "user2" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1470932221338 masterKeyId: -1668377323 rmIdentifier: 1470931600597 priority { priority: 1 } creationTime: 1470931621325 nodeLabelExpression: "" containerType: TASK executionType: GUARANTEED 2016-08-11 21:37:21,360 DEBUG [main] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 2 cluster_timestamp: 100 } attemptId: 2 } id: 6 } nmHostAddr: "test:123" appSubmitter: "user2" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1470932221338 masterKeyId: -1668377323 rmIdentifier: 1470931600597 priority { priority: 1 } creationTime: 1470931621325 nodeLabelExpression: "" containerType: TASK executionType: GUARANTEED 2016-08-11 21:37:21,361 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0002_02_000006 of type ACQUIRED 2016-08-11 21:37:21,362 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0002_02_000006 Container Transitioned from ALLOCATED to ACQUIRED 2016-08-11 21:37:21,362 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppRunningOnNodeEvent.EventType: APP_RUNNING_ON_NODE 2016-08-11 21:37:21,363 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:37:21,365 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1297)) - Trying to schedule on node: test, available: 2016-08-11 21:37:21,365 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(437)) - Trying to assign containers to child-queue of root 2016-08-11 21:37:21,366 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.15625238), label=(*)root.busedCapacity=(1.8754579), label=(*) 2016-08-11 21:37:21,366 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 2016-08-11 21:37:21,366 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=test #applications=1 2016-08-11 21:37:21,367 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(909)) - Skip this queue=root.a, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 21:37:21,368 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 --> , NODE_LOCAL 2016-08-11 21:37:21,368 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=1.8754579, absoluteUsedCapacity=0.375, numApps=1, numContainers=6 2016-08-11 21:37:21,369 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=test #applications=1 2016-08-11 21:37:21,369 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:37:21,369 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimitAndSetHeadroom(1120)) - Headroom calculation for user user2: userLimit= queueMaxAvailRes= consumed= headroom= 2016-08-11 21:37:21,370 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:assignContainers(511)) - pre-assignContainers for application application_100_0002 2016-08-11 21:37:21,370 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:37:21,370 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=6144 2016-08-11 21:37:21,371 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:37:21,371 DEBUG [main] allocator.IncreaseContainerAllocator (IncreaseContainerAllocator.java:assignContainers(187)) - Skip allocating increase request since we don't have any increase request on this node=test:123 2016-08-11 21:37:21,372 DEBUG [main] allocator.RegularContainerAllocator (RegularContainerAllocator.java:assignContainer(473)) - assignContainers: node=test application=application_100_0002 priority=1 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } type=OFF_SWITCH 2016-08-11 21:37:21,372 DEBUG [main] scheduler.ActiveUsersManager (ActiveUsersManager.java:deactivateApplication(94)) - User user2 removed from activeUsers, currently: 0 2016-08-11 21:37:21,374 DEBUG [main] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:allocate(631)) - allocate: applicationId=application_100_0002 container=container_100_0002_02_000007 host=test:123 user=user2 resource= type=OFF_SWITCH 2016-08-11 21:37:21,374 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0002_02_000007 of type START 2016-08-11 21:37:21,374 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: CONTAINER_ALLOCATED 2016-08-11 21:37:21,374 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0002_02_000007 Container Transitioned from NEW to ALLOCATED 2016-08-11 21:37:21,375 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:allocate(237)) - allocate: applicationAttemptId=appattempt_100_0002_000002 container=container_100_0002_02_000007 host=test type=OFF_SWITCH 2016-08-11 21:37:21,376 INFO [main] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(159)) - USER=user2 OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_100_0002 CONTAINERID=container_100_0002_02_000007 RESOURCE= 2016-08-11 21:37:21,376 DEBUG [main] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(157)) - Assigned container container_100_0002_02_000007 of capacity on host test:2, which has 3 containers, used and available after allocation 2016-08-11 21:37:21,377 INFO [main] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(120)) - assignedContainer application attempt=appattempt_100_0002_000002 container=container_100_0002_02_000007 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@386f0da3 clusterResource= type=OFF_SWITCH 2016-08-11 21:37:21,378 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(962)) - post-assignContainers for application application_100_0002 2016-08-11 21:37:21,378 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:37:21,379 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=7168 2016-08-11 21:37:21,379 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 0, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:37:21,380 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:37:21,380 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:37:21,380 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:37:21,381 DEBUG [main] capacity.LeafQueue (LeafQueue.java:allocateResource(1443)) - b user=user2 used= numContainers=7 headroom = user-resources= 2016-08-11 21:37:21,381 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=2.1880343, absoluteUsedCapacity=0.4375, numApps=1, numContainers=7 --> , OFF_SWITCH 2016-08-11 21:37:21,381 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(670)) - Re-sorting assigned queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=2.1880343, absoluteUsedCapacity=0.4375, numApps=1, numContainers=7 2016-08-11 21:37:21,382 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.15625238), label=(*)root.busedCapacity=(2.1880343), label=(*) 2016-08-11 21:37:21,382 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainers(522)) - assignedContainer queue=root usedCapacity=0.5625 absoluteUsedCapacity=0.5625 used= cluster= 2016-08-11 21:37:21,382 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(544)) - ParentQ=root assignedSoFarInThisIteration= usedCapacity=0.5625 absoluteUsedCapacity=0.5625 2016-08-11 21:37:21,383 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(555)) - Not assigning more than one off-switch container, assignments so far: resource:; type:OFF_SWITCH; excessReservation:null; applicationid:null; skipped:NONE; fulfilled reservation:false; allocations(count/resource):1/; reservations(count/resource):0/ 2016-08-11 21:37:21,383 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1297)) - Trying to schedule on node: 127.0.0.1, available: 2016-08-11 21:37:21,383 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(416)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 21:37:21,384 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1297)) - Trying to schedule on node: test, available: 2016-08-11 21:37:21,384 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(416)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 21:37:21,390 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(991)) - allocate: pre-update appattempt_100_0002_000002 ask size =1 2016-08-11 21:37:21,390 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:37:21,390 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=7168 2016-08-11 21:37:21,391 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 0, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:37:21,391 DEBUG [main] scheduler.ActiveUsersManager (ActiveUsersManager.java:activateApplication(68)) - User user2 added to activeUsers, currently: 1 2016-08-11 21:37:21,391 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(1003)) - allocate: post-update 2016-08-11 21:37:21,392 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:37:21,392 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=7168 2016-08-11 21:37:21,393 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:37:21,394 DEBUG [main] security.BaseContainerTokenSecretManager (BaseContainerTokenSecretManager.java:createPassword(90)) - Creating password for container_100_0002_02_000007 for user container_100_0002_02_000007 (auth:SIMPLE) to be run on NM test:123 2016-08-11 21:37:21,395 DEBUG [main] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 2 cluster_timestamp: 100 } attemptId: 2 } id: 7 } nmHostAddr: "test:123" appSubmitter: "user2" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1470932241393 masterKeyId: -1668377323 rmIdentifier: 1470931600597 priority { priority: 1 } creationTime: 1470931641372 nodeLabelExpression: "" containerType: TASK executionType: GUARANTEED 2016-08-11 21:37:41,414 DEBUG [main] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 2 cluster_timestamp: 100 } attemptId: 2 } id: 7 } nmHostAddr: "test:123" appSubmitter: "user2" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1470932241393 masterKeyId: -1668377323 rmIdentifier: 1470931600597 priority { priority: 1 } creationTime: 1470931641372 nodeLabelExpression: "" containerType: TASK executionType: GUARANTEED 2016-08-11 21:37:41,415 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0002_02_000007 of type ACQUIRED 2016-08-11 21:37:41,415 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0002_02_000007 Container Transitioned from ALLOCATED to ACQUIRED 2016-08-11 21:37:41,415 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppRunningOnNodeEvent.EventType: APP_RUNNING_ON_NODE 2016-08-11 21:37:41,416 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:37:41,416 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1297)) - Trying to schedule on node: 127.0.0.1, available: 2016-08-11 21:37:41,416 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(437)) - Trying to assign containers to child-queue of root 2016-08-11 21:37:41,416 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.15625238), label=(*)root.busedCapacity=(2.1880343), label=(*) 2016-08-11 21:37:41,416 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 2016-08-11 21:37:41,417 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=127.0.0.1 #applications=1 2016-08-11 21:37:41,417 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(909)) - Skip this queue=root.a, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 21:37:41,417 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 --> , NODE_LOCAL 2016-08-11 21:37:41,417 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=2.1880343, absoluteUsedCapacity=0.4375, numApps=1, numContainers=7 2016-08-11 21:37:41,417 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=127.0.0.1 #applications=1 2016-08-11 21:37:41,417 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:37:41,417 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimitAndSetHeadroom(1120)) - Headroom calculation for user user2: userLimit= queueMaxAvailRes= consumed= headroom= 2016-08-11 21:37:41,417 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:assignContainers(511)) - pre-assignContainers for application application_100_0002 2016-08-11 21:37:41,418 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:37:41,418 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=7168 2016-08-11 21:37:41,418 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:37:41,418 DEBUG [main] allocator.IncreaseContainerAllocator (IncreaseContainerAllocator.java:assignContainers(187)) - Skip allocating increase request since we don't have any increase request on this node=127.0.0.1:123 2016-08-11 21:37:41,418 DEBUG [main] allocator.RegularContainerAllocator (RegularContainerAllocator.java:assignContainer(473)) - assignContainers: node=127.0.0.1 application=application_100_0002 priority=1 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } type=OFF_SWITCH 2016-08-11 21:37:41,418 DEBUG [main] scheduler.ActiveUsersManager (ActiveUsersManager.java:deactivateApplication(94)) - User user2 removed from activeUsers, currently: 0 2016-08-11 21:37:41,419 DEBUG [main] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:allocate(631)) - allocate: applicationId=application_100_0002 container=container_100_0002_02_000008 host=127.0.0.1:123 user=user2 resource= type=OFF_SWITCH 2016-08-11 21:37:41,419 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0002_02_000008 of type START 2016-08-11 21:37:41,419 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0002_02_000008 Container Transitioned from NEW to ALLOCATED 2016-08-11 21:37:41,419 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: CONTAINER_ALLOCATED 2016-08-11 21:37:41,419 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:allocate(237)) - allocate: applicationAttemptId=appattempt_100_0002_000002 container=container_100_0002_02_000008 host=127.0.0.1 type=OFF_SWITCH 2016-08-11 21:37:41,420 INFO [main] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(159)) - USER=user2 OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_100_0002 CONTAINERID=container_100_0002_02_000008 RESOURCE= 2016-08-11 21:37:41,420 DEBUG [main] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(157)) - Assigned container container_100_0002_02_000008 of capacity on host 127.0.0.1:1, which has 6 containers, used and available after allocation 2016-08-11 21:37:41,421 INFO [main] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(120)) - assignedContainer application attempt=appattempt_100_0002_000002 container=container_100_0002_02_000008 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@386f0da3 clusterResource= type=OFF_SWITCH 2016-08-11 21:37:41,421 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(962)) - post-assignContainers for application application_100_0002 2016-08-11 21:37:41,421 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:37:41,421 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=8192 2016-08-11 21:37:41,421 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 0, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:37:41,421 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:37:41,421 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:37:41,422 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:37:41,422 DEBUG [main] capacity.LeafQueue (LeafQueue.java:allocateResource(1443)) - b user=user2 used= numContainers=8 headroom = user-resources= 2016-08-11 21:37:41,422 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=2.5006106, absoluteUsedCapacity=0.5, numApps=1, numContainers=8 --> , OFF_SWITCH 2016-08-11 21:37:41,422 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(670)) - Re-sorting assigned queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=2.5006106, absoluteUsedCapacity=0.5, numApps=1, numContainers=8 2016-08-11 21:37:41,422 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.15625238), label=(*)root.busedCapacity=(2.5006106), label=(*) 2016-08-11 21:37:41,422 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainers(522)) - assignedContainer queue=root usedCapacity=0.625 absoluteUsedCapacity=0.625 used= cluster= 2016-08-11 21:37:41,422 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(544)) - ParentQ=root assignedSoFarInThisIteration= usedCapacity=0.625 absoluteUsedCapacity=0.625 2016-08-11 21:37:41,422 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(555)) - Not assigning more than one off-switch container, assignments so far: resource:; type:OFF_SWITCH; excessReservation:null; applicationid:null; skipped:NONE; fulfilled reservation:false; allocations(count/resource):1/; reservations(count/resource):0/ 2016-08-11 21:37:41,423 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1297)) - Trying to schedule on node: test, available: 2016-08-11 21:37:41,423 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(416)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 21:37:41,423 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1297)) - Trying to schedule on node: 127.0.0.1, available: 2016-08-11 21:37:41,423 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(416)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 21:37:41,423 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1297)) - Trying to schedule on node: test, available: 2016-08-11 21:37:41,423 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(416)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 21:37:41,429 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(991)) - allocate: pre-update appattempt_100_0002_000002 ask size =1 2016-08-11 21:37:41,430 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:37:41,430 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=8192 2016-08-11 21:37:41,430 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 0, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:37:41,430 DEBUG [main] scheduler.ActiveUsersManager (ActiveUsersManager.java:activateApplication(68)) - User user2 added to activeUsers, currently: 1 2016-08-11 21:37:41,430 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(1003)) - allocate: post-update 2016-08-11 21:37:41,431 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:37:41,431 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=8192 2016-08-11 21:37:41,431 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:37:41,431 DEBUG [main] security.BaseContainerTokenSecretManager (BaseContainerTokenSecretManager.java:createPassword(90)) - Creating password for container_100_0002_02_000008 for user container_100_0002_02_000008 (auth:SIMPLE) to be run on NM 127.0.0.1:123 2016-08-11 21:37:41,432 DEBUG [main] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 2 cluster_timestamp: 100 } attemptId: 2 } id: 8 } nmHostAddr: "127.0.0.1:123" appSubmitter: "user2" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1470932261431 masterKeyId: -1668377323 rmIdentifier: 1470931600597 priority { priority: 1 } creationTime: 1470931661418 nodeLabelExpression: "" containerType: TASK executionType: GUARANTEED 2016-08-11 21:37:41,452 DEBUG [main] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 2 cluster_timestamp: 100 } attemptId: 2 } id: 8 } nmHostAddr: "127.0.0.1:123" appSubmitter: "user2" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1470932261431 masterKeyId: -1668377323 rmIdentifier: 1470931600597 priority { priority: 1 } creationTime: 1470931661418 nodeLabelExpression: "" containerType: TASK executionType: GUARANTEED 2016-08-11 21:37:41,455 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0002_02_000008 of type ACQUIRED 2016-08-11 21:37:41,456 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppRunningOnNodeEvent.EventType: APP_RUNNING_ON_NODE 2016-08-11 21:37:41,457 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0002_02_000008 Container Transitioned from ALLOCATED to ACQUIRED 2016-08-11 21:37:41,457 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:37:41,458 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1297)) - Trying to schedule on node: test, available: 2016-08-11 21:37:41,458 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(437)) - Trying to assign containers to child-queue of root 2016-08-11 21:37:41,459 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.15625238), label=(*)root.busedCapacity=(2.5006106), label=(*) 2016-08-11 21:37:41,459 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 2016-08-11 21:37:41,459 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=test #applications=1 2016-08-11 21:37:41,459 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(909)) - Skip this queue=root.a, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 21:37:41,460 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 --> , NODE_LOCAL 2016-08-11 21:37:41,460 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=2.5006106, absoluteUsedCapacity=0.5, numApps=1, numContainers=8 2016-08-11 21:37:41,460 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=test #applications=1 2016-08-11 21:37:41,460 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:37:41,460 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimitAndSetHeadroom(1120)) - Headroom calculation for user user2: userLimit= queueMaxAvailRes= consumed= headroom= 2016-08-11 21:37:41,461 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:assignContainers(511)) - pre-assignContainers for application application_100_0002 2016-08-11 21:37:41,461 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:37:41,461 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=8192 2016-08-11 21:37:41,461 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:37:41,461 DEBUG [main] allocator.IncreaseContainerAllocator (IncreaseContainerAllocator.java:assignContainers(187)) - Skip allocating increase request since we don't have any increase request on this node=test:123 2016-08-11 21:37:41,462 DEBUG [main] allocator.RegularContainerAllocator (RegularContainerAllocator.java:assignContainer(473)) - assignContainers: node=test application=application_100_0002 priority=1 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } type=OFF_SWITCH 2016-08-11 21:37:41,462 DEBUG [main] scheduler.ActiveUsersManager (ActiveUsersManager.java:deactivateApplication(94)) - User user2 removed from activeUsers, currently: 0 2016-08-11 21:37:41,462 DEBUG [main] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:allocate(631)) - allocate: applicationId=application_100_0002 container=container_100_0002_02_000009 host=test:123 user=user2 resource= type=OFF_SWITCH 2016-08-11 21:37:41,463 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0002_02_000009 of type START 2016-08-11 21:37:41,463 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0002_02_000009 Container Transitioned from NEW to ALLOCATED 2016-08-11 21:37:41,463 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: CONTAINER_ALLOCATED 2016-08-11 21:37:41,463 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:allocate(237)) - allocate: applicationAttemptId=appattempt_100_0002_000002 container=container_100_0002_02_000009 host=test type=OFF_SWITCH 2016-08-11 21:37:41,463 INFO [main] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(159)) - USER=user2 OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_100_0002 CONTAINERID=container_100_0002_02_000009 RESOURCE= 2016-08-11 21:37:41,463 DEBUG [main] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(157)) - Assigned container container_100_0002_02_000009 of capacity on host test:2, which has 4 containers, used and available after allocation 2016-08-11 21:37:41,464 INFO [main] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(120)) - assignedContainer application attempt=appattempt_100_0002_000002 container=container_100_0002_02_000009 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@386f0da3 clusterResource= type=OFF_SWITCH 2016-08-11 21:37:41,464 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(962)) - post-assignContainers for application application_100_0002 2016-08-11 21:37:41,464 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:37:41,465 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=9216 2016-08-11 21:37:41,465 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 0, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:37:41,465 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:37:41,465 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:37:41,465 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:37:41,466 DEBUG [main] capacity.LeafQueue (LeafQueue.java:allocateResource(1443)) - b user=user2 used= numContainers=9 headroom = user-resources= 2016-08-11 21:37:41,466 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=2.813187, absoluteUsedCapacity=0.5625, numApps=1, numContainers=9 --> , OFF_SWITCH 2016-08-11 21:37:41,466 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(670)) - Re-sorting assigned queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=2.813187, absoluteUsedCapacity=0.5625, numApps=1, numContainers=9 2016-08-11 21:37:41,466 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.15625238), label=(*)root.busedCapacity=(2.813187), label=(*) 2016-08-11 21:37:41,466 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainers(522)) - assignedContainer queue=root usedCapacity=0.6875 absoluteUsedCapacity=0.6875 used= cluster= 2016-08-11 21:37:41,466 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(544)) - ParentQ=root assignedSoFarInThisIteration= usedCapacity=0.6875 absoluteUsedCapacity=0.6875 2016-08-11 21:37:41,467 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(555)) - Not assigning more than one off-switch container, assignments so far: resource:; type:OFF_SWITCH; excessReservation:null; applicationid:null; skipped:NONE; fulfilled reservation:false; allocations(count/resource):1/; reservations(count/resource):0/ 2016-08-11 21:37:41,467 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1297)) - Trying to schedule on node: 127.0.0.1, available: 2016-08-11 21:37:41,467 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(416)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 21:37:41,467 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1297)) - Trying to schedule on node: test, available: 2016-08-11 21:37:41,467 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(416)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 21:37:41,472 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(991)) - allocate: pre-update appattempt_100_0002_000002 ask size =1 2016-08-11 21:37:41,473 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:37:41,473 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=9216 2016-08-11 21:37:41,473 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 0, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:37:41,473 DEBUG [main] scheduler.ActiveUsersManager (ActiveUsersManager.java:activateApplication(68)) - User user2 added to activeUsers, currently: 1 2016-08-11 21:37:41,473 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(1003)) - allocate: post-update 2016-08-11 21:37:41,474 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:37:41,474 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=9216 2016-08-11 21:37:41,474 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:37:41,474 DEBUG [main] security.BaseContainerTokenSecretManager (BaseContainerTokenSecretManager.java:createPassword(90)) - Creating password for container_100_0002_02_000009 for user container_100_0002_02_000009 (auth:SIMPLE) to be run on NM test:123 2016-08-11 21:37:41,475 DEBUG [main] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 2 cluster_timestamp: 100 } attemptId: 2 } id: 9 } nmHostAddr: "test:123" appSubmitter: "user2" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1470932261474 masterKeyId: -1668377323 rmIdentifier: 1470931600597 priority { priority: 1 } creationTime: 1470931661462 nodeLabelExpression: "" containerType: TASK executionType: GUARANTEED 2016-08-11 21:38:01,497 DEBUG [main] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 2 cluster_timestamp: 100 } attemptId: 2 } id: 9 } nmHostAddr: "test:123" appSubmitter: "user2" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1470932261474 masterKeyId: -1668377323 rmIdentifier: 1470931600597 priority { priority: 1 } creationTime: 1470931661462 nodeLabelExpression: "" containerType: TASK executionType: GUARANTEED 2016-08-11 21:38:01,499 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0002_02_000009 of type ACQUIRED 2016-08-11 21:38:01,499 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0002_02_000009 Container Transitioned from ALLOCATED to ACQUIRED 2016-08-11 21:38:01,499 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppRunningOnNodeEvent.EventType: APP_RUNNING_ON_NODE 2016-08-11 21:38:01,500 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:38:01,502 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1297)) - Trying to schedule on node: test, available: 2016-08-11 21:38:01,502 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(437)) - Trying to assign containers to child-queue of root 2016-08-11 21:38:01,503 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.15625238), label=(*)root.busedCapacity=(2.813187), label=(*) 2016-08-11 21:38:01,503 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 2016-08-11 21:38:01,503 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=test #applications=1 2016-08-11 21:38:01,504 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(909)) - Skip this queue=root.a, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 21:38:01,504 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 --> , NODE_LOCAL 2016-08-11 21:38:01,504 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=2.813187, absoluteUsedCapacity=0.5625, numApps=1, numContainers=9 2016-08-11 21:38:01,505 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=test #applications=1 2016-08-11 21:38:01,505 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:38:01,506 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimitAndSetHeadroom(1120)) - Headroom calculation for user user2: userLimit= queueMaxAvailRes= consumed= headroom= 2016-08-11 21:38:01,506 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:assignContainers(511)) - pre-assignContainers for application application_100_0002 2016-08-11 21:38:01,507 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:38:01,507 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=9216 2016-08-11 21:38:01,507 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:38:01,508 DEBUG [main] allocator.IncreaseContainerAllocator (IncreaseContainerAllocator.java:assignContainers(187)) - Skip allocating increase request since we don't have any increase request on this node=test:123 2016-08-11 21:38:01,508 DEBUG [main] allocator.RegularContainerAllocator (RegularContainerAllocator.java:assignContainer(473)) - assignContainers: node=test application=application_100_0002 priority=1 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } type=OFF_SWITCH 2016-08-11 21:38:01,509 DEBUG [main] scheduler.ActiveUsersManager (ActiveUsersManager.java:deactivateApplication(94)) - User user2 removed from activeUsers, currently: 0 2016-08-11 21:38:01,510 DEBUG [main] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:allocate(631)) - allocate: applicationId=application_100_0002 container=container_100_0002_02_000010 host=test:123 user=user2 resource= type=OFF_SWITCH 2016-08-11 21:38:01,510 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0002_02_000010 of type START 2016-08-11 21:38:01,511 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0002_02_000010 Container Transitioned from NEW to ALLOCATED 2016-08-11 21:38:01,511 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: CONTAINER_ALLOCATED 2016-08-11 21:38:01,511 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:allocate(237)) - allocate: applicationAttemptId=appattempt_100_0002_000002 container=container_100_0002_02_000010 host=test type=OFF_SWITCH 2016-08-11 21:38:01,512 INFO [main] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(159)) - USER=user2 OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_100_0002 CONTAINERID=container_100_0002_02_000010 RESOURCE= 2016-08-11 21:38:01,512 DEBUG [main] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(157)) - Assigned container container_100_0002_02_000010 of capacity on host test:2, which has 5 containers, used and available after allocation 2016-08-11 21:38:01,514 INFO [main] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(120)) - assignedContainer application attempt=appattempt_100_0002_000002 container=container_100_0002_02_000010 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@386f0da3 clusterResource= type=OFF_SWITCH 2016-08-11 21:38:01,514 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(962)) - post-assignContainers for application application_100_0002 2016-08-11 21:38:01,515 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:38:01,515 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=10240 2016-08-11 21:38:01,515 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 0, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:38:01,516 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:38:01,516 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:38:01,517 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:38:01,517 DEBUG [main] capacity.LeafQueue (LeafQueue.java:allocateResource(1443)) - b user=user2 used= numContainers=10 headroom = user-resources= 2016-08-11 21:38:01,517 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=3.1257632, absoluteUsedCapacity=0.625, numApps=1, numContainers=10 --> , OFF_SWITCH 2016-08-11 21:38:01,518 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(670)) - Re-sorting assigned queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=3.1257632, absoluteUsedCapacity=0.625, numApps=1, numContainers=10 2016-08-11 21:38:01,518 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.15625238), label=(*)root.busedCapacity=(3.1257632), label=(*) 2016-08-11 21:38:01,519 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainers(522)) - assignedContainer queue=root usedCapacity=0.75 absoluteUsedCapacity=0.75 used= cluster= 2016-08-11 21:38:01,519 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(544)) - ParentQ=root assignedSoFarInThisIteration= usedCapacity=0.75 absoluteUsedCapacity=0.75 2016-08-11 21:38:01,519 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(555)) - Not assigning more than one off-switch container, assignments so far: resource:; type:OFF_SWITCH; excessReservation:null; applicationid:null; skipped:NONE; fulfilled reservation:false; allocations(count/resource):1/; reservations(count/resource):0/ 2016-08-11 21:38:01,519 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1297)) - Trying to schedule on node: 127.0.0.1, available: 2016-08-11 21:38:01,520 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(416)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 21:38:01,520 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1297)) - Trying to schedule on node: test, available: 2016-08-11 21:38:01,520 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(416)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 21:38:01,526 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(991)) - allocate: pre-update appattempt_100_0002_000002 ask size =1 2016-08-11 21:38:01,526 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:38:01,527 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=10240 2016-08-11 21:38:01,527 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 0, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:38:01,527 DEBUG [main] scheduler.ActiveUsersManager (ActiveUsersManager.java:activateApplication(68)) - User user2 added to activeUsers, currently: 1 2016-08-11 21:38:01,528 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(1003)) - allocate: post-update 2016-08-11 21:38:01,528 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:38:01,529 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=10240 2016-08-11 21:38:01,529 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:38:01,530 DEBUG [main] security.BaseContainerTokenSecretManager (BaseContainerTokenSecretManager.java:createPassword(90)) - Creating password for container_100_0002_02_000010 for user container_100_0002_02_000010 (auth:SIMPLE) to be run on NM test:123 2016-08-11 21:38:01,531 DEBUG [main] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 2 cluster_timestamp: 100 } attemptId: 2 } id: 10 } nmHostAddr: "test:123" appSubmitter: "user2" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1470932281529 masterKeyId: -1668377323 rmIdentifier: 1470931600597 priority { priority: 1 } creationTime: 1470931681509 nodeLabelExpression: "" containerType: TASK executionType: GUARANTEED 2016-08-11 21:38:21,546 DEBUG [main] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 2 cluster_timestamp: 100 } attemptId: 2 } id: 10 } nmHostAddr: "test:123" appSubmitter: "user2" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1470932281529 masterKeyId: -1668377323 rmIdentifier: 1470931600597 priority { priority: 1 } creationTime: 1470931681509 nodeLabelExpression: "" containerType: TASK executionType: GUARANTEED 2016-08-11 21:38:21,547 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0002_02_000010 of type ACQUIRED 2016-08-11 21:38:21,547 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0002_02_000010 Container Transitioned from ALLOCATED to ACQUIRED 2016-08-11 21:38:21,547 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppRunningOnNodeEvent.EventType: APP_RUNNING_ON_NODE 2016-08-11 21:38:21,549 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:38:21,550 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1297)) - Trying to schedule on node: 127.0.0.1, available: 2016-08-11 21:38:21,550 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(437)) - Trying to assign containers to child-queue of root 2016-08-11 21:38:21,551 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.15625238), label=(*)root.busedCapacity=(3.1257632), label=(*) 2016-08-11 21:38:21,551 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 2016-08-11 21:38:21,551 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=127.0.0.1 #applications=1 2016-08-11 21:38:21,551 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(909)) - Skip this queue=root.a, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 21:38:21,552 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 --> , NODE_LOCAL 2016-08-11 21:38:21,553 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=3.1257632, absoluteUsedCapacity=0.625, numApps=1, numContainers=10 2016-08-11 21:38:21,553 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=127.0.0.1 #applications=1 2016-08-11 21:38:21,553 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:38:21,554 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimitAndSetHeadroom(1120)) - Headroom calculation for user user2: userLimit= queueMaxAvailRes= consumed= headroom= 2016-08-11 21:38:21,554 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:assignContainers(511)) - pre-assignContainers for application application_100_0002 2016-08-11 21:38:21,554 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:38:21,555 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=10240 2016-08-11 21:38:21,555 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:38:21,555 DEBUG [main] allocator.IncreaseContainerAllocator (IncreaseContainerAllocator.java:assignContainers(187)) - Skip allocating increase request since we don't have any increase request on this node=127.0.0.1:123 2016-08-11 21:38:21,556 DEBUG [main] allocator.RegularContainerAllocator (RegularContainerAllocator.java:assignContainer(473)) - assignContainers: node=127.0.0.1 application=application_100_0002 priority=1 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } type=OFF_SWITCH 2016-08-11 21:38:21,556 DEBUG [main] scheduler.ActiveUsersManager (ActiveUsersManager.java:deactivateApplication(94)) - User user2 removed from activeUsers, currently: 0 2016-08-11 21:38:21,557 DEBUG [main] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:allocate(631)) - allocate: applicationId=application_100_0002 container=container_100_0002_02_000011 host=127.0.0.1:123 user=user2 resource= type=OFF_SWITCH 2016-08-11 21:38:21,557 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0002_02_000011 of type START 2016-08-11 21:38:21,557 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0002_02_000011 Container Transitioned from NEW to ALLOCATED 2016-08-11 21:38:21,557 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: CONTAINER_ALLOCATED 2016-08-11 21:38:21,558 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:allocate(237)) - allocate: applicationAttemptId=appattempt_100_0002_000002 container=container_100_0002_02_000011 host=127.0.0.1 type=OFF_SWITCH 2016-08-11 21:38:21,558 INFO [main] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(159)) - USER=user2 OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_100_0002 CONTAINERID=container_100_0002_02_000011 RESOURCE= 2016-08-11 21:38:21,559 DEBUG [main] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(157)) - Assigned container container_100_0002_02_000011 of capacity on host 127.0.0.1:1, which has 7 containers, used and available after allocation 2016-08-11 21:38:21,560 INFO [main] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(120)) - assignedContainer application attempt=appattempt_100_0002_000002 container=container_100_0002_02_000011 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@386f0da3 clusterResource= type=OFF_SWITCH 2016-08-11 21:38:21,560 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(962)) - post-assignContainers for application application_100_0002 2016-08-11 21:38:21,561 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:38:21,561 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=11264 2016-08-11 21:38:21,561 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 0, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:38:21,562 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:38:21,562 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:38:21,563 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:38:21,563 DEBUG [main] capacity.LeafQueue (LeafQueue.java:allocateResource(1443)) - b user=user2 used= numContainers=11 headroom = user-resources= 2016-08-11 21:38:21,563 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=3.4383395, absoluteUsedCapacity=0.6875, numApps=1, numContainers=11 --> , OFF_SWITCH 2016-08-11 21:38:21,564 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(670)) - Re-sorting assigned queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=3.4383395, absoluteUsedCapacity=0.6875, numApps=1, numContainers=11 2016-08-11 21:38:21,564 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.15625238), label=(*)root.busedCapacity=(3.4383395), label=(*) 2016-08-11 21:38:21,565 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainers(522)) - assignedContainer queue=root usedCapacity=0.8125 absoluteUsedCapacity=0.8125 used= cluster= 2016-08-11 21:38:21,565 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(544)) - ParentQ=root assignedSoFarInThisIteration= usedCapacity=0.8125 absoluteUsedCapacity=0.8125 2016-08-11 21:38:21,566 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(555)) - Not assigning more than one off-switch container, assignments so far: resource:; type:OFF_SWITCH; excessReservation:null; applicationid:null; skipped:NONE; fulfilled reservation:false; allocations(count/resource):1/; reservations(count/resource):0/ 2016-08-11 21:38:21,566 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1297)) - Trying to schedule on node: test, available: 2016-08-11 21:38:21,567 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(416)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 21:38:21,567 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1297)) - Trying to schedule on node: test, available: 2016-08-11 21:38:21,568 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(416)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 21:38:21,573 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(991)) - allocate: pre-update appattempt_100_0002_000002 ask size =1 2016-08-11 21:38:21,574 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:38:21,574 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=11264 2016-08-11 21:38:21,574 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 0, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:38:21,575 DEBUG [main] scheduler.ActiveUsersManager (ActiveUsersManager.java:activateApplication(68)) - User user2 added to activeUsers, currently: 1 2016-08-11 21:38:21,575 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(1003)) - allocate: post-update 2016-08-11 21:38:21,575 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:38:21,576 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=11264 2016-08-11 21:38:21,576 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:38:21,577 DEBUG [main] security.BaseContainerTokenSecretManager (BaseContainerTokenSecretManager.java:createPassword(90)) - Creating password for container_100_0002_02_000011 for user container_100_0002_02_000011 (auth:SIMPLE) to be run on NM 127.0.0.1:123 2016-08-11 21:38:21,577 DEBUG [main] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 2 cluster_timestamp: 100 } attemptId: 2 } id: 11 } nmHostAddr: "127.0.0.1:123" appSubmitter: "user2" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1470932301576 masterKeyId: -1668377323 rmIdentifier: 1470931600597 priority { priority: 1 } creationTime: 1470931701556 nodeLabelExpression: "" containerType: TASK executionType: GUARANTEED 2016-08-11 21:38:21,578 DEBUG [main] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 2 cluster_timestamp: 100 } attemptId: 2 } id: 11 } nmHostAddr: "127.0.0.1:123" appSubmitter: "user2" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1470932301576 masterKeyId: -1668377323 rmIdentifier: 1470931600597 priority { priority: 1 } creationTime: 1470931701556 nodeLabelExpression: "" containerType: TASK executionType: GUARANTEED 2016-08-11 21:38:21,578 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0002_02_000011 of type ACQUIRED 2016-08-11 21:38:21,578 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppRunningOnNodeEvent.EventType: APP_RUNNING_ON_NODE 2016-08-11 21:38:21,578 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0002_02_000011 Container Transitioned from ALLOCATED to ACQUIRED 2016-08-11 21:38:21,579 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:38:21,579 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1297)) - Trying to schedule on node: test, available: 2016-08-11 21:38:21,580 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(437)) - Trying to assign containers to child-queue of root 2016-08-11 21:38:21,580 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.15625238), label=(*)root.busedCapacity=(3.4383395), label=(*) 2016-08-11 21:38:21,580 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 2016-08-11 21:38:21,580 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=test #applications=1 2016-08-11 21:38:21,580 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(909)) - Skip this queue=root.a, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 21:38:21,581 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 --> , NODE_LOCAL 2016-08-11 21:38:21,581 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=3.4383395, absoluteUsedCapacity=0.6875, numApps=1, numContainers=11 2016-08-11 21:38:21,581 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=test #applications=1 2016-08-11 21:38:21,581 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:38:21,582 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimitAndSetHeadroom(1120)) - Headroom calculation for user user2: userLimit= queueMaxAvailRes= consumed= headroom= 2016-08-11 21:38:21,582 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:assignContainers(511)) - pre-assignContainers for application application_100_0002 2016-08-11 21:38:21,582 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:38:21,582 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=11264 2016-08-11 21:38:21,582 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:38:21,582 DEBUG [main] allocator.IncreaseContainerAllocator (IncreaseContainerAllocator.java:assignContainers(187)) - Skip allocating increase request since we don't have any increase request on this node=test:123 2016-08-11 21:38:21,583 DEBUG [main] allocator.RegularContainerAllocator (RegularContainerAllocator.java:assignContainer(473)) - assignContainers: node=test application=application_100_0002 priority=1 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } type=OFF_SWITCH 2016-08-11 21:38:21,583 DEBUG [main] scheduler.ActiveUsersManager (ActiveUsersManager.java:deactivateApplication(94)) - User user2 removed from activeUsers, currently: 0 2016-08-11 21:38:21,583 DEBUG [main] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:allocate(631)) - allocate: applicationId=application_100_0002 container=container_100_0002_02_000012 host=test:123 user=user2 resource= type=OFF_SWITCH 2016-08-11 21:38:21,583 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0002_02_000012 of type START 2016-08-11 21:38:21,584 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0002_02_000012 Container Transitioned from NEW to ALLOCATED 2016-08-11 21:38:21,584 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: CONTAINER_ALLOCATED 2016-08-11 21:38:21,584 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:allocate(237)) - allocate: applicationAttemptId=appattempt_100_0002_000002 container=container_100_0002_02_000012 host=test type=OFF_SWITCH 2016-08-11 21:38:21,584 INFO [main] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(159)) - USER=user2 OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_100_0002 CONTAINERID=container_100_0002_02_000012 RESOURCE= 2016-08-11 21:38:21,584 DEBUG [main] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(157)) - Assigned container container_100_0002_02_000012 of capacity on host test:2, which has 6 containers, used and available after allocation 2016-08-11 21:38:21,585 INFO [main] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(120)) - assignedContainer application attempt=appattempt_100_0002_000002 container=container_100_0002_02_000012 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@386f0da3 clusterResource= type=OFF_SWITCH 2016-08-11 21:38:21,585 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(962)) - post-assignContainers for application application_100_0002 2016-08-11 21:38:21,585 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:38:21,586 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=12288 2016-08-11 21:38:21,586 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 0, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:38:21,586 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:38:21,586 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:38:21,586 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:38:21,587 DEBUG [main] capacity.LeafQueue (LeafQueue.java:allocateResource(1443)) - b user=user2 used= numContainers=12 headroom = user-resources= 2016-08-11 21:38:21,587 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=3.7509158, absoluteUsedCapacity=0.75, numApps=1, numContainers=12 --> , OFF_SWITCH 2016-08-11 21:38:21,587 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(670)) - Re-sorting assigned queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=3.7509158, absoluteUsedCapacity=0.75, numApps=1, numContainers=12 2016-08-11 21:38:21,587 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.15625238), label=(*)root.busedCapacity=(3.7509158), label=(*) 2016-08-11 21:38:21,587 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainers(522)) - assignedContainer queue=root usedCapacity=0.875 absoluteUsedCapacity=0.875 used= cluster= 2016-08-11 21:38:21,587 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(544)) - ParentQ=root assignedSoFarInThisIteration= usedCapacity=0.875 absoluteUsedCapacity=0.875 2016-08-11 21:38:21,588 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(555)) - Not assigning more than one off-switch container, assignments so far: resource:; type:OFF_SWITCH; excessReservation:null; applicationid:null; skipped:NONE; fulfilled reservation:false; allocations(count/resource):1/; reservations(count/resource):0/ 2016-08-11 21:38:21,588 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1297)) - Trying to schedule on node: test, available: 2016-08-11 21:38:21,588 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(416)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 21:38:21,593 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(991)) - allocate: pre-update appattempt_100_0002_000002 ask size =1 2016-08-11 21:38:21,593 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:38:21,594 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=12288 2016-08-11 21:38:21,594 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 0, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:38:21,594 DEBUG [main] scheduler.ActiveUsersManager (ActiveUsersManager.java:activateApplication(68)) - User user2 added to activeUsers, currently: 1 2016-08-11 21:38:21,594 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(1003)) - allocate: post-update 2016-08-11 21:38:21,594 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:38:21,594 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=12288 2016-08-11 21:38:21,595 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:38:21,595 DEBUG [main] security.BaseContainerTokenSecretManager (BaseContainerTokenSecretManager.java:createPassword(90)) - Creating password for container_100_0002_02_000012 for user container_100_0002_02_000012 (auth:SIMPLE) to be run on NM test:123 2016-08-11 21:38:21,595 DEBUG [main] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 2 cluster_timestamp: 100 } attemptId: 2 } id: 12 } nmHostAddr: "test:123" appSubmitter: "user2" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1470932301595 masterKeyId: -1668377323 rmIdentifier: 1470931600597 priority { priority: 1 } creationTime: 1470931701583 nodeLabelExpression: "" containerType: TASK executionType: GUARANTEED 2016-08-11 21:38:41,616 DEBUG [main] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 2 cluster_timestamp: 100 } attemptId: 2 } id: 12 } nmHostAddr: "test:123" appSubmitter: "user2" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1470932301595 masterKeyId: -1668377323 rmIdentifier: 1470931600597 priority { priority: 1 } creationTime: 1470931701583 nodeLabelExpression: "" containerType: TASK executionType: GUARANTEED 2016-08-11 21:38:41,618 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0002_02_000012 of type ACQUIRED 2016-08-11 21:38:41,618 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppRunningOnNodeEvent.EventType: APP_RUNNING_ON_NODE 2016-08-11 21:38:41,619 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0002_02_000012 Container Transitioned from ALLOCATED to ACQUIRED 2016-08-11 21:38:41,620 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:38:41,621 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1297)) - Trying to schedule on node: test, available: 2016-08-11 21:38:41,621 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(437)) - Trying to assign containers to child-queue of root 2016-08-11 21:38:41,622 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.15625238), label=(*)root.busedCapacity=(3.7509158), label=(*) 2016-08-11 21:38:41,622 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 2016-08-11 21:38:41,623 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=test #applications=1 2016-08-11 21:38:41,623 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(909)) - Skip this queue=root.a, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 21:38:41,623 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 --> , NODE_LOCAL 2016-08-11 21:38:41,624 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=3.7509158, absoluteUsedCapacity=0.75, numApps=1, numContainers=12 2016-08-11 21:38:41,624 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=test #applications=1 2016-08-11 21:38:41,625 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:38:41,625 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimitAndSetHeadroom(1120)) - Headroom calculation for user user2: userLimit= queueMaxAvailRes= consumed= headroom= 2016-08-11 21:38:41,625 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:assignContainers(511)) - pre-assignContainers for application application_100_0002 2016-08-11 21:38:41,626 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:38:41,626 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=12288 2016-08-11 21:38:41,627 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:38:41,627 DEBUG [main] allocator.IncreaseContainerAllocator (IncreaseContainerAllocator.java:assignContainers(187)) - Skip allocating increase request since we don't have any increase request on this node=test:123 2016-08-11 21:38:41,628 DEBUG [main] allocator.RegularContainerAllocator (RegularContainerAllocator.java:assignContainer(473)) - assignContainers: node=test application=application_100_0002 priority=1 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } type=OFF_SWITCH 2016-08-11 21:38:41,628 DEBUG [main] scheduler.ActiveUsersManager (ActiveUsersManager.java:deactivateApplication(94)) - User user2 removed from activeUsers, currently: 0 2016-08-11 21:38:41,629 DEBUG [main] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:allocate(631)) - allocate: applicationId=application_100_0002 container=container_100_0002_02_000013 host=test:123 user=user2 resource= type=OFF_SWITCH 2016-08-11 21:38:41,629 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0002_02_000013 of type START 2016-08-11 21:38:41,630 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0002_02_000013 Container Transitioned from NEW to ALLOCATED 2016-08-11 21:38:41,630 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: CONTAINER_ALLOCATED 2016-08-11 21:38:41,630 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:allocate(237)) - allocate: applicationAttemptId=appattempt_100_0002_000002 container=container_100_0002_02_000013 host=test type=OFF_SWITCH 2016-08-11 21:38:41,631 INFO [main] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(159)) - USER=user2 OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_100_0002 CONTAINERID=container_100_0002_02_000013 RESOURCE= 2016-08-11 21:38:41,631 DEBUG [main] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(157)) - Assigned container container_100_0002_02_000013 of capacity on host test:2, which has 7 containers, used and available after allocation 2016-08-11 21:38:41,632 INFO [main] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(120)) - assignedContainer application attempt=appattempt_100_0002_000002 container=container_100_0002_02_000013 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@386f0da3 clusterResource= type=OFF_SWITCH 2016-08-11 21:38:41,633 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(962)) - post-assignContainers for application application_100_0002 2016-08-11 21:38:41,633 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:38:41,634 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=13312 2016-08-11 21:38:41,634 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 0, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:38:41,634 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:38:41,635 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:38:41,635 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:38:41,636 DEBUG [main] capacity.LeafQueue (LeafQueue.java:allocateResource(1443)) - b user=user2 used= numContainers=13 headroom = user-resources= 2016-08-11 21:38:41,636 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=4.0634923, absoluteUsedCapacity=0.8125, numApps=1, numContainers=13 --> , OFF_SWITCH 2016-08-11 21:38:41,637 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(670)) - Re-sorting assigned queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=4.0634923, absoluteUsedCapacity=0.8125, numApps=1, numContainers=13 2016-08-11 21:38:41,637 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.15625238), label=(*)root.busedCapacity=(4.0634923), label=(*) 2016-08-11 21:38:41,638 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainers(522)) - assignedContainer queue=root usedCapacity=0.9375 absoluteUsedCapacity=0.9375 used= cluster= 2016-08-11 21:38:41,638 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(544)) - ParentQ=root assignedSoFarInThisIteration= usedCapacity=0.9375 absoluteUsedCapacity=0.9375 2016-08-11 21:38:41,639 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(555)) - Not assigning more than one off-switch container, assignments so far: resource:; type:OFF_SWITCH; excessReservation:null; applicationid:null; skipped:NONE; fulfilled reservation:false; allocations(count/resource):1/; reservations(count/resource):0/ 2016-08-11 21:38:41,639 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1297)) - Trying to schedule on node: test, available: 2016-08-11 21:38:41,639 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(416)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 21:38:41,645 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(991)) - allocate: pre-update appattempt_100_0001_000001 ask size =1 2016-08-11 21:38:41,645 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user1 in queue a userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:38:41,646 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0001 headRoom= currentConsumption=2048 2016-08-11 21:38:41,646 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0001 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 0, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:38:41,646 DEBUG [main] scheduler.ActiveUsersManager (ActiveUsersManager.java:activateApplication(68)) - User user1 added to activeUsers, currently: 1 2016-08-11 21:38:41,647 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(1003)) - allocate: post-update 2016-08-11 21:38:41,647 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user1 in queue a userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:38:41,647 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0001 headRoom= currentConsumption=2048 2016-08-11 21:38:41,647 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0001 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:38:41,648 DEBUG [main] security.BaseContainerTokenSecretManager (BaseContainerTokenSecretManager.java:createPassword(90)) - Creating password for container_100_0001_01_000001 for user container_100_0001_01_000001 (auth:SIMPLE) to be run on NM 127.0.0.1:123 2016-08-11 21:38:41,649 DEBUG [main] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 1 cluster_timestamp: 100 } attemptId: 1 } id: 1 } nmHostAddr: "127.0.0.1:123" appSubmitter: "user1" resource { memory: 2048 virtual_cores: 1 } expiryTimeStamp: 1470932321648 masterKeyId: -1668377323 rmIdentifier: 1470931600597 priority { priority: 1 } creationTime: 1470931600973 nodeLabelExpression: "" containerType: TASK executionType: GUARANTEED 2016-08-11 21:38:41,650 DEBUG [main] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 1 cluster_timestamp: 100 } attemptId: 1 } id: 1 } nmHostAddr: "127.0.0.1:123" appSubmitter: "user1" resource { memory: 2048 virtual_cores: 1 } expiryTimeStamp: 1470932321648 masterKeyId: -1668377323 rmIdentifier: 1470931600597 priority { priority: 1 } creationTime: 1470931600973 nodeLabelExpression: "" containerType: TASK executionType: GUARANTEED 2016-08-11 21:38:41,651 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0001_01_000001 of type ACQUIRED 2016-08-11 21:38:41,651 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0001_01_000001 Container Transitioned from ALLOCATED to ACQUIRED 2016-08-11 21:38:41,651 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppRunningOnNodeEvent.EventType: APP_RUNNING_ON_NODE 2016-08-11 21:38:41,652 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user1 in queue a userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:38:41,653 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1297)) - Trying to schedule on node: test, available: 2016-08-11 21:38:41,653 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(437)) - Trying to assign containers to child-queue of root 2016-08-11 21:38:41,653 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.15625238), label=(*)root.busedCapacity=(4.0634923), label=(*) 2016-08-11 21:38:41,654 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 2016-08-11 21:38:41,654 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=test #applications=1 2016-08-11 21:38:41,654 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user1 in queue a userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:38:41,655 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimitAndSetHeadroom(1120)) - Headroom calculation for user user1: userLimit= queueMaxAvailRes= consumed= headroom= 2016-08-11 21:38:41,655 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:assignContainers(511)) - pre-assignContainers for application application_100_0001 2016-08-11 21:38:41,655 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user1 in queue a userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:38:41,655 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0001 headRoom= currentConsumption=2048 2016-08-11 21:38:41,656 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0001 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:38:41,656 DEBUG [main] allocator.IncreaseContainerAllocator (IncreaseContainerAllocator.java:assignContainers(187)) - Skip allocating increase request since we don't have any increase request on this node=test:123 2016-08-11 21:38:41,656 DEBUG [main] allocator.RegularContainerAllocator (RegularContainerAllocator.java:preCheckForNewContainer(170)) - cannot allocate required resource= because of headroom 2016-08-11 21:38:41,657 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(962)) - post-assignContainers for application application_100_0001 2016-08-11 21:38:41,657 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user1 in queue a userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:38:41,657 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0001 headRoom= currentConsumption=2048 2016-08-11 21:38:41,658 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0001 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:38:41,658 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 --> , NODE_LOCAL 2016-08-11 21:38:41,658 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(688)) - Decrease parentLimits for root by as childQueue=a is blocked 2016-08-11 21:38:41,658 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=4.0634923, absoluteUsedCapacity=0.8125, numApps=1, numContainers=13 2016-08-11 21:38:41,659 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=test #applications=1 2016-08-11 21:38:41,659 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(909)) - Skip this queue=root.b, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 21:38:41,659 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=4.0634923, absoluteUsedCapacity=0.8125, numApps=1, numContainers=13 --> , NODE_LOCAL 2016-08-11 21:38:41,660 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1297)) - Trying to schedule on node: test, available: 2016-08-11 21:38:41,660 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(437)) - Trying to assign containers to child-queue of root 2016-08-11 21:38:41,660 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.15625238), label=(*)root.busedCapacity=(4.0634923), label=(*) 2016-08-11 21:38:41,660 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 2016-08-11 21:38:41,661 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=test #applications=1 2016-08-11 21:38:41,661 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user1 in queue a userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:38:41,661 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimitAndSetHeadroom(1120)) - Headroom calculation for user user1: userLimit= queueMaxAvailRes= consumed= headroom= 2016-08-11 21:38:41,661 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:assignContainers(511)) - pre-assignContainers for application application_100_0001 2016-08-11 21:38:41,662 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user1 in queue a userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:38:41,662 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0001 headRoom= currentConsumption=2048 2016-08-11 21:38:41,662 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0001 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:38:41,663 DEBUG [main] allocator.IncreaseContainerAllocator (IncreaseContainerAllocator.java:assignContainers(187)) - Skip allocating increase request since we don't have any increase request on this node=test:123 2016-08-11 21:38:41,663 DEBUG [main] allocator.RegularContainerAllocator (RegularContainerAllocator.java:preCheckForNewContainer(170)) - cannot allocate required resource= because of headroom 2016-08-11 21:38:41,663 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(962)) - post-assignContainers for application application_100_0001 2016-08-11 21:38:41,663 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user1 in queue a userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:38:41,664 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0001 headRoom= currentConsumption=2048 2016-08-11 21:38:41,664 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0001 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:38:41,664 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 --> , NODE_LOCAL 2016-08-11 21:38:41,664 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(688)) - Decrease parentLimits for root by as childQueue=a is blocked 2016-08-11 21:38:41,665 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=4.0634923, absoluteUsedCapacity=0.8125, numApps=1, numContainers=13 2016-08-11 21:38:41,665 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=test #applications=1 2016-08-11 21:38:41,665 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(909)) - Skip this queue=root.b, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 21:38:41,666 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=4.0634923, absoluteUsedCapacity=0.8125, numApps=1, numContainers=13 --> , NODE_LOCAL 2016-08-11 21:38:41,671 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(991)) - allocate: pre-update appattempt_100_0002_000002 ask size =1 2016-08-11 21:38:41,672 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:38:41,672 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=13312 2016-08-11 21:38:41,672 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 0, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:38:41,673 DEBUG [main] scheduler.ActiveUsersManager (ActiveUsersManager.java:activateApplication(68)) - User user2 added to activeUsers, currently: 1 2016-08-11 21:38:41,673 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(1003)) - allocate: post-update 2016-08-11 21:38:41,673 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:38:41,674 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=13312 2016-08-11 21:38:41,674 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:38:41,675 DEBUG [main] security.BaseContainerTokenSecretManager (BaseContainerTokenSecretManager.java:createPassword(90)) - Creating password for container_100_0002_02_000013 for user container_100_0002_02_000013 (auth:SIMPLE) to be run on NM test:123 2016-08-11 21:38:41,676 DEBUG [main] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 2 cluster_timestamp: 100 } attemptId: 2 } id: 13 } nmHostAddr: "test:123" appSubmitter: "user2" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1470932321674 masterKeyId: -1668377323 rmIdentifier: 1470931600597 priority { priority: 1 } creationTime: 1470931721628 nodeLabelExpression: "" containerType: TASK executionType: GUARANTEED 2016-08-11 21:39:01,699 DEBUG [main] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 2 cluster_timestamp: 100 } attemptId: 2 } id: 13 } nmHostAddr: "test:123" appSubmitter: "user2" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1470932321674 masterKeyId: -1668377323 rmIdentifier: 1470931600597 priority { priority: 1 } creationTime: 1470931721628 nodeLabelExpression: "" containerType: TASK executionType: GUARANTEED 2016-08-11 21:39:01,700 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0002_02_000013 of type ACQUIRED 2016-08-11 21:39:01,700 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0002_02_000013 Container Transitioned from ALLOCATED to ACQUIRED 2016-08-11 21:39:01,700 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppRunningOnNodeEvent.EventType: APP_RUNNING_ON_NODE 2016-08-11 21:39:01,701 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:39:01,703 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1297)) - Trying to schedule on node: test, available: 2016-08-11 21:39:01,704 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(437)) - Trying to assign containers to child-queue of root 2016-08-11 21:39:01,704 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.15625238), label=(*)root.busedCapacity=(4.0634923), label=(*) 2016-08-11 21:39:01,705 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 2016-08-11 21:39:01,705 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=test #applications=1 2016-08-11 21:39:01,706 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user1 in queue a userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:39:01,706 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimitAndSetHeadroom(1120)) - Headroom calculation for user user1: userLimit= queueMaxAvailRes= consumed= headroom= 2016-08-11 21:39:01,707 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:assignContainers(511)) - pre-assignContainers for application application_100_0001 2016-08-11 21:39:01,707 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user1 in queue a userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:39:01,708 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0001 headRoom= currentConsumption=2048 2016-08-11 21:39:01,708 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0001 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:39:01,709 DEBUG [main] allocator.IncreaseContainerAllocator (IncreaseContainerAllocator.java:assignContainers(187)) - Skip allocating increase request since we don't have any increase request on this node=test:123 2016-08-11 21:39:01,711 DEBUG [main] allocator.RegularContainerAllocator (RegularContainerAllocator.java:preCheckForNewContainer(170)) - cannot allocate required resource= because of headroom 2016-08-11 21:39:01,711 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(962)) - post-assignContainers for application application_100_0001 2016-08-11 21:39:01,712 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user1 in queue a userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:39:01,713 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0001 headRoom= currentConsumption=2048 2016-08-11 21:39:01,713 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0001 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:39:01,714 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 --> , NODE_LOCAL 2016-08-11 21:39:01,714 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(688)) - Decrease parentLimits for root by as childQueue=a is blocked 2016-08-11 21:39:01,715 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=4.0634923, absoluteUsedCapacity=0.8125, numApps=1, numContainers=13 2016-08-11 21:39:01,715 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=test #applications=1 2016-08-11 21:39:01,716 DEBUG [main] capacity.AbstractCSQueue (AbstractCSQueue.java:canAssignToThisQueue(530)) - bCheck assign to queue, nodePartition= usedResources: clusterResources: currentUsedCapacity 0.8125 max-capacity: 1.0) 2016-08-11 21:39:01,717 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=4.0634923, absoluteUsedCapacity=0.8125, numApps=1, numContainers=13 --> , NODE_LOCAL 2016-08-11 21:39:01,717 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1297)) - Trying to schedule on node: test, available: 2016-08-11 21:39:01,718 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(437)) - Trying to assign containers to child-queue of root 2016-08-11 21:39:01,718 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.15625238), label=(*)root.busedCapacity=(4.0634923), label=(*) 2016-08-11 21:39:01,719 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 2016-08-11 21:39:01,719 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=test #applications=1 2016-08-11 21:39:01,719 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user1 in queue a userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:39:01,720 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimitAndSetHeadroom(1120)) - Headroom calculation for user user1: userLimit= queueMaxAvailRes= consumed= headroom= 2016-08-11 21:39:01,720 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:assignContainers(511)) - pre-assignContainers for application application_100_0001 2016-08-11 21:39:01,721 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user1 in queue a userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:39:01,721 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0001 headRoom= currentConsumption=2048 2016-08-11 21:39:01,722 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0001 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:39:01,722 DEBUG [main] allocator.IncreaseContainerAllocator (IncreaseContainerAllocator.java:assignContainers(187)) - Skip allocating increase request since we don't have any increase request on this node=test:123 2016-08-11 21:39:01,723 DEBUG [main] allocator.RegularContainerAllocator (RegularContainerAllocator.java:preCheckForNewContainer(170)) - cannot allocate required resource= because of headroom 2016-08-11 21:39:01,723 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(962)) - post-assignContainers for application application_100_0001 2016-08-11 21:39:01,724 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user1 in queue a userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:39:01,724 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0001 headRoom= currentConsumption=2048 2016-08-11 21:39:01,724 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0001 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:39:01,725 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 --> , NODE_LOCAL 2016-08-11 21:39:01,725 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(688)) - Decrease parentLimits for root by as childQueue=a is blocked 2016-08-11 21:39:01,725 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=4.0634923, absoluteUsedCapacity=0.8125, numApps=1, numContainers=13 2016-08-11 21:39:01,725 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=test #applications=1 2016-08-11 21:39:01,726 DEBUG [main] capacity.AbstractCSQueue (AbstractCSQueue.java:canAssignToThisQueue(530)) - bCheck assign to queue, nodePartition= usedResources: clusterResources: currentUsedCapacity 0.8125 max-capacity: 1.0) 2016-08-11 21:39:01,726 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=4.0634923, absoluteUsedCapacity=0.8125, numApps=1, numContainers=13 --> , NODE_LOCAL 2016-08-11 21:39:01,804 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0002_02_000010 of type EXPIRE 2016-08-11 21:39:01,805 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeCleanContainerEvent.EventType: CLEANUP_CONTAINER 2016-08-11 21:39:01,806 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptContainerFinishedEvent.EventType: CONTAINER_FINISHED 2016-08-11 21:39:01,807 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0002_02_000010 Container Transitioned from ACQUIRED to EXPIRED 2016-08-11 21:39:01,807 INFO [main] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(159)) - USER=user2 OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_100_0002 CONTAINERID=container_100_0002_02_000010 RESOURCE= 2016-08-11 21:39:01,808 DEBUG [main] scheduler.SchedulerNode (SchedulerNode.java:releaseContainer(269)) - Released container container_100_0002_02_000010 of capacity on host test:2, which currently has 6 containers, used and available, release resources=true 2016-08-11 21:39:01,808 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:39:01,809 DEBUG [main] capacity.LeafQueue (LeafQueue.java:releaseResource(1479)) - b used= numContainers=12 user=user2 user-resources= 2016-08-11 21:39:01,809 DEBUG [main] capacity.ParentQueue (ParentQueue.java:internalReleaseResource(728)) - completedContainer root: numChildQueue= 2, capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.875, numApps=2, numContainers=13, cluster= 2016-08-11 21:39:01,809 DEBUG [main] capacity.ParentQueue (ParentQueue.java:internalReleaseResource(741)) - Re-sorting completed queue: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=3.7509158, absoluteUsedCapacity=0.75, numApps=1, numContainers=12 2016-08-11 21:39:01,810 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0002_02_000011 of type EXPIRE 2016-08-11 21:39:01,810 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeCleanContainerEvent.EventType: CLEANUP_CONTAINER 2016-08-11 21:39:01,811 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptContainerFinishedEvent.EventType: CONTAINER_FINISHED 2016-08-11 21:39:01,811 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0002_02_000011 Container Transitioned from ACQUIRED to EXPIRED 2016-08-11 21:39:01,811 INFO [main] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(159)) - USER=user2 OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_100_0002 CONTAINERID=container_100_0002_02_000011 RESOURCE= 2016-08-11 21:39:01,812 DEBUG [main] scheduler.SchedulerNode (SchedulerNode.java:releaseContainer(269)) - Released container container_100_0002_02_000011 of capacity on host 127.0.0.1:1, which currently has 6 containers, used and available, release resources=true 2016-08-11 21:39:01,812 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:39:01,812 DEBUG [main] capacity.LeafQueue (LeafQueue.java:releaseResource(1479)) - b used= numContainers=11 user=user2 user-resources= 2016-08-11 21:39:01,813 DEBUG [main] capacity.ParentQueue (ParentQueue.java:internalReleaseResource(728)) - completedContainer root: numChildQueue= 2, capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.8125, numApps=2, numContainers=12, cluster= 2016-08-11 21:39:01,813 DEBUG [main] capacity.ParentQueue (ParentQueue.java:internalReleaseResource(741)) - Re-sorting completed queue: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=3.4383395, absoluteUsedCapacity=0.6875, numApps=1, numContainers=11 2016-08-11 21:39:01,813 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1297)) - Trying to schedule on node: 127.0.0.1, available: 2016-08-11 21:39:01,813 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(437)) - Trying to assign containers to child-queue of root 2016-08-11 21:39:01,814 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.15625238), label=(*)root.busedCapacity=(3.4383395), label=(*) 2016-08-11 21:39:01,814 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 2016-08-11 21:39:01,814 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=127.0.0.1 #applications=1 2016-08-11 21:39:01,814 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user1 in queue a userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:39:01,815 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimitAndSetHeadroom(1120)) - Headroom calculation for user user1: userLimit= queueMaxAvailRes= consumed= headroom= 2016-08-11 21:39:01,815 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:assignContainers(511)) - pre-assignContainers for application application_100_0001 2016-08-11 21:39:01,815 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user1 in queue a userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:39:01,815 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0001 headRoom= currentConsumption=2048 2016-08-11 21:39:01,815 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0001 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:39:01,816 DEBUG [main] allocator.IncreaseContainerAllocator (IncreaseContainerAllocator.java:assignContainers(187)) - Skip allocating increase request since we don't have any increase request on this node=127.0.0.1:123 2016-08-11 21:39:01,816 DEBUG [main] allocator.RegularContainerAllocator (RegularContainerAllocator.java:assignContainer(473)) - assignContainers: node=127.0.0.1 application=application_100_0001 priority=1 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } type=OFF_SWITCH 2016-08-11 21:39:01,817 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0001_01_000002 of type RESERVED 2016-08-11 21:39:01,817 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0001_01_000002 Container Transitioned from NEW to RESERVED 2016-08-11 21:39:01,818 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:commonReserve(416)) - Application attempt appattempt_100_0001_000001 reserved container container_100_0001_01_000002 on node host: 127.0.0.1:1 #containers=6 available= used=. This attempt currently has 1 reserved containers at priority 1; currentReservation 2016-08-11 21:39:01,818 DEBUG [main] fica.FiCaSchedulerNode (FiCaSchedulerNode.java:reserveResource(92)) - Reserved container container_100_0001_01_000002 on node host: 127.0.0.1:1 #containers=6 available= used= for application attempt appattempt_100_0001_000001 2016-08-11 21:39:01,818 INFO [main] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(91)) - Reserved container application=application_100_0001 resource= queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@22356acd cluster= 2016-08-11 21:39:01,818 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(962)) - post-assignContainers for application application_100_0001 2016-08-11 21:39:01,818 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user1 in queue a userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:39:01,819 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0001 headRoom= currentConsumption=2048 2016-08-11 21:39:01,819 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0001 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:39:01,819 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user1 in queue a userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:39:01,819 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user1 in queue a userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:39:01,820 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user1 in queue a userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:39:01,820 DEBUG [main] capacity.LeafQueue (LeafQueue.java:allocateResource(1443)) - a user=user1 used= numContainers=2 headroom = user-resources= 2016-08-11 21:39:01,820 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.31250477, absoluteUsedCapacity=0.25, numApps=1, numContainers=2 --> , OFF_SWITCH 2016-08-11 21:39:01,820 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(670)) - Re-sorting assigned queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.31250477, absoluteUsedCapacity=0.25, numApps=1, numContainers=2 2016-08-11 21:39:01,820 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.31250477), label=(*)root.busedCapacity=(3.4383395), label=(*) 2016-08-11 21:39:01,821 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainers(522)) - assignedContainer queue=root usedCapacity=0.9375 absoluteUsedCapacity=0.9375 used= cluster= 2016-08-11 21:39:01,821 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(544)) - ParentQ=root assignedSoFarInThisIteration= usedCapacity=0.9375 absoluteUsedCapacity=0.9375 2016-08-11 21:39:01,821 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(555)) - Not assigning more than one off-switch container, assignments so far: resource:; type:OFF_SWITCH; excessReservation:null; applicationid:null; skipped:NONE; fulfilled reservation:false; allocations(count/resource):0/; reservations(count/resource):1/ 2016-08-11 21:39:01,821 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1297)) - Trying to schedule on node: test, available: 2016-08-11 21:39:01,821 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(437)) - Trying to assign containers to child-queue of root 2016-08-11 21:39:01,822 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.31250477), label=(*)root.busedCapacity=(3.4383395), label=(*) 2016-08-11 21:39:01,822 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.31250477, absoluteUsedCapacity=0.25, numApps=1, numContainers=2 2016-08-11 21:39:01,822 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=test #applications=1 2016-08-11 21:39:01,822 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user1 in queue a userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:39:01,822 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimitAndSetHeadroom(1120)) - Headroom calculation for user user1: userLimit= queueMaxAvailRes= consumed= headroom= 2016-08-11 21:39:01,823 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:assignContainers(511)) - pre-assignContainers for application application_100_0001 2016-08-11 21:39:01,823 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user1 in queue a userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:39:01,823 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0001 headRoom= currentConsumption=2048 2016-08-11 21:39:01,823 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0001 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:39:01,823 DEBUG [main] allocator.IncreaseContainerAllocator (IncreaseContainerAllocator.java:assignContainers(187)) - Skip allocating increase request since we don't have any increase request on this node=test:123 2016-08-11 21:39:01,823 DEBUG [main] allocator.RegularContainerAllocator (RegularContainerAllocator.java:assignContainer(473)) - assignContainers: node=test application=application_100_0001 priority=1 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } type=OFF_SWITCH 2016-08-11 21:39:01,824 DEBUG [main] allocator.RegularContainerAllocator (RegularContainerAllocator.java:shouldAllocOrReserveNewContainer(654)) - needsContainers: app.#re-reserve=0 reserved=1 nodeFactor=0.25 minAllocFactor=0.875 starvation=0 2016-08-11 21:39:01,824 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:getNodeIdToUnreserve(398)) - unreserving node with reservation size: in order to allocate container with size: 2016-08-11 21:39:01,824 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:findNodeToUnreserve(490)) - unreserving for app: application_100_0001 on nodeId: 127.0.0.1:123 in order to replace reserved application and place it on node: test:123 needing: 2016-08-11 21:39:01,824 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user1 in queue a userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:39:01,826 DEBUG [main] scheduler.ActiveUsersManager (ActiveUsersManager.java:deactivateApplication(94)) - User user1 removed from activeUsers, currently: 0 2016-08-11 21:39:01,827 DEBUG [main] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:allocate(631)) - allocate: applicationId=application_100_0001 container=container_100_0001_01_000003 host=test:123 user=user1 resource= type=OFF_SWITCH 2016-08-11 21:39:01,827 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0001_01_000003 of type START 2016-08-11 21:39:01,827 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0001_01_000003 Container Transitioned from NEW to ALLOCATED 2016-08-11 21:39:01,827 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: CONTAINER_ALLOCATED 2016-08-11 21:39:01,827 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:allocate(237)) - allocate: applicationAttemptId=appattempt_100_0001_000001 container=container_100_0001_01_000003 host=test type=OFF_SWITCH 2016-08-11 21:39:01,827 INFO [main] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(159)) - USER=user1 OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_100_0001 CONTAINERID=container_100_0001_01_000003 RESOURCE= 2016-08-11 21:39:01,828 DEBUG [main] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(157)) - Assigned container container_100_0001_01_000003 of capacity on host test:2, which has 7 containers, used and available after allocation 2016-08-11 21:39:01,828 INFO [main] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(120)) - assignedContainer application attempt=appattempt_100_0001_000001 container=container_100_0001_01_000003 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@22356acd clusterResource= type=OFF_SWITCH 2016-08-11 21:39:01,828 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(962)) - post-assignContainers for application application_100_0001 2016-08-11 21:39:01,829 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user1 in queue a userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:39:01,829 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0001 headRoom= currentConsumption=4096 2016-08-11 21:39:01,829 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0001 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 0, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:39:01,829 INFO [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:internalUnreserve(292)) - Application application_100_0001 unreserved on node host: 127.0.0.1:1 #containers=6 available= used=, currently has 0 at priority 1; currentReservation on node-label= 2016-08-11 21:39:01,830 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user1 in queue a userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:39:01,830 DEBUG [main] capacity.LeafQueue (LeafQueue.java:releaseResource(1479)) - a used= numContainers=1 user=user1 user-resources= 2016-08-11 21:39:01,831 DEBUG [main] capacity.ParentQueue (ParentQueue.java:internalReleaseResource(728)) - completedContainer root: numChildQueue= 2, capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.8125, numApps=2, numContainers=12, cluster= 2016-08-11 21:39:01,831 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user1 in queue a userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:39:01,831 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user1 in queue a userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:39:01,831 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user1 in queue a userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:39:01,831 DEBUG [main] capacity.LeafQueue (LeafQueue.java:allocateResource(1443)) - a user=user1 used= numContainers=2 headroom = user-resources= 2016-08-11 21:39:01,831 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.31250477, absoluteUsedCapacity=0.25, numApps=1, numContainers=2 --> , OFF_SWITCH 2016-08-11 21:39:01,831 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(670)) - Re-sorting assigned queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.31250477, absoluteUsedCapacity=0.25, numApps=1, numContainers=2 2016-08-11 21:39:01,832 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.31250477), label=(*)root.busedCapacity=(3.4383395), label=(*) 2016-08-11 21:39:01,832 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainers(522)) - assignedContainer queue=root usedCapacity=0.9375 absoluteUsedCapacity=0.9375 used= cluster= 2016-08-11 21:39:01,832 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(544)) - ParentQ=root assignedSoFarInThisIteration= usedCapacity=0.9375 absoluteUsedCapacity=0.9375 2016-08-11 21:39:01,832 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(555)) - Not assigning more than one off-switch container, assignments so far: resource:; type:OFF_SWITCH; excessReservation:null; applicationid:null; skipped:NONE; fulfilled reservation:false; allocations(count/resource):1/; reservations(count/resource):0/ 2016-08-11 21:39:01,832 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1297)) - Trying to schedule on node: 127.0.0.1, available: 2016-08-11 21:39:01,832 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(437)) - Trying to assign containers to child-queue of root 2016-08-11 21:39:01,833 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.31250477), label=(*)root.busedCapacity=(3.4383395), label=(*) 2016-08-11 21:39:01,833 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.31250477, absoluteUsedCapacity=0.25, numApps=1, numContainers=2 2016-08-11 21:39:01,833 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=127.0.0.1 #applications=1 2016-08-11 21:39:01,833 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(909)) - Skip this queue=root.a, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 21:39:01,833 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.31250477, absoluteUsedCapacity=0.25, numApps=1, numContainers=2 --> , NODE_LOCAL 2016-08-11 21:39:01,833 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=3.4383395, absoluteUsedCapacity=0.6875, numApps=1, numContainers=11 2016-08-11 21:39:01,834 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=127.0.0.1 #applications=1 2016-08-11 21:39:01,834 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:39:01,834 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimitAndSetHeadroom(1120)) - Headroom calculation for user user2: userLimit= queueMaxAvailRes= consumed= headroom= 2016-08-11 21:39:01,835 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:assignContainers(511)) - pre-assignContainers for application application_100_0002 2016-08-11 21:39:01,835 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 21:39:01,835 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=11264 2016-08-11 21:39:01,835 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:39:01,836 DEBUG [main] allocator.IncreaseContainerAllocator (IncreaseContainerAllocator.java:assignContainers(187)) - Skip allocating increase request since we don't have any increase request on this node=127.0.0.1:123 2016-08-11 21:39:01,836 DEBUG [main] allocator.RegularContainerAllocator (RegularContainerAllocator.java:assignContainer(473)) - assignContainers: node=127.0.0.1 application=application_100_0002 priority=1 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } type=OFF_SWITCH 2016-08-11 21:39:01,836 DEBUG [main] scheduler.ActiveUsersManager (ActiveUsersManager.java:deactivateApplication(94)) - User user2 removed from activeUsers, currently: 0 2016-08-11 21:39:01,842 DEBUG [main] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:allocate(631)) - allocate: applicationId=application_100_0002 container=container_100_0002_02_000014 host=127.0.0.1:123 user=user2 resource= type=OFF_SWITCH 2016-08-11 21:39:01,842 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0002_02_000014 of type START 2016-08-11 21:39:01,842 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0002_02_000014 Container Transitioned from NEW to ALLOCATED 2016-08-11 21:39:01,842 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: CONTAINER_ALLOCATED 2016-08-11 21:39:01,842 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:allocate(237)) - allocate: applicationAttemptId=appattempt_100_0002_000002 container=container_100_0002_02_000014 host=127.0.0.1 type=OFF_SWITCH 2016-08-11 21:39:01,846 INFO [main] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(159)) - USER=user2 OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_100_0002 CONTAINERID=container_100_0002_02_000014 RESOURCE= 2016-08-11 21:39:01,846 DEBUG [main] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(157)) - Assigned container container_100_0002_02_000014 of capacity on host 127.0.0.1:1, which has 7 containers, used and available after allocation 2016-08-11 21:39:01,847 INFO [main] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(120)) - assignedContainer application attempt=appattempt_100_0002_000002 container=container_100_0002_02_000014 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@386f0da3 clusterResource= type=OFF_SWITCH 2016-08-11 21:39:01,847 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(962)) - post-assignContainers for application application_100_0002 2016-08-11 21:39:01,848 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:39:01,848 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=12288 2016-08-11 21:39:01,848 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 0, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 21:39:01,849 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:39:01,849 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:39:01,849 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 21:39:01,850 DEBUG [main] capacity.LeafQueue (LeafQueue.java:allocateResource(1443)) - b user=user2 used= numContainers=12 headroom = user-resources= 2016-08-11 21:39:01,850 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=3.7509158, absoluteUsedCapacity=0.75, numApps=1, numContainers=12 --> , OFF_SWITCH 2016-08-11 21:39:01,850 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(670)) - Re-sorting assigned queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=3.7509158, absoluteUsedCapacity=0.75, numApps=1, numContainers=12 2016-08-11 21:39:01,851 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.31250477), label=(*)root.busedCapacity=(3.7509158), label=(*) 2016-08-11 21:39:01,851 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainers(522)) - assignedContainer queue=root usedCapacity=1.0 absoluteUsedCapacity=1.0 used= cluster= 2016-08-11 21:39:01,851 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(544)) - ParentQ=root assignedSoFarInThisIteration= usedCapacity=1.0 absoluteUsedCapacity=1.0 2016-08-11 21:39:01,851 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(555)) - Not assigning more than one off-switch container, assignments so far: resource:; type:OFF_SWITCH; excessReservation:null; applicationid:null; skipped:NONE; fulfilled reservation:false; allocations(count/resource):1/; reservations(count/resource):0/ 2016-08-11 21:39:01,857 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: ResourceManager entered state STOPPED 2016-08-11 21:39:01,857 DEBUG [main] service.CompositeService (CompositeService.java:serviceStop(129)) - ResourceManager: stopping services, size=3 2016-08-11 21:39:01,858 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #2: Service org.apache.hadoop.yarn.server.resourcemanager.ahs.RMApplicationHistoryWriter in state org.apache.hadoop.yarn.server.resourcemanager.ahs.RMApplicationHistoryWriter: STARTED 2016-08-11 21:39:01,859 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.ahs.RMApplicationHistoryWriter entered state STOPPED 2016-08-11 21:39:01,859 DEBUG [main] service.CompositeService (CompositeService.java:serviceStop(129)) - org.apache.hadoop.yarn.server.resourcemanager.ahs.RMApplicationHistoryWriter: stopping services, size=0 2016-08-11 21:39:01,859 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #1: Service org.apache.hadoop.yarn.server.resourcemanager.AdminService in state org.apache.hadoop.yarn.server.resourcemanager.AdminService: STARTED 2016-08-11 21:39:01,860 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.AdminService entered state STOPPED 2016-08-11 21:39:01,860 DEBUG [main] service.CompositeService (CompositeService.java:serviceStop(129)) - org.apache.hadoop.yarn.server.resourcemanager.AdminService: stopping services, size=0 2016-08-11 21:39:01,860 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #0: Service Dispatcher in state Dispatcher: STARTED 2016-08-11 21:39:01,860 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: Dispatcher entered state STOPPED 2016-08-11 21:39:01,861 INFO [main] resourcemanager.ResourceManager (ResourceManager.java:transitionToStandby(1108)) - Transitioning to standby state 2016-08-11 21:39:01,861 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: RMActiveServices entered state STOPPED 2016-08-11 21:39:01,862 DEBUG [main] service.CompositeService (CompositeService.java:serviceStop(129)) - RMActiveServices: stopping services, size=14 2016-08-11 21:39:01,862 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #13: Service org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher in state org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher: STARTED 2016-08-11 21:39:01,862 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher entered state STOPPED 2016-08-11 21:39:01,863 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #12: Service org.apache.hadoop.yarn.server.resourcemanager.ClientRMService in state org.apache.hadoop.yarn.server.resourcemanager.ClientRMService: STARTED 2016-08-11 21:39:01,863 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.ClientRMService entered state STOPPED 2016-08-11 21:39:01,863 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #11: Service org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService in state org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService: STARTED 2016-08-11 21:39:01,863 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService entered state STOPPED 2016-08-11 21:39:01,864 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #10: Service org.apache.hadoop.util.JvmPauseMonitor in state org.apache.hadoop.util.JvmPauseMonitor: STARTED 2016-08-11 21:39:01,864 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.util.JvmPauseMonitor entered state STOPPED 2016-08-11 21:39:01,864 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #9: Service org.apache.hadoop.yarn.server.resourcemanager.ResourceTrackerService in state org.apache.hadoop.yarn.server.resourcemanager.ResourceTrackerService: STARTED 2016-08-11 21:39:01,865 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.ResourceTrackerService entered state STOPPED 2016-08-11 21:39:01,865 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #8: Service NMLivelinessMonitor in state NMLivelinessMonitor: STARTED 2016-08-11 21:39:01,865 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: NMLivelinessMonitor entered state STOPPED 2016-08-11 21:39:01,865 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #7: Service SchedulerEventDispatcher in state SchedulerEventDispatcher: STARTED 2016-08-11 21:39:01,865 INFO [Ping Checker] util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(139)) - NMLivelinessMonitor thread interrupted 2016-08-11 21:39:01,865 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: SchedulerEventDispatcher entered state STOPPED 2016-08-11 21:39:01,866 ERROR [SchedulerEventDispatcher:Event Processor] event.EventDispatcher (EventDispatcher.java:run(61)) - Returning, interrupted : java.lang.InterruptedException 2016-08-11 21:39:01,866 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #6: Service org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler in state org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler: STARTED 2016-08-11 21:39:01,866 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler entered state STOPPED 2016-08-11 21:39:01,867 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #5: Service org.apache.hadoop.yarn.server.resourcemanager.NodesListManager in state org.apache.hadoop.yarn.server.resourcemanager.NodesListManager: STARTED 2016-08-11 21:39:01,867 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.NodesListManager entered state STOPPED 2016-08-11 21:39:01,867 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #4: Service org.apache.hadoop.yarn.nodelabels.CommonNodeLabelsManager in state org.apache.hadoop.yarn.nodelabels.CommonNodeLabelsManager: STARTED 2016-08-11 21:39:01,867 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.nodelabels.CommonNodeLabelsManager entered state STOPPED 2016-08-11 21:39:01,868 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #3: Service AMLivelinessMonitor in state AMLivelinessMonitor: STARTED 2016-08-11 21:39:01,868 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: AMLivelinessMonitor entered state STOPPED 2016-08-11 21:39:01,868 INFO [Ping Checker] util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(139)) - AMLivelinessMonitor thread interrupted 2016-08-11 21:39:01,868 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #2: Service AMLivelinessMonitor in state AMLivelinessMonitor: STARTED 2016-08-11 21:39:01,869 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: AMLivelinessMonitor entered state STOPPED 2016-08-11 21:39:01,869 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #1: Service org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.ContainerAllocationExpirer in state org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.ContainerAllocationExpirer: STARTED 2016-08-11 21:39:01,869 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.ContainerAllocationExpirer entered state STOPPED 2016-08-11 21:39:01,869 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #0: Service org.apache.hadoop.yarn.server.resourcemanager.RMSecretManagerService in state org.apache.hadoop.yarn.server.resourcemanager.RMSecretManagerService: STARTED 2016-08-11 21:39:01,870 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.RMSecretManagerService entered state STOPPED 2016-08-11 21:39:01,870 DEBUG [main] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:stopThreads(619)) - Stopping expired delegation token remover thread 2016-08-11 21:39:01,870 INFO [Ping Checker] util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(139)) - org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.ContainerAllocationExpirer thread interrupted 2016-08-11 21:39:01,870 ERROR [Thread[Thread-18,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(670)) - ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted 2016-08-11 21:39:01,870 INFO [Ping Checker] util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(139)) - AMLivelinessMonitor thread interrupted 2016-08-11 21:39:01,871 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:shutdown(587)) - refCount=1 2016-08-11 21:39:01,871 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stop(211)) - Stopping ResourceManager metrics system... 2016-08-11 21:39:01,871 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stopSources(459)) - Stopping metrics source UgiMetrics: class=class org.apache.hadoop.metrics2.lib.MetricsSourceBuilder$1 2016-08-11 21:39:01,872 DEBUG [main] util.MBeans (MBeans.java:unregister(105)) - Unregistering Hadoop:service=ResourceManager,name=UgiMetrics 2016-08-11 21:39:01,872 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stopSources(459)) - Stopping metrics source JvmMetrics: class=class org.apache.hadoop.metrics2.source.JvmMetrics 2016-08-11 21:39:01,872 DEBUG [main] util.MBeans (MBeans.java:unregister(105)) - Unregistering Hadoop:service=ResourceManager,name=JvmMetrics 2016-08-11 21:39:01,873 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stopSources(459)) - Stopping metrics source QueueMetrics,q0=root: class=class org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CSQueueMetrics 2016-08-11 21:39:01,873 DEBUG [main] util.MBeans (MBeans.java:unregister(105)) - Unregistering Hadoop:service=ResourceManager,name=QueueMetrics,q0=root 2016-08-11 21:39:01,873 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stopSources(459)) - Stopping metrics source QueueMetrics,q0=root,q1=a: class=class org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CSQueueMetrics 2016-08-11 21:39:01,874 DEBUG [main] util.MBeans (MBeans.java:unregister(105)) - Unregistering Hadoop:service=ResourceManager,name=QueueMetrics,q0=root,q1=a 2016-08-11 21:39:01,874 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stopSources(459)) - Stopping metrics source QueueMetrics,q0=root,q1=a,q2=a1: class=class org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CSQueueMetrics 2016-08-11 21:39:01,874 DEBUG [main] util.MBeans (MBeans.java:unregister(105)) - Unregistering Hadoop:service=ResourceManager,name=QueueMetrics,q0=root,q1=a,q2=a1 2016-08-11 21:39:01,874 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stopSources(459)) - Stopping metrics source QueueMetrics,q0=root,q1=a,q2=a2: class=class org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CSQueueMetrics 2016-08-11 21:39:01,874 DEBUG [main] util.MBeans (MBeans.java:unregister(105)) - Unregistering Hadoop:service=ResourceManager,name=QueueMetrics,q0=root,q1=a,q2=a2 2016-08-11 21:39:01,875 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stopSources(459)) - Stopping metrics source QueueMetrics,q0=root,q1=b: class=class org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CSQueueMetrics 2016-08-11 21:39:01,875 DEBUG [main] util.MBeans (MBeans.java:unregister(105)) - Unregistering Hadoop:service=ResourceManager,name=QueueMetrics,q0=root,q1=b 2016-08-11 21:39:01,875 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stopSources(459)) - Stopping metrics source QueueMetrics,q0=root,q1=b,q2=b1: class=class org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CSQueueMetrics 2016-08-11 21:39:01,875 DEBUG [main] util.MBeans (MBeans.java:unregister(105)) - Unregistering Hadoop:service=ResourceManager,name=QueueMetrics,q0=root,q1=b,q2=b1 2016-08-11 21:39:01,875 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stopSources(459)) - Stopping metrics source QueueMetrics,q0=root,q1=b,q2=b2: class=class org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CSQueueMetrics 2016-08-11 21:39:01,875 DEBUG [main] util.MBeans (MBeans.java:unregister(105)) - Unregistering Hadoop:service=ResourceManager,name=QueueMetrics,q0=root,q1=b,q2=b2 2016-08-11 21:39:01,876 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stopSources(459)) - Stopping metrics source QueueMetrics,q0=root,q1=b,q2=b3: class=class org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CSQueueMetrics 2016-08-11 21:39:01,876 DEBUG [main] util.MBeans (MBeans.java:unregister(105)) - Unregistering Hadoop:service=ResourceManager,name=QueueMetrics,q0=root,q1=b,q2=b3 2016-08-11 21:39:01,876 DEBUG [main] util.MBeans (MBeans.java:unregister(105)) - Unregistering Hadoop:service=ResourceManager,name=MetricsSystem,sub=Stats 2016-08-11 21:39:01,876 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stop(217)) - ResourceManager metrics system stopped. 2016-08-11 21:39:01,876 DEBUG [main] util.MBeans (MBeans.java:unregister(105)) - Unregistering Hadoop:service=ResourceManager,name=MetricsSystem,sub=Control 2016-08-11 21:39:01,877 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:shutdown(607)) - ResourceManager metrics system shutdown complete. 2016-08-11 21:39:01,877 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore entered state STOPPED 2016-08-11 21:39:01,877 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: Dispatcher entered state STOPPED 2016-08-11 21:39:01,877 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:serviceStop(145)) - AsyncDispatcher is draining to stop, igonring any new events. 2016-08-11 21:39:01,879 INFO [main] resourcemanager.ResourceManager (ResourceManager.java:transitionToStandby(1115)) - Transitioned to standby state 2016-08-11 21:39:01,879 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: ResourceManager entered state STOPPED 2016-08-11 21:39:01,879 DEBUG [main] service.CompositeService (CompositeService.java:serviceStop(129)) - ResourceManager: stopping services, size=3 2016-08-11 21:39:01,879 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #2: Service org.apache.hadoop.yarn.server.resourcemanager.ahs.RMApplicationHistoryWriter in state org.apache.hadoop.yarn.server.resourcemanager.ahs.RMApplicationHistoryWriter: INITED 2016-08-11 21:39:01,879 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.ahs.RMApplicationHistoryWriter entered state STOPPED 2016-08-11 21:39:01,879 DEBUG [main] service.CompositeService (CompositeService.java:serviceStop(129)) - org.apache.hadoop.yarn.server.resourcemanager.ahs.RMApplicationHistoryWriter: stopping services, size=0 2016-08-11 21:39:01,880 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #1: Service org.apache.hadoop.yarn.server.resourcemanager.AdminService in state org.apache.hadoop.yarn.server.resourcemanager.AdminService: INITED 2016-08-11 21:39:01,880 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.AdminService entered state STOPPED 2016-08-11 21:39:01,880 DEBUG [main] service.CompositeService (CompositeService.java:serviceStop(129)) - org.apache.hadoop.yarn.server.resourcemanager.AdminService: stopping services, size=0 2016-08-11 21:39:01,880 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #0: Service Dispatcher in state Dispatcher: STARTED 2016-08-11 21:39:01,880 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: Dispatcher entered state STOPPED 2016-08-11 21:39:01,880 INFO [main] resourcemanager.ResourceManager (ResourceManager.java:transitionToStandby(1108)) - Transitioning to standby state 2016-08-11 21:39:01,881 INFO [main] resourcemanager.ResourceManager (ResourceManager.java:transitionToStandby(1115)) - Transitioned to standby state