2016-08-11 02:31:22,298 INFO [main] capacity.TestCapacityScheduler (TestCapacityScheduler.java:setupQueueConfiguration(419)) - Setup top-level queues a and b 2016-08-11 02:31:22,371 INFO [main] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2434)) - found resource core-site.xml at file:/opt/hadooptrunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/target/test-classes/core-site.xml 2016-08-11 02:31:22,456 INFO [main] security.Groups (Groups.java:refresh(400)) - clearing userToGroupsMap cache 2016-08-11 02:31:22,525 INFO [main] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2434)) - found resource yarn-site.xml at file:/opt/hadooptrunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/target/test-classes/yarn-site.xml 2016-08-11 02:31:22,692 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(212)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.RMFatalEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMFatalEventDispatcher 2016-08-11 02:31:22,868 INFO [main] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:(75)) - NMTokenKeyRollingInterval: 86400000ms and NMTokenKeyActivationDelay: 900000ms 2016-08-11 02:31:22,871 INFO [main] security.RMContainerTokenSecretManager (RMContainerTokenSecretManager.java:(78)) - ContainerTokenKeyRollingInterval: 86400000ms and ContainerTokenKeyActivationDelay: 900000ms 2016-08-11 02:31:22,875 INFO [main] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:(94)) - AMRMTokenKeyRollingInterval: 86400000ms and AMRMTokenKeyActivationDelay: 900000 ms 2016-08-11 02:31:22,892 INFO [main] security.YarnAuthorizationProvider (YarnAuthorizationProvider.java:getInstance(57)) - org.apache.hadoop.yarn.security.ConfiguredYarnAuthorizer is instiantiated. 2016-08-11 02:31:22,907 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(212)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStoreEventType for class org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$ForwardingEventHandler 2016-08-11 02:31:22,910 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(212)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.NodesListManagerEventType for class org.apache.hadoop.yarn.server.resourcemanager.NodesListManager 2016-08-11 02:31:22,910 INFO [main] resourcemanager.ResourceManager (ResourceManager.java:createScheduler(396)) - Using Scheduler: org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler 2016-08-11 02:31:22,926 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(212)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.SchedulerEventType for class org.apache.hadoop.yarn.event.EventDispatcher 2016-08-11 02:31:22,927 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(212)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationEventDispatcher 2016-08-11 02:31:22,928 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(212)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationAttemptEventDispatcher 2016-08-11 02:31:22,929 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(212)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$NodeEventDispatcher 2016-08-11 02:31:22,968 WARN [main] impl.MetricsConfig (MetricsConfig.java:loadFirst(127)) - Cannot locate configuration: tried hadoop-metrics2-resourcemanager.properties,hadoop-metrics2.properties 2016-08-11 02:31:23,026 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:startTimer(375)) - Scheduled snapshot period at 10 second(s). 2016-08-11 02:31:23,026 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:start(192)) - ResourceManager metrics system started 2016-08-11 02:31:23,045 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(212)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.RMAppManagerEventType for class org.apache.hadoop.yarn.server.resourcemanager.RMAppManager 2016-08-11 02:31:23,054 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(212)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncherEventType for class org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher 2016-08-11 02:31:23,055 INFO [main] resourcemanager.RMNMInfo (RMNMInfo.java:(63)) - Registered RMNMInfo MBean 2016-08-11 02:31:23,057 INFO [main] util.HostsFileReader (HostsFileReader.java:refresh(160)) - Refreshing hosts (include/exclude) list 2016-08-11 02:31:23,063 INFO [main] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2434)) - found resource capacity-scheduler.xml at file:/opt/hadooptrunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/target/test-classes/capacity-scheduler.xml 2016-08-11 02:31:23,115 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(692)) - max alloc mb per queue for root is undefined 2016-08-11 02:31:23,115 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(696)) - max alloc vcore per queue for root is undefined 2016-08-11 02:31:23,124 INFO [main] capacity.ParentQueue (ParentQueue.java:setupQueueConfigs(126)) - root, capacity=1.0, asboluteCapacity=1.0, maxCapacity=1.0, asboluteMaxCapacity=1.0, state=RUNNING, acls=SUBMIT_APP:*ADMINISTER_QUEUE:*, labels=*, , reservationsContinueLooking=true 2016-08-11 02:31:23,124 INFO [main] capacity.ParentQueue (ParentQueue.java:(105)) - Initialized parent-queue root name=root, fullname=root 2016-08-11 02:31:23,127 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(692)) - max alloc mb per queue for root.a is undefined 2016-08-11 02:31:23,127 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(696)) - max alloc vcore per queue for root.a is undefined 2016-08-11 02:31:23,127 INFO [main] capacity.ParentQueue (ParentQueue.java:setupQueueConfigs(126)) - a, capacity=0.105, asboluteCapacity=0.105, maxCapacity=1.0, asboluteMaxCapacity=1.0, state=RUNNING, acls=SUBMIT_APP: ADMINISTER_QUEUE: , labels=*, , reservationsContinueLooking=true 2016-08-11 02:31:23,128 INFO [main] capacity.ParentQueue (ParentQueue.java:(105)) - Initialized parent-queue a name=a, fullname=root.a 2016-08-11 02:31:23,136 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(692)) - max alloc mb per queue for root.a.a1 is undefined 2016-08-11 02:31:23,137 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(696)) - max alloc vcore per queue for root.a.a1 is undefined 2016-08-11 02:31:23,142 INFO [main] capacity.LeafQueue (LeafQueue.java:setupQueueConfigs(219)) - Initializing a1 capacity = 0.3 [= (float) configuredCapacity / 100 ] asboluteCapacity = 0.0315 [= parentAbsoluteCapacity * capacity ] maxCapacity = 1.0 [= configuredMaxCapacity ] absoluteMaxCapacity = 1.0 [= 1.0 maximumCapacity undefined, (parentAbsoluteMaxCapacity * maximumCapacity) / 100 otherwise ] userLimit = 100 [= configuredUserLimit ] userLimitFactor = 100.0 [= configuredUserLimitFactor ] maxApplications = 315 [= configuredMaximumSystemApplicationsPerQueue or (int)(configuredMaximumSystemApplications * absoluteCapacity)] maxApplicationsPerUser = 315 [= (int)(maxApplications * (userLimit / 100.0f) * userLimitFactor) ] usedCapacity = 0.0 [= usedResourcesMemory / (clusterResourceMemory * absoluteCapacity)] absoluteUsedCapacity = 0.0 [= usedResourcesMemory / clusterResourceMemory] maxAMResourcePerQueuePercent = 0.1 [= configuredMaximumAMResourcePercent ] minimumAllocationFactor = 0.875 [= (float)(maximumAllocationMemory - minimumAllocationMemory) / maximumAllocationMemory ] maximumAllocation = [= configuredMaxAllocation ] numContainers = 0 [= currentNumContainers ] state = RUNNING [= configuredState ] acls = SUBMIT_APP: ADMINISTER_QUEUE: [= configuredAcls ] nodeLocalityDelay = 40 labels=*, reservationsContinueLooking = true preemptionDisabled = true defaultAppPriorityPerQueue = 0 2016-08-11 02:31:23,142 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:parseQueue(669)) - Initialized queue: a1: capacity=0.3, absoluteCapacity=0.0315, usedResources=, usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=0, numContainers=0 2016-08-11 02:31:23,145 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(692)) - max alloc mb per queue for root.a.a2 is undefined 2016-08-11 02:31:23,146 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(696)) - max alloc vcore per queue for root.a.a2 is undefined 2016-08-11 02:31:23,147 INFO [main] capacity.LeafQueue (LeafQueue.java:setupQueueConfigs(219)) - Initializing a2 capacity = 0.7 [= (float) configuredCapacity / 100 ] asboluteCapacity = 0.0735 [= parentAbsoluteCapacity * capacity ] maxCapacity = 1.0 [= configuredMaxCapacity ] absoluteMaxCapacity = 1.0 [= 1.0 maximumCapacity undefined, (parentAbsoluteMaxCapacity * maximumCapacity) / 100 otherwise ] userLimit = 100 [= configuredUserLimit ] userLimitFactor = 100.0 [= configuredUserLimitFactor ] maxApplications = 735 [= configuredMaximumSystemApplicationsPerQueue or (int)(configuredMaximumSystemApplications * absoluteCapacity)] maxApplicationsPerUser = 735 [= (int)(maxApplications * (userLimit / 100.0f) * userLimitFactor) ] usedCapacity = 0.0 [= usedResourcesMemory / (clusterResourceMemory * absoluteCapacity)] absoluteUsedCapacity = 0.0 [= usedResourcesMemory / clusterResourceMemory] maxAMResourcePerQueuePercent = 0.1 [= configuredMaximumAMResourcePercent ] minimumAllocationFactor = 0.875 [= (float)(maximumAllocationMemory - minimumAllocationMemory) / maximumAllocationMemory ] maximumAllocation = [= configuredMaxAllocation ] numContainers = 0 [= currentNumContainers ] state = RUNNING [= configuredState ] acls = SUBMIT_APP: ADMINISTER_QUEUE: [= configuredAcls ] nodeLocalityDelay = 40 labels=*, reservationsContinueLooking = true preemptionDisabled = true defaultAppPriorityPerQueue = 0 2016-08-11 02:31:23,148 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:parseQueue(669)) - Initialized queue: a2: capacity=0.7, absoluteCapacity=0.0735, usedResources=, usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=0, numContainers=0 2016-08-11 02:31:23,148 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:parseQueue(669)) - Initialized queue: a: numChildQueue= 2, capacity=0.105, absoluteCapacity=0.105, usedResources=usedCapacity=0.0, numApps=0, numContainers=0 2016-08-11 02:31:23,152 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(692)) - max alloc mb per queue for root.b is undefined 2016-08-11 02:31:23,153 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(696)) - max alloc vcore per queue for root.b is undefined 2016-08-11 02:31:23,153 INFO [main] capacity.ParentQueue (ParentQueue.java:setupQueueConfigs(126)) - b, capacity=0.895, asboluteCapacity=0.895, maxCapacity=1.0, asboluteMaxCapacity=1.0, state=RUNNING, acls=SUBMIT_APP: ADMINISTER_QUEUE: , labels=*, , reservationsContinueLooking=true 2016-08-11 02:31:23,154 INFO [main] capacity.ParentQueue (ParentQueue.java:(105)) - Initialized parent-queue b name=b, fullname=root.b 2016-08-11 02:31:23,157 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(692)) - max alloc mb per queue for root.b.b1 is undefined 2016-08-11 02:31:23,157 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(696)) - max alloc vcore per queue for root.b.b1 is undefined 2016-08-11 02:31:23,159 INFO [main] capacity.LeafQueue (LeafQueue.java:setupQueueConfigs(219)) - Initializing b1 capacity = 0.792 [= (float) configuredCapacity / 100 ] asboluteCapacity = 0.70883995 [= parentAbsoluteCapacity * capacity ] maxCapacity = 1.0 [= configuredMaxCapacity ] absoluteMaxCapacity = 1.0 [= 1.0 maximumCapacity undefined, (parentAbsoluteMaxCapacity * maximumCapacity) / 100 otherwise ] userLimit = 100 [= configuredUserLimit ] userLimitFactor = 100.0 [= configuredUserLimitFactor ] maxApplications = 7088 [= configuredMaximumSystemApplicationsPerQueue or (int)(configuredMaximumSystemApplications * absoluteCapacity)] maxApplicationsPerUser = 7088 [= (int)(maxApplications * (userLimit / 100.0f) * userLimitFactor) ] usedCapacity = 0.0 [= usedResourcesMemory / (clusterResourceMemory * absoluteCapacity)] absoluteUsedCapacity = 0.0 [= usedResourcesMemory / clusterResourceMemory] maxAMResourcePerQueuePercent = 0.1 [= configuredMaximumAMResourcePercent ] minimumAllocationFactor = 0.875 [= (float)(maximumAllocationMemory - minimumAllocationMemory) / maximumAllocationMemory ] maximumAllocation = [= configuredMaxAllocation ] numContainers = 0 [= currentNumContainers ] state = RUNNING [= configuredState ] acls = SUBMIT_APP: ADMINISTER_QUEUE: [= configuredAcls ] nodeLocalityDelay = 40 labels=*, reservationsContinueLooking = true preemptionDisabled = true defaultAppPriorityPerQueue = 0 2016-08-11 02:31:23,159 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:parseQueue(669)) - Initialized queue: b1: capacity=0.792, absoluteCapacity=0.70883995, usedResources=, usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=0, numContainers=0 2016-08-11 02:31:23,163 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(692)) - max alloc mb per queue for root.b.b2 is undefined 2016-08-11 02:31:23,163 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(696)) - max alloc vcore per queue for root.b.b2 is undefined 2016-08-11 02:31:23,165 INFO [main] capacity.LeafQueue (LeafQueue.java:setupQueueConfigs(219)) - Initializing b2 capacity = 0.008 [= (float) configuredCapacity / 100 ] asboluteCapacity = 0.00716 [= parentAbsoluteCapacity * capacity ] maxCapacity = 1.0 [= configuredMaxCapacity ] absoluteMaxCapacity = 1.0 [= 1.0 maximumCapacity undefined, (parentAbsoluteMaxCapacity * maximumCapacity) / 100 otherwise ] userLimit = 100 [= configuredUserLimit ] userLimitFactor = 100.0 [= configuredUserLimitFactor ] maxApplications = 71 [= configuredMaximumSystemApplicationsPerQueue or (int)(configuredMaximumSystemApplications * absoluteCapacity)] maxApplicationsPerUser = 71 [= (int)(maxApplications * (userLimit / 100.0f) * userLimitFactor) ] usedCapacity = 0.0 [= usedResourcesMemory / (clusterResourceMemory * absoluteCapacity)] absoluteUsedCapacity = 0.0 [= usedResourcesMemory / clusterResourceMemory] maxAMResourcePerQueuePercent = 0.1 [= configuredMaximumAMResourcePercent ] minimumAllocationFactor = 0.875 [= (float)(maximumAllocationMemory - minimumAllocationMemory) / maximumAllocationMemory ] maximumAllocation = [= configuredMaxAllocation ] numContainers = 0 [= currentNumContainers ] state = RUNNING [= configuredState ] acls = SUBMIT_APP: ADMINISTER_QUEUE: [= configuredAcls ] nodeLocalityDelay = 40 labels=*, reservationsContinueLooking = true preemptionDisabled = true defaultAppPriorityPerQueue = 0 2016-08-11 02:31:23,165 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:parseQueue(669)) - Initialized queue: b2: capacity=0.008, absoluteCapacity=0.00716, usedResources=, usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=0, numContainers=0 2016-08-11 02:31:23,168 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(692)) - max alloc mb per queue for root.b.b3 is undefined 2016-08-11 02:31:23,169 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(696)) - max alloc vcore per queue for root.b.b3 is undefined 2016-08-11 02:31:23,170 INFO [main] capacity.LeafQueue (LeafQueue.java:setupQueueConfigs(219)) - Initializing b3 capacity = 0.2 [= (float) configuredCapacity / 100 ] asboluteCapacity = 0.179 [= parentAbsoluteCapacity * capacity ] maxCapacity = 1.0 [= configuredMaxCapacity ] absoluteMaxCapacity = 1.0 [= 1.0 maximumCapacity undefined, (parentAbsoluteMaxCapacity * maximumCapacity) / 100 otherwise ] userLimit = 100 [= configuredUserLimit ] userLimitFactor = 100.0 [= configuredUserLimitFactor ] maxApplications = 1790 [= configuredMaximumSystemApplicationsPerQueue or (int)(configuredMaximumSystemApplications * absoluteCapacity)] maxApplicationsPerUser = 1790 [= (int)(maxApplications * (userLimit / 100.0f) * userLimitFactor) ] usedCapacity = 0.0 [= usedResourcesMemory / (clusterResourceMemory * absoluteCapacity)] absoluteUsedCapacity = 0.0 [= usedResourcesMemory / clusterResourceMemory] maxAMResourcePerQueuePercent = 0.1 [= configuredMaximumAMResourcePercent ] minimumAllocationFactor = 0.875 [= (float)(maximumAllocationMemory - minimumAllocationMemory) / maximumAllocationMemory ] maximumAllocation = [= configuredMaxAllocation ] numContainers = 0 [= currentNumContainers ] state = RUNNING [= configuredState ] acls = SUBMIT_APP: ADMINISTER_QUEUE: [= configuredAcls ] nodeLocalityDelay = 40 labels=*, reservationsContinueLooking = true preemptionDisabled = true defaultAppPriorityPerQueue = 0 2016-08-11 02:31:23,171 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:parseQueue(669)) - Initialized queue: b3: capacity=0.2, absoluteCapacity=0.179, usedResources=, usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=0, numContainers=0 2016-08-11 02:31:23,171 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:parseQueue(669)) - Initialized queue: b: numChildQueue= 3, capacity=0.895, absoluteCapacity=0.895, usedResources=usedCapacity=0.0, numApps=0, numContainers=0 2016-08-11 02:31:23,171 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:parseQueue(669)) - Initialized queue: root: numChildQueue= 2, capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.0, numApps=0, numContainers=0 2016-08-11 02:31:23,173 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:initializeQueues(504)) - Initialized root queue root: numChildQueue= 2, capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.0, numApps=0, numContainers=0 2016-08-11 02:31:23,173 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:getUserGroupMappingPlacementRule(452)) - Initialized queue mappings, override: false 2016-08-11 02:31:23,175 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:initScheduler(313)) - Initialized CapacityScheduler with calculator=class org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator, minimumAllocation=<>, maximumAllocation=<>, asynchronousScheduling=false, asyncScheduleInterval=5ms 2016-08-11 02:31:23,180 INFO [main] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2431)) - dynamic-resources.xml not found 2016-08-11 02:31:23,192 INFO [main] resourcemanager.ResourceManager (ResourceManager.java:createSystemMetricsPublisher(486)) - TimelineServicePublisher is not configured 2016-08-11 02:31:23,193 INFO [main] security.RMContainerTokenSecretManager (RMContainerTokenSecretManager.java:rollMasterKey(106)) - Rolling master-key for container-tokens 2016-08-11 02:31:23,209 INFO [main] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:rollMasterKey(95)) - Rolling master-key for nm-tokens 2016-08-11 02:31:23,352 INFO [main] capacity.TestCapacityScheduler (TestCapacityScheduler.java:setupBlockedQueueConfiguration(436)) - Setup top-level queues a and b 2016-08-11 02:31:23,387 INFO [main] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2434)) - found resource core-site.xml at file:/opt/hadooptrunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/target/test-classes/core-site.xml 2016-08-11 02:31:23,410 INFO [main] security.Groups (Groups.java:refresh(400)) - clearing userToGroupsMap cache 2016-08-11 02:31:23,413 INFO [main] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2434)) - found resource yarn-site.xml at file:/opt/hadooptrunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/target/test-classes/yarn-site.xml 2016-08-11 02:31:23,438 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(212)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.RMFatalEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMFatalEventDispatcher 2016-08-11 02:31:23,442 INFO [main] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:(75)) - NMTokenKeyRollingInterval: 86400000ms and NMTokenKeyActivationDelay: 900000ms 2016-08-11 02:31:23,443 INFO [main] security.RMContainerTokenSecretManager (RMContainerTokenSecretManager.java:(78)) - ContainerTokenKeyRollingInterval: 86400000ms and ContainerTokenKeyActivationDelay: 900000ms 2016-08-11 02:31:23,443 INFO [main] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:(94)) - AMRMTokenKeyRollingInterval: 86400000ms and AMRMTokenKeyActivationDelay: 900000 ms 2016-08-11 02:31:23,445 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(212)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStoreEventType for class org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$ForwardingEventHandler 2016-08-11 02:31:23,446 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(212)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.NodesListManagerEventType for class org.apache.hadoop.yarn.server.resourcemanager.NodesListManager 2016-08-11 02:31:23,446 INFO [main] resourcemanager.ResourceManager (ResourceManager.java:createScheduler(396)) - Using Scheduler: org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler 2016-08-11 02:31:23,447 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(212)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.SchedulerEventType for class org.apache.hadoop.yarn.event.EventDispatcher 2016-08-11 02:31:23,447 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(212)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationEventDispatcher 2016-08-11 02:31:23,447 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(212)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationAttemptEventDispatcher 2016-08-11 02:31:23,447 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(212)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$NodeEventDispatcher 2016-08-11 02:31:23,447 INFO [main] security.RMContainerTokenSecretManager (RMContainerTokenSecretManager.java:rollMasterKey(106)) - Rolling master-key for container-tokens 2016-08-11 02:31:23,448 INFO [main] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:rollMasterKey(95)) - Rolling master-key for nm-tokens 2016-08-11 02:31:23,448 WARN [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:init(152)) - ResourceManager metrics system already initialized! 2016-08-11 02:31:23,448 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(212)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.RMAppManagerEventType for class org.apache.hadoop.yarn.server.resourcemanager.RMAppManager 2016-08-11 02:31:23,449 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(212)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncherEventType for class org.apache.hadoop.yarn.server.resourcemanager.MockRM$6 2016-08-11 02:31:23,450 WARN [main] util.MBeans (MBeans.java:getMBeanName(125)) - Error creating MBean object name: Hadoop:service=ResourceManager,name=RMNMInfo org.apache.hadoop.metrics2.MetricsException: org.apache.hadoop.metrics2.MetricsException: Hadoop:service=ResourceManager,name=RMNMInfo already exists! at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.newObjectName(DefaultMetricsSystem.java:135) at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.newMBeanName(DefaultMetricsSystem.java:110) at org.apache.hadoop.metrics2.util.MBeans.getMBeanName(MBeans.java:123) at org.apache.hadoop.metrics2.util.MBeans.register(MBeans.java:64) at org.apache.hadoop.yarn.server.resourcemanager.RMNMInfo.(RMNMInfo.java:59) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceInit(ResourceManager.java:692) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAndInitActiveServices(ResourceManager.java:1037) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:294) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.yarn.server.resourcemanager.MockRM.(MockRM.java:135) at org.apache.hadoop.yarn.server.resourcemanager.MockRM.(MockRM.java:128) at org.apache.hadoop.yarn.server.resourcemanager.MockRM.(MockRM.java:124) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler.testCSQueueBlocked(TestCapacityScheduler.java:3594) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229) at org.junit.runners.ParentRunner.run(ParentRunner.java:309) at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:254) at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:149) at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:124) at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:200) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:153) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103) Caused by: org.apache.hadoop.metrics2.MetricsException: Hadoop:service=ResourceManager,name=RMNMInfo already exists! at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.newObjectName(DefaultMetricsSystem.java:131) ... 38 more 2016-08-11 02:31:23,454 INFO [main] resourcemanager.RMNMInfo (RMNMInfo.java:(63)) - Registered RMNMInfo MBean 2016-08-11 02:31:23,455 INFO [main] util.HostsFileReader (HostsFileReader.java:refresh(160)) - Refreshing hosts (include/exclude) list 2016-08-11 02:31:23,457 INFO [main] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2434)) - found resource capacity-scheduler.xml at file:/opt/hadooptrunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/target/test-classes/capacity-scheduler.xml 2016-08-11 02:31:23,483 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(692)) - max alloc mb per queue for root is undefined 2016-08-11 02:31:23,483 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(696)) - max alloc vcore per queue for root is undefined 2016-08-11 02:31:23,483 INFO [main] capacity.ParentQueue (ParentQueue.java:setupQueueConfigs(126)) - root, capacity=1.0, asboluteCapacity=1.0, maxCapacity=1.0, asboluteMaxCapacity=1.0, state=RUNNING, acls=SUBMIT_APP:*ADMINISTER_QUEUE:*, labels=*, , reservationsContinueLooking=true 2016-08-11 02:31:23,484 INFO [main] capacity.ParentQueue (ParentQueue.java:(105)) - Initialized parent-queue root name=root, fullname=root 2016-08-11 02:31:23,485 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(692)) - max alloc mb per queue for root.a is undefined 2016-08-11 02:31:23,485 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(696)) - max alloc vcore per queue for root.a is undefined 2016-08-11 02:31:23,487 INFO [main] capacity.LeafQueue (LeafQueue.java:setupQueueConfigs(219)) - Initializing a capacity = 0.8 [= (float) configuredCapacity / 100 ] asboluteCapacity = 0.8 [= parentAbsoluteCapacity * capacity ] maxCapacity = 1.0 [= configuredMaxCapacity ] absoluteMaxCapacity = 1.0 [= 1.0 maximumCapacity undefined, (parentAbsoluteMaxCapacity * maximumCapacity) / 100 otherwise ] userLimit = 100 [= configuredUserLimit ] userLimitFactor = 100.0 [= configuredUserLimitFactor ] maxApplications = 8000 [= configuredMaximumSystemApplicationsPerQueue or (int)(configuredMaximumSystemApplications * absoluteCapacity)] maxApplicationsPerUser = 8000 [= (int)(maxApplications * (userLimit / 100.0f) * userLimitFactor) ] usedCapacity = 0.0 [= usedResourcesMemory / (clusterResourceMemory * absoluteCapacity)] absoluteUsedCapacity = 0.0 [= usedResourcesMemory / clusterResourceMemory] maxAMResourcePerQueuePercent = 0.1 [= configuredMaximumAMResourcePercent ] minimumAllocationFactor = 0.875 [= (float)(maximumAllocationMemory - minimumAllocationMemory) / maximumAllocationMemory ] maximumAllocation = [= configuredMaxAllocation ] numContainers = 0 [= currentNumContainers ] state = RUNNING [= configuredState ] acls = SUBMIT_APP: ADMINISTER_QUEUE: [= configuredAcls ] nodeLocalityDelay = 40 labels=*, reservationsContinueLooking = true preemptionDisabled = true defaultAppPriorityPerQueue = 0 2016-08-11 02:31:23,487 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:parseQueue(669)) - Initialized queue: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=0, numContainers=0 2016-08-11 02:31:23,489 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(692)) - max alloc mb per queue for root.b is undefined 2016-08-11 02:31:23,489 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(696)) - max alloc vcore per queue for root.b is undefined 2016-08-11 02:31:23,491 INFO [main] capacity.LeafQueue (LeafQueue.java:setupQueueConfigs(219)) - Initializing b capacity = 0.2 [= (float) configuredCapacity / 100 ] asboluteCapacity = 0.2 [= parentAbsoluteCapacity * capacity ] maxCapacity = 1.0 [= configuredMaxCapacity ] absoluteMaxCapacity = 1.0 [= 1.0 maximumCapacity undefined, (parentAbsoluteMaxCapacity * maximumCapacity) / 100 otherwise ] userLimit = 100 [= configuredUserLimit ] userLimitFactor = 100.0 [= configuredUserLimitFactor ] maxApplications = 2000 [= configuredMaximumSystemApplicationsPerQueue or (int)(configuredMaximumSystemApplications * absoluteCapacity)] maxApplicationsPerUser = 2000 [= (int)(maxApplications * (userLimit / 100.0f) * userLimitFactor) ] usedCapacity = 0.0 [= usedResourcesMemory / (clusterResourceMemory * absoluteCapacity)] absoluteUsedCapacity = 0.0 [= usedResourcesMemory / clusterResourceMemory] maxAMResourcePerQueuePercent = 0.1 [= configuredMaximumAMResourcePercent ] minimumAllocationFactor = 0.875 [= (float)(maximumAllocationMemory - minimumAllocationMemory) / maximumAllocationMemory ] maximumAllocation = [= configuredMaxAllocation ] numContainers = 0 [= currentNumContainers ] state = RUNNING [= configuredState ] acls = SUBMIT_APP: ADMINISTER_QUEUE: [= configuredAcls ] nodeLocalityDelay = 40 labels=*, reservationsContinueLooking = true preemptionDisabled = true defaultAppPriorityPerQueue = 0 2016-08-11 02:31:23,491 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:parseQueue(669)) - Initialized queue: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=0, numContainers=0 2016-08-11 02:31:23,491 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:parseQueue(669)) - Initialized queue: root: numChildQueue= 2, capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.0, numApps=0, numContainers=0 2016-08-11 02:31:23,492 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:initializeQueues(504)) - Initialized root queue root: numChildQueue= 2, capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.0, numApps=0, numContainers=0 2016-08-11 02:31:23,492 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:getUserGroupMappingPlacementRule(452)) - Initialized queue mappings, override: false 2016-08-11 02:31:23,493 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:initScheduler(313)) - Initialized CapacityScheduler with calculator=class org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator, minimumAllocation=<>, maximumAllocation=<>, asynchronousScheduling=false, asyncScheduleInterval=5ms 2016-08-11 02:31:23,493 INFO [main] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2431)) - dynamic-resources.xml not found 2016-08-11 02:31:23,495 INFO [main] resourcemanager.ResourceManager (ResourceManager.java:createSystemMetricsPublisher(486)) - TimelineServicePublisher is not configured 2016-08-11 02:31:23,496 INFO [main] resourcemanager.ResourceManager (ResourceManager.java:transitionToActive(1081)) - Transitioning to active state 2016-08-11 02:31:23,497 DEBUG [main] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1817)) - PrivilegedAction as:root (auth:SIMPLE) from:org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.transitionToActive(ResourceManager.java:1083) 2016-08-11 02:31:23,501 DEBUG [main] service.AbstractService (AbstractService.java:start(197)) - Service Dispatcher is started 2016-08-11 02:31:23,501 DEBUG [main] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore is started 2016-08-11 02:31:23,501 DEBUG [main] service.CompositeService (CompositeService.java:serviceStart(115)) - RMActiveServices: starting services, size=14 2016-08-11 02:31:23,507 DEBUG [main] recovery.RMStateStore (RMStateStore.java:handleStoreEvent(1070)) - Processing event of type UPDATE_AMRM_TOKEN 2016-08-11 02:31:23,507 INFO [main] recovery.RMStateStore (RMStateStore.java:transition(487)) - Updating AMRMToken 2016-08-11 02:31:23,508 INFO [main] security.RMContainerTokenSecretManager (RMContainerTokenSecretManager.java:rollMasterKey(106)) - Rolling master-key for container-tokens 2016-08-11 02:31:23,508 INFO [main] security.RMContainerTokenSecretManager (RMContainerTokenSecretManager.java:rollMasterKey(111)) - Going to activate master-key with key-id 100511641 in 900000ms 2016-08-11 02:31:23,508 INFO [main] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:rollMasterKey(95)) - Rolling master-key for nm-tokens 2016-08-11 02:31:23,509 INFO [main] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:rollMasterKey(100)) - Going to activate master-key with key-id -2002122830 in 900000ms 2016-08-11 02:31:23,509 INFO [main] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2016-08-11 02:31:23,510 INFO [main] security.RMDelegationTokenSecretManager (RMDelegationTokenSecretManager.java:storeNewMasterKey(87)) - storing master key with keyID 1 2016-08-11 02:31:23,510 DEBUG [main] recovery.RMStateStore (RMStateStore.java:handleStoreEvent(1070)) - Processing event of type STORE_MASTERKEY 2016-08-11 02:31:23,511 INFO [main] recovery.RMStateStore (RMStateStore.java:transition(438)) - Storing RMDTMasterKey. 2016-08-11 02:31:23,512 DEBUG [main] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.RMSecretManagerService is started 2016-08-11 02:31:23,512 INFO [Thread[Thread-18,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2016-08-11 02:31:23,512 INFO [Thread[Thread-18,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2016-08-11 02:31:23,512 INFO [Thread[Thread-18,5,main]] security.RMDelegationTokenSecretManager (RMDelegationTokenSecretManager.java:storeNewMasterKey(87)) - storing master key with keyID 2 2016-08-11 02:31:23,512 DEBUG [Thread[Thread-18,5,main]] recovery.RMStateStore (RMStateStore.java:handleStoreEvent(1070)) - Processing event of type STORE_MASTERKEY 2016-08-11 02:31:23,513 INFO [Thread[Thread-18,5,main]] recovery.RMStateStore (RMStateStore.java:transition(438)) - Storing RMDTMasterKey. 2016-08-11 02:31:23,513 DEBUG [main] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.ContainerAllocationExpirer is started 2016-08-11 02:31:23,514 DEBUG [main] service.AbstractService (AbstractService.java:start(197)) - Service AMLivelinessMonitor is started 2016-08-11 02:31:23,514 DEBUG [main] service.AbstractService (AbstractService.java:start(197)) - Service AMLivelinessMonitor is started 2016-08-11 02:31:23,515 DEBUG [main] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.nodelabels.CommonNodeLabelsManager is started 2016-08-11 02:31:23,515 DEBUG [main] service.CompositeService (CompositeService.java:serviceStart(115)) - org.apache.hadoop.yarn.server.resourcemanager.NodesListManager: starting services, size=0 2016-08-11 02:31:23,515 DEBUG [main] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.NodesListManager is started 2016-08-11 02:31:23,516 DEBUG [main] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.scheduler.activities.ActivitiesManager is started 2016-08-11 02:31:23,516 DEBUG [main] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler is started 2016-08-11 02:31:23,516 DEBUG [main] service.AbstractService (AbstractService.java:start(197)) - Service SchedulerEventDispatcher is started 2016-08-11 02:31:23,517 DEBUG [main] service.AbstractService (AbstractService.java:start(197)) - Service NMLivelinessMonitor is started 2016-08-11 02:31:23,517 DEBUG [main] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.ResourceTrackerService is started 2016-08-11 02:31:23,518 DEBUG [main] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.util.JvmPauseMonitor is started 2016-08-11 02:31:23,518 DEBUG [main] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService is started 2016-08-11 02:31:23,518 DEBUG [main] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.ClientRMService is started 2016-08-11 02:31:23,518 DEBUG [main] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher is started 2016-08-11 02:31:23,518 DEBUG [main] service.AbstractService (AbstractService.java:start(197)) - Service RMActiveServices is started 2016-08-11 02:31:23,519 INFO [main] resourcemanager.ResourceManager (ResourceManager.java:transitionToActive(1097)) - Transitioned to active state 2016-08-11 02:31:23,519 INFO [org.apache.hadoop.util.JvmPauseMonitor$Monitor@6f44a157] util.JvmPauseMonitor (JvmPauseMonitor.java:run(188)) - Starting JVM pause monitor 2016-08-11 02:31:23,519 DEBUG [main] service.CompositeService (CompositeService.java:serviceStart(115)) - ResourceManager: starting services, size=3 2016-08-11 02:31:23,520 DEBUG [main] service.AbstractService (AbstractService.java:start(197)) - Service Dispatcher is started 2016-08-11 02:31:23,520 DEBUG [main] service.CompositeService (CompositeService.java:serviceStart(115)) - org.apache.hadoop.yarn.server.resourcemanager.AdminService: starting services, size=0 2016-08-11 02:31:23,520 DEBUG [main] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.AdminService is started 2016-08-11 02:31:23,520 DEBUG [main] service.CompositeService (CompositeService.java:serviceStart(115)) - org.apache.hadoop.yarn.server.resourcemanager.ahs.RMApplicationHistoryWriter: starting services, size=0 2016-08-11 02:31:23,520 DEBUG [main] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.ahs.RMApplicationHistoryWriter is started 2016-08-11 02:31:23,520 DEBUG [main] service.AbstractService (AbstractService.java:start(197)) - Service ResourceManager is started 2016-08-11 02:31:23,531 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeLabelsUpdateSchedulerEvent.EventType: NODE_LABELS_UPDATE 2016-08-11 02:31:23,532 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:addNode(1500)) - Added node 127.0.0.1:1 clusterResource: 2016-08-11 02:31:23,532 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeLabelsUpdateSchedulerEvent.EventType: NODE_LABELS_UPDATE 2016-08-11 02:31:23,533 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:addNode(1500)) - Added node test:2 clusterResource: 2016-08-11 02:31:23,633 INFO [main] capacity.ParentQueue (ParentQueue.java:addApplication(350)) - Application added - appId: application_100_0001 user: user1 leaf-queue of parent: root #applications: 1 2016-08-11 02:31:23,634 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:addApplication(777)) - Accepted application application_100_0001 from user: user1, in queue: a 2016-08-11 02:31:23,634 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEvent.EventType: APP_ACCEPTED 2016-08-11 02:31:23,667 DEBUG [main] capacity.LeafQueue (LeafQueue.java:activateApplications(648)) - application application_100_0001 AMResource maxAMResourcePerQueuePercent 0.1 amLimit lastClusterResource amIfStarted AM node-partition name 2016-08-11 02:31:23,668 INFO [main] capacity.LeafQueue (LeafQueue.java:activateApplications(722)) - Application application_100_0001 from user: user1 activated in queue: a 2016-08-11 02:31:23,669 INFO [main] capacity.LeafQueue (LeafQueue.java:addApplicationAttempt(737)) - Application added - appId: application_100_0001 user: user1, leaf-queue: a #user-pending-applications: 0 #user-active-applications: 1 #queue-pending-applications: 0 #queue-active-applications: 1 2016-08-11 02:31:23,669 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:addApplicationAttempt(817)) - Added Application Attempt appattempt_100_0001_000001 to scheduler from user user1 in queue a 2016-08-11 02:31:23,669 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: ATTEMPT_ADDED 2016-08-11 02:31:23,671 INFO [main] capacity.ParentQueue (ParentQueue.java:addApplication(350)) - Application added - appId: application_100_0002 user: user2 leaf-queue of parent: root #applications: 2 2016-08-11 02:31:23,672 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:addApplication(777)) - Accepted application application_100_0002 from user: user2, in queue: b 2016-08-11 02:31:23,672 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEvent.EventType: APP_ACCEPTED 2016-08-11 02:31:23,673 DEBUG [main] capacity.LeafQueue (LeafQueue.java:activateApplications(648)) - application application_100_0002 AMResource maxAMResourcePerQueuePercent 0.1 amLimit lastClusterResource amIfStarted AM node-partition name 2016-08-11 02:31:23,673 INFO [main] capacity.LeafQueue (LeafQueue.java:activateApplications(722)) - Application application_100_0002 from user: user2 activated in queue: b 2016-08-11 02:31:23,673 INFO [main] capacity.LeafQueue (LeafQueue.java:addApplicationAttempt(737)) - Application added - appId: application_100_0002 user: user2, leaf-queue: b #user-pending-applications: 0 #user-active-applications: 1 #queue-pending-applications: 0 #queue-active-applications: 1 2016-08-11 02:31:23,674 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:addApplicationAttempt(817)) - Added Application Attempt appattempt_100_0002_000002 to scheduler from user user2 in queue b 2016-08-11 02:31:23,674 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: ATTEMPT_ADDED 2016-08-11 02:31:23,686 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(976)) - allocate: pre-update appattempt_100_0001_000001 ask size =1 2016-08-11 02:31:23,688 DEBUG [main] scheduler.ActiveUsersManager (ActiveUsersManager.java:activateApplication(68)) - User user1 added to activeUsers, currently: 1 2016-08-11 02:31:23,688 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(988)) - allocate: post-update 2016-08-11 02:31:23,688 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0001 headRoom= currentConsumption=0 2016-08-11 02:31:23,688 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0001 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:31:23,693 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1282)) - Trying to schedule on node: test, available: 2016-08-11 02:31:23,693 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(437)) - Trying to assign containers to child-queue of root 2016-08-11 02:31:23,695 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.0), label=(*)root.busedCapacity=(0.0), label=(*) 2016-08-11 02:31:23,695 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=1, numContainers=0 2016-08-11 02:31:23,695 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=test #applications=1 2016-08-11 02:31:23,696 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user1 in queue a userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:31:23,696 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimitAndSetHeadroom(1120)) - Headroom calculation for user user1: userLimit= queueMaxAvailRes= consumed= headroom= 2016-08-11 02:31:23,697 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:assignContainers(511)) - pre-assignContainers for application application_100_0001 2016-08-11 02:31:23,697 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user1 in queue a userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:31:23,697 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0001 headRoom= currentConsumption=0 2016-08-11 02:31:23,697 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0001 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:31:23,698 DEBUG [main] allocator.IncreaseContainerAllocator (IncreaseContainerAllocator.java:assignContainers(187)) - Skip allocating increase request since we don't have any increase request on this node=test:123 2016-08-11 02:31:23,699 DEBUG [main] allocator.RegularContainerAllocator (RegularContainerAllocator.java:assignContainer(473)) - assignContainers: node=test application=application_100_0001 priority=1 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } type=OFF_SWITCH 2016-08-11 02:31:23,716 DEBUG [main] scheduler.ActiveUsersManager (ActiveUsersManager.java:deactivateApplication(94)) - User user1 removed from activeUsers, currently: 0 2016-08-11 02:31:23,717 DEBUG [main] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:allocate(631)) - allocate: applicationId=application_100_0001 container=container_100_0001_01_000001 host=test:123 user=user1 resource= type=OFF_SWITCH 2016-08-11 02:31:23,718 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0001_01_000001 of type START 2016-08-11 02:31:23,718 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: CONTAINER_ALLOCATED 2016-08-11 02:31:23,718 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0001_01_000001 Container Transitioned from NEW to ALLOCATED 2016-08-11 02:31:23,718 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:allocate(237)) - allocate: applicationAttemptId=appattempt_100_0001_000001 container=container_100_0001_01_000001 host=test type=OFF_SWITCH 2016-08-11 02:31:23,722 INFO [main] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(156)) - USER=user1 OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_100_0001 CONTAINERID=container_100_0001_01_000001 RESOURCE= 2016-08-11 02:31:23,722 DEBUG [main] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(157)) - Assigned container container_100_0001_01_000001 of capacity on host test:2, which has 1 containers, used and available after allocation 2016-08-11 02:31:23,723 INFO [main] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(120)) - assignedContainer application attempt=appattempt_100_0001_000001 container=container_100_0001_01_000001 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@2e1ef60 clusterResource= type=OFF_SWITCH 2016-08-11 02:31:23,724 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(962)) - post-assignContainers for application application_100_0001 2016-08-11 02:31:23,724 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user1 in queue a userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:31:23,725 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0001 headRoom= currentConsumption=2048 2016-08-11 02:31:23,725 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0001 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 0, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:31:23,725 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user1 in queue a userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:31:23,726 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user1 in queue a userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:31:23,726 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user1 in queue a userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:31:23,726 DEBUG [main] capacity.LeafQueue (LeafQueue.java:allocateResource(1443)) - a user=user1 used= numContainers=1 headroom = user-resources= 2016-08-11 02:31:23,727 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 --> , OFF_SWITCH 2016-08-11 02:31:23,727 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(670)) - Re-sorting assigned queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 2016-08-11 02:31:23,728 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.busedCapacity=(0.0), label=(*)root.ausedCapacity=(0.15625238), label=(*) 2016-08-11 02:31:23,729 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainers(522)) - assignedContainer queue=root usedCapacity=0.125 absoluteUsedCapacity=0.125 used= cluster= 2016-08-11 02:31:23,729 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(544)) - ParentQ=root assignedSoFarInThisIteration= usedCapacity=0.125 absoluteUsedCapacity=0.125 2016-08-11 02:31:23,730 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(555)) - Not assigning more than one off-switch container, assignments so far: resource:; type:OFF_SWITCH; excessReservation:null; applicationid:null; skipped:NONE; fulfilled reservation:false; allocations(count/resource):1/; reservations(count/resource):0/ 2016-08-11 02:31:23,730 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1282)) - Trying to schedule on node: 127.0.0.1, available: 2016-08-11 02:31:23,730 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(416)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 02:31:23,730 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1282)) - Trying to schedule on node: test, available: 2016-08-11 02:31:23,730 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(416)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 02:31:23,736 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(976)) - allocate: pre-update appattempt_100_0002_000002 ask size =1 2016-08-11 02:31:23,736 DEBUG [main] scheduler.ActiveUsersManager (ActiveUsersManager.java:activateApplication(68)) - User user2 added to activeUsers, currently: 1 2016-08-11 02:31:23,736 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(988)) - allocate: post-update 2016-08-11 02:31:23,736 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=0 2016-08-11 02:31:23,736 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:31:23,737 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1282)) - Trying to schedule on node: 127.0.0.1, available: 2016-08-11 02:31:23,737 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(437)) - Trying to assign containers to child-queue of root 2016-08-11 02:31:23,737 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.busedCapacity=(0.0), label=(*)root.ausedCapacity=(0.15625238), label=(*) 2016-08-11 02:31:23,737 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=1, numContainers=0 2016-08-11 02:31:23,738 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=127.0.0.1 #applications=1 2016-08-11 02:31:23,738 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:31:23,738 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimitAndSetHeadroom(1120)) - Headroom calculation for user user2: userLimit= queueMaxAvailRes= consumed= headroom= 2016-08-11 02:31:23,738 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:assignContainers(511)) - pre-assignContainers for application application_100_0002 2016-08-11 02:31:23,738 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:31:23,738 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=0 2016-08-11 02:31:23,738 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:31:23,738 DEBUG [main] allocator.IncreaseContainerAllocator (IncreaseContainerAllocator.java:assignContainers(187)) - Skip allocating increase request since we don't have any increase request on this node=127.0.0.1:123 2016-08-11 02:31:23,739 DEBUG [main] allocator.RegularContainerAllocator (RegularContainerAllocator.java:assignContainer(473)) - assignContainers: node=127.0.0.1 application=application_100_0002 priority=1 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } type=OFF_SWITCH 2016-08-11 02:31:23,739 DEBUG [main] scheduler.ActiveUsersManager (ActiveUsersManager.java:deactivateApplication(94)) - User user2 removed from activeUsers, currently: 0 2016-08-11 02:31:23,739 DEBUG [main] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:allocate(631)) - allocate: applicationId=application_100_0002 container=container_100_0002_02_000001 host=127.0.0.1:123 user=user2 resource= type=OFF_SWITCH 2016-08-11 02:31:23,739 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0002_02_000001 of type START 2016-08-11 02:31:23,739 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0002_02_000001 Container Transitioned from NEW to ALLOCATED 2016-08-11 02:31:23,739 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: CONTAINER_ALLOCATED 2016-08-11 02:31:23,740 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:allocate(237)) - allocate: applicationAttemptId=appattempt_100_0002_000002 container=container_100_0002_02_000001 host=127.0.0.1 type=OFF_SWITCH 2016-08-11 02:31:23,740 INFO [main] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(156)) - USER=user2 OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_100_0002 CONTAINERID=container_100_0002_02_000001 RESOURCE= 2016-08-11 02:31:23,740 DEBUG [main] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(157)) - Assigned container container_100_0002_02_000001 of capacity on host 127.0.0.1:1, which has 1 containers, used and available after allocation 2016-08-11 02:31:23,740 INFO [main] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(120)) - assignedContainer application attempt=appattempt_100_0002_000002 container=container_100_0002_02_000001 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@22356acd clusterResource= type=OFF_SWITCH 2016-08-11 02:31:23,741 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(962)) - post-assignContainers for application application_100_0002 2016-08-11 02:31:23,741 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:31:23,741 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=1024 2016-08-11 02:31:23,741 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 0, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:31:23,741 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:31:23,741 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:31:23,742 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:31:23,742 DEBUG [main] capacity.LeafQueue (LeafQueue.java:allocateResource(1443)) - b user=user2 used= numContainers=1 headroom = user-resources= 2016-08-11 02:31:23,742 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=0.31257632, absoluteUsedCapacity=0.0625, numApps=1, numContainers=1 --> , OFF_SWITCH 2016-08-11 02:31:23,742 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(670)) - Re-sorting assigned queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=0.31257632, absoluteUsedCapacity=0.0625, numApps=1, numContainers=1 2016-08-11 02:31:23,742 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.15625238), label=(*)root.busedCapacity=(0.31257632), label=(*) 2016-08-11 02:31:23,742 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainers(522)) - assignedContainer queue=root usedCapacity=0.1875 absoluteUsedCapacity=0.1875 used= cluster= 2016-08-11 02:31:23,743 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(544)) - ParentQ=root assignedSoFarInThisIteration= usedCapacity=0.1875 absoluteUsedCapacity=0.1875 2016-08-11 02:31:23,743 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(555)) - Not assigning more than one off-switch container, assignments so far: resource:; type:OFF_SWITCH; excessReservation:null; applicationid:null; skipped:NONE; fulfilled reservation:false; allocations(count/resource):1/; reservations(count/resource):0/ 2016-08-11 02:31:23,743 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1282)) - Trying to schedule on node: test, available: 2016-08-11 02:31:23,743 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(416)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 02:31:23,743 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1282)) - Trying to schedule on node: 127.0.0.1, available: 2016-08-11 02:31:23,743 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(416)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 02:31:23,743 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1282)) - Trying to schedule on node: test, available: 2016-08-11 02:31:23,743 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(416)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 02:31:23,749 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(976)) - allocate: pre-update appattempt_100_0002_000002 ask size =1 2016-08-11 02:31:23,749 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:31:23,749 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=1024 2016-08-11 02:31:23,749 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 0, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:31:23,749 DEBUG [main] scheduler.ActiveUsersManager (ActiveUsersManager.java:activateApplication(68)) - User user2 added to activeUsers, currently: 1 2016-08-11 02:31:23,750 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(988)) - allocate: post-update 2016-08-11 02:31:23,750 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:31:23,750 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=1024 2016-08-11 02:31:23,750 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:31:23,763 DEBUG [main] security.BaseContainerTokenSecretManager (BaseContainerTokenSecretManager.java:createPassword(90)) - Creating password for container_100_0002_02_000001 for user container_100_0002_02_000001 (auth:SIMPLE) to be run on NM 127.0.0.1:123 2016-08-11 02:31:23,898 DEBUG [main] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 2 cluster_timestamp: 100 } attemptId: 2 } id: 1 } nmHostAddr: "127.0.0.1:123" appSubmitter: "user2" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1470863483751 masterKeyId: 100511640 rmIdentifier: 1470862883498 priority { priority: 1 } creationTime: 1470862883739 nodeLabelExpression: "" containerType: TASK executionType: GUARANTEED 2016-08-11 02:31:23,902 DEBUG [main] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 2 cluster_timestamp: 100 } attemptId: 2 } id: 1 } nmHostAddr: "127.0.0.1:123" appSubmitter: "user2" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1470863483751 masterKeyId: 100511640 rmIdentifier: 1470862883498 priority { priority: 1 } creationTime: 1470862883739 nodeLabelExpression: "" containerType: TASK executionType: GUARANTEED 2016-08-11 02:31:23,904 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0002_02_000001 of type ACQUIRED 2016-08-11 02:31:23,904 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0002_02_000001 Container Transitioned from ALLOCATED to ACQUIRED 2016-08-11 02:31:23,904 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppRunningOnNodeEvent.EventType: APP_RUNNING_ON_NODE 2016-08-11 02:31:23,904 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:31:23,905 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1282)) - Trying to schedule on node: test, available: 2016-08-11 02:31:23,905 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(437)) - Trying to assign containers to child-queue of root 2016-08-11 02:31:23,905 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.15625238), label=(*)root.busedCapacity=(0.31257632), label=(*) 2016-08-11 02:31:23,905 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 2016-08-11 02:31:23,905 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=test #applications=1 2016-08-11 02:31:23,906 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(909)) - Skip this queue=root.a, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 02:31:23,906 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 --> , NODE_LOCAL 2016-08-11 02:31:23,906 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=0.31257632, absoluteUsedCapacity=0.0625, numApps=1, numContainers=1 2016-08-11 02:31:23,906 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=test #applications=1 2016-08-11 02:31:23,906 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:31:23,906 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimitAndSetHeadroom(1120)) - Headroom calculation for user user2: userLimit= queueMaxAvailRes= consumed= headroom= 2016-08-11 02:31:23,906 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:assignContainers(511)) - pre-assignContainers for application application_100_0002 2016-08-11 02:31:23,907 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:31:23,907 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=1024 2016-08-11 02:31:23,907 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:31:23,907 DEBUG [main] allocator.IncreaseContainerAllocator (IncreaseContainerAllocator.java:assignContainers(187)) - Skip allocating increase request since we don't have any increase request on this node=test:123 2016-08-11 02:31:23,907 DEBUG [main] allocator.RegularContainerAllocator (RegularContainerAllocator.java:assignContainer(473)) - assignContainers: node=test application=application_100_0002 priority=1 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } type=OFF_SWITCH 2016-08-11 02:31:23,907 DEBUG [main] scheduler.ActiveUsersManager (ActiveUsersManager.java:deactivateApplication(94)) - User user2 removed from activeUsers, currently: 0 2016-08-11 02:31:23,908 DEBUG [main] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:allocate(631)) - allocate: applicationId=application_100_0002 container=container_100_0002_02_000002 host=test:123 user=user2 resource= type=OFF_SWITCH 2016-08-11 02:31:23,908 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0002_02_000002 of type START 2016-08-11 02:31:23,908 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: CONTAINER_ALLOCATED 2016-08-11 02:31:23,908 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0002_02_000002 Container Transitioned from NEW to ALLOCATED 2016-08-11 02:31:23,908 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:allocate(237)) - allocate: applicationAttemptId=appattempt_100_0002_000002 container=container_100_0002_02_000002 host=test type=OFF_SWITCH 2016-08-11 02:31:23,909 INFO [main] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(156)) - USER=user2 OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_100_0002 CONTAINERID=container_100_0002_02_000002 RESOURCE= 2016-08-11 02:31:23,909 DEBUG [main] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(157)) - Assigned container container_100_0002_02_000002 of capacity on host test:2, which has 2 containers, used and available after allocation 2016-08-11 02:31:23,909 INFO [main] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(120)) - assignedContainer application attempt=appattempt_100_0002_000002 container=container_100_0002_02_000002 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@22356acd clusterResource= type=OFF_SWITCH 2016-08-11 02:31:23,909 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(962)) - post-assignContainers for application application_100_0002 2016-08-11 02:31:23,909 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:31:23,910 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=2048 2016-08-11 02:31:23,910 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 0, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:31:23,910 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:31:23,910 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:31:23,910 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:31:23,910 DEBUG [main] capacity.LeafQueue (LeafQueue.java:allocateResource(1443)) - b user=user2 used= numContainers=2 headroom = user-resources= 2016-08-11 02:31:23,911 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=0.62515265, absoluteUsedCapacity=0.125, numApps=1, numContainers=2 --> , OFF_SWITCH 2016-08-11 02:31:23,911 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(670)) - Re-sorting assigned queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=0.62515265, absoluteUsedCapacity=0.125, numApps=1, numContainers=2 2016-08-11 02:31:23,911 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.15625238), label=(*)root.busedCapacity=(0.62515265), label=(*) 2016-08-11 02:31:23,911 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainers(522)) - assignedContainer queue=root usedCapacity=0.25 absoluteUsedCapacity=0.25 used= cluster= 2016-08-11 02:31:23,911 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(544)) - ParentQ=root assignedSoFarInThisIteration= usedCapacity=0.25 absoluteUsedCapacity=0.25 2016-08-11 02:31:23,911 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(555)) - Not assigning more than one off-switch container, assignments so far: resource:; type:OFF_SWITCH; excessReservation:null; applicationid:null; skipped:NONE; fulfilled reservation:false; allocations(count/resource):1/; reservations(count/resource):0/ 2016-08-11 02:31:23,912 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1282)) - Trying to schedule on node: 127.0.0.1, available: 2016-08-11 02:31:23,912 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(416)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 02:31:23,912 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1282)) - Trying to schedule on node: test, available: 2016-08-11 02:31:23,912 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(416)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 02:31:23,917 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(976)) - allocate: pre-update appattempt_100_0002_000002 ask size =1 2016-08-11 02:31:23,918 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:31:23,918 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=2048 2016-08-11 02:31:23,918 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 0, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:31:23,918 DEBUG [main] scheduler.ActiveUsersManager (ActiveUsersManager.java:activateApplication(68)) - User user2 added to activeUsers, currently: 1 2016-08-11 02:31:23,918 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(988)) - allocate: post-update 2016-08-11 02:31:23,919 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:31:23,919 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=2048 2016-08-11 02:31:23,919 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:31:23,919 DEBUG [main] security.BaseContainerTokenSecretManager (BaseContainerTokenSecretManager.java:createPassword(90)) - Creating password for container_100_0002_02_000002 for user container_100_0002_02_000002 (auth:SIMPLE) to be run on NM test:123 2016-08-11 02:31:23,920 DEBUG [main] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 2 cluster_timestamp: 100 } attemptId: 2 } id: 2 } nmHostAddr: "test:123" appSubmitter: "user2" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1470863483919 masterKeyId: 100511640 rmIdentifier: 1470862883498 priority { priority: 1 } creationTime: 1470862883907 nodeLabelExpression: "" containerType: TASK executionType: GUARANTEED 2016-08-11 02:31:43,943 DEBUG [main] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 2 cluster_timestamp: 100 } attemptId: 2 } id: 2 } nmHostAddr: "test:123" appSubmitter: "user2" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1470863483919 masterKeyId: 100511640 rmIdentifier: 1470862883498 priority { priority: 1 } creationTime: 1470862883907 nodeLabelExpression: "" containerType: TASK executionType: GUARANTEED 2016-08-11 02:31:43,944 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0002_02_000002 of type ACQUIRED 2016-08-11 02:31:43,944 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppRunningOnNodeEvent.EventType: APP_RUNNING_ON_NODE 2016-08-11 02:31:43,944 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0002_02_000002 Container Transitioned from ALLOCATED to ACQUIRED 2016-08-11 02:31:43,946 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:31:43,947 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1282)) - Trying to schedule on node: test, available: 2016-08-11 02:31:43,948 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(437)) - Trying to assign containers to child-queue of root 2016-08-11 02:31:43,948 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.15625238), label=(*)root.busedCapacity=(0.62515265), label=(*) 2016-08-11 02:31:43,948 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 2016-08-11 02:31:43,949 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=test #applications=1 2016-08-11 02:31:43,949 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(909)) - Skip this queue=root.a, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 02:31:43,949 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 --> , NODE_LOCAL 2016-08-11 02:31:43,950 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=0.62515265, absoluteUsedCapacity=0.125, numApps=1, numContainers=2 2016-08-11 02:31:43,950 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=test #applications=1 2016-08-11 02:31:43,951 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:31:43,951 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimitAndSetHeadroom(1120)) - Headroom calculation for user user2: userLimit= queueMaxAvailRes= consumed= headroom= 2016-08-11 02:31:43,952 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:assignContainers(511)) - pre-assignContainers for application application_100_0002 2016-08-11 02:31:43,953 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:31:43,953 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=2048 2016-08-11 02:31:43,953 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:31:43,954 DEBUG [main] allocator.IncreaseContainerAllocator (IncreaseContainerAllocator.java:assignContainers(187)) - Skip allocating increase request since we don't have any increase request on this node=test:123 2016-08-11 02:31:43,954 DEBUG [main] allocator.RegularContainerAllocator (RegularContainerAllocator.java:assignContainer(473)) - assignContainers: node=test application=application_100_0002 priority=1 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } type=OFF_SWITCH 2016-08-11 02:31:43,955 DEBUG [main] scheduler.ActiveUsersManager (ActiveUsersManager.java:deactivateApplication(94)) - User user2 removed from activeUsers, currently: 0 2016-08-11 02:31:43,956 DEBUG [main] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:allocate(631)) - allocate: applicationId=application_100_0002 container=container_100_0002_02_000003 host=test:123 user=user2 resource= type=OFF_SWITCH 2016-08-11 02:31:43,957 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0002_02_000003 of type START 2016-08-11 02:31:43,957 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0002_02_000003 Container Transitioned from NEW to ALLOCATED 2016-08-11 02:31:43,957 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: CONTAINER_ALLOCATED 2016-08-11 02:31:43,959 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:allocate(237)) - allocate: applicationAttemptId=appattempt_100_0002_000002 container=container_100_0002_02_000003 host=test type=OFF_SWITCH 2016-08-11 02:31:43,959 INFO [main] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(156)) - USER=user2 OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_100_0002 CONTAINERID=container_100_0002_02_000003 RESOURCE= 2016-08-11 02:31:43,959 DEBUG [main] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(157)) - Assigned container container_100_0002_02_000003 of capacity on host test:2, which has 3 containers, used and available after allocation 2016-08-11 02:31:43,961 INFO [main] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(120)) - assignedContainer application attempt=appattempt_100_0002_000002 container=container_100_0002_02_000003 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@22356acd clusterResource= type=OFF_SWITCH 2016-08-11 02:31:43,961 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(962)) - post-assignContainers for application application_100_0002 2016-08-11 02:31:43,962 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:31:43,962 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=3072 2016-08-11 02:31:43,963 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 0, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:31:43,963 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:31:43,964 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:31:43,964 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:31:43,965 DEBUG [main] capacity.LeafQueue (LeafQueue.java:allocateResource(1443)) - b user=user2 used= numContainers=3 headroom = user-resources= 2016-08-11 02:31:43,965 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=0.93772894, absoluteUsedCapacity=0.1875, numApps=1, numContainers=3 --> , OFF_SWITCH 2016-08-11 02:31:43,965 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(670)) - Re-sorting assigned queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=0.93772894, absoluteUsedCapacity=0.1875, numApps=1, numContainers=3 2016-08-11 02:31:43,966 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.15625238), label=(*)root.busedCapacity=(0.93772894), label=(*) 2016-08-11 02:31:43,966 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainers(522)) - assignedContainer queue=root usedCapacity=0.3125 absoluteUsedCapacity=0.3125 used= cluster= 2016-08-11 02:31:43,967 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(544)) - ParentQ=root assignedSoFarInThisIteration= usedCapacity=0.3125 absoluteUsedCapacity=0.3125 2016-08-11 02:31:43,967 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(555)) - Not assigning more than one off-switch container, assignments so far: resource:; type:OFF_SWITCH; excessReservation:null; applicationid:null; skipped:NONE; fulfilled reservation:false; allocations(count/resource):1/; reservations(count/resource):0/ 2016-08-11 02:31:43,968 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1282)) - Trying to schedule on node: 127.0.0.1, available: 2016-08-11 02:31:43,968 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(416)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 02:31:43,968 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1282)) - Trying to schedule on node: test, available: 2016-08-11 02:31:43,968 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(416)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 02:31:43,974 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(976)) - allocate: pre-update appattempt_100_0002_000002 ask size =1 2016-08-11 02:31:43,975 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:31:43,975 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=3072 2016-08-11 02:31:43,975 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 0, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:31:43,976 DEBUG [main] scheduler.ActiveUsersManager (ActiveUsersManager.java:activateApplication(68)) - User user2 added to activeUsers, currently: 1 2016-08-11 02:31:43,976 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(988)) - allocate: post-update 2016-08-11 02:31:43,977 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:31:43,977 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=3072 2016-08-11 02:31:43,977 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:31:43,978 DEBUG [main] security.BaseContainerTokenSecretManager (BaseContainerTokenSecretManager.java:createPassword(90)) - Creating password for container_100_0002_02_000003 for user container_100_0002_02_000003 (auth:SIMPLE) to be run on NM test:123 2016-08-11 02:31:43,980 DEBUG [main] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 2 cluster_timestamp: 100 } attemptId: 2 } id: 3 } nmHostAddr: "test:123" appSubmitter: "user2" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1470863503978 masterKeyId: 100511640 rmIdentifier: 1470862883498 priority { priority: 1 } creationTime: 1470862903955 nodeLabelExpression: "" containerType: TASK executionType: GUARANTEED 2016-08-11 02:32:03,999 DEBUG [main] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 2 cluster_timestamp: 100 } attemptId: 2 } id: 3 } nmHostAddr: "test:123" appSubmitter: "user2" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1470863503978 masterKeyId: 100511640 rmIdentifier: 1470862883498 priority { priority: 1 } creationTime: 1470862903955 nodeLabelExpression: "" containerType: TASK executionType: GUARANTEED 2016-08-11 02:32:04,001 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0002_02_000003 of type ACQUIRED 2016-08-11 02:32:04,002 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppRunningOnNodeEvent.EventType: APP_RUNNING_ON_NODE 2016-08-11 02:32:04,002 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0002_02_000003 Container Transitioned from ALLOCATED to ACQUIRED 2016-08-11 02:32:04,003 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:32:04,005 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1282)) - Trying to schedule on node: test, available: 2016-08-11 02:32:04,005 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(437)) - Trying to assign containers to child-queue of root 2016-08-11 02:32:04,006 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.15625238), label=(*)root.busedCapacity=(0.93772894), label=(*) 2016-08-11 02:32:04,007 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 2016-08-11 02:32:04,007 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=test #applications=1 2016-08-11 02:32:04,008 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(909)) - Skip this queue=root.a, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 02:32:04,012 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 --> , NODE_LOCAL 2016-08-11 02:32:04,012 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=0.93772894, absoluteUsedCapacity=0.1875, numApps=1, numContainers=3 2016-08-11 02:32:04,013 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=test #applications=1 2016-08-11 02:32:04,013 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:32:04,014 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimitAndSetHeadroom(1120)) - Headroom calculation for user user2: userLimit= queueMaxAvailRes= consumed= headroom= 2016-08-11 02:32:04,014 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:assignContainers(511)) - pre-assignContainers for application application_100_0002 2016-08-11 02:32:04,015 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:32:04,015 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=3072 2016-08-11 02:32:04,015 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:32:04,016 DEBUG [main] allocator.IncreaseContainerAllocator (IncreaseContainerAllocator.java:assignContainers(187)) - Skip allocating increase request since we don't have any increase request on this node=test:123 2016-08-11 02:32:04,016 DEBUG [main] allocator.RegularContainerAllocator (RegularContainerAllocator.java:assignContainer(473)) - assignContainers: node=test application=application_100_0002 priority=1 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } type=OFF_SWITCH 2016-08-11 02:32:04,017 DEBUG [main] scheduler.ActiveUsersManager (ActiveUsersManager.java:deactivateApplication(94)) - User user2 removed from activeUsers, currently: 0 2016-08-11 02:32:04,018 DEBUG [main] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:allocate(631)) - allocate: applicationId=application_100_0002 container=container_100_0002_02_000004 host=test:123 user=user2 resource= type=OFF_SWITCH 2016-08-11 02:32:04,019 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0002_02_000004 of type START 2016-08-11 02:32:04,019 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: CONTAINER_ALLOCATED 2016-08-11 02:32:04,019 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0002_02_000004 Container Transitioned from NEW to ALLOCATED 2016-08-11 02:32:04,020 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:allocate(237)) - allocate: applicationAttemptId=appattempt_100_0002_000002 container=container_100_0002_02_000004 host=test type=OFF_SWITCH 2016-08-11 02:32:04,020 INFO [main] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(156)) - USER=user2 OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_100_0002 CONTAINERID=container_100_0002_02_000004 RESOURCE= 2016-08-11 02:32:04,021 DEBUG [main] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(157)) - Assigned container container_100_0002_02_000004 of capacity on host test:2, which has 4 containers, used and available after allocation 2016-08-11 02:32:04,022 INFO [main] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(120)) - assignedContainer application attempt=appattempt_100_0002_000002 container=container_100_0002_02_000004 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@22356acd clusterResource= type=OFF_SWITCH 2016-08-11 02:32:04,022 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(962)) - post-assignContainers for application application_100_0002 2016-08-11 02:32:04,023 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:32:04,023 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=4096 2016-08-11 02:32:04,023 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 0, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:32:04,024 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:32:04,024 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:32:04,024 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:32:04,025 DEBUG [main] capacity.LeafQueue (LeafQueue.java:allocateResource(1443)) - b user=user2 used= numContainers=4 headroom = user-resources= 2016-08-11 02:32:04,025 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=1.2503053, absoluteUsedCapacity=0.25, numApps=1, numContainers=4 --> , OFF_SWITCH 2016-08-11 02:32:04,025 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(670)) - Re-sorting assigned queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=1.2503053, absoluteUsedCapacity=0.25, numApps=1, numContainers=4 2016-08-11 02:32:04,026 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.15625238), label=(*)root.busedCapacity=(1.2503053), label=(*) 2016-08-11 02:32:04,026 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainers(522)) - assignedContainer queue=root usedCapacity=0.375 absoluteUsedCapacity=0.375 used= cluster= 2016-08-11 02:32:04,026 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(544)) - ParentQ=root assignedSoFarInThisIteration= usedCapacity=0.375 absoluteUsedCapacity=0.375 2016-08-11 02:32:04,026 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(555)) - Not assigning more than one off-switch container, assignments so far: resource:; type:OFF_SWITCH; excessReservation:null; applicationid:null; skipped:NONE; fulfilled reservation:false; allocations(count/resource):1/; reservations(count/resource):0/ 2016-08-11 02:32:04,027 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1282)) - Trying to schedule on node: 127.0.0.1, available: 2016-08-11 02:32:04,027 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(416)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 02:32:04,027 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1282)) - Trying to schedule on node: test, available: 2016-08-11 02:32:04,028 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(416)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 02:32:04,034 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(976)) - allocate: pre-update appattempt_100_0002_000002 ask size =1 2016-08-11 02:32:04,034 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:32:04,035 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=4096 2016-08-11 02:32:04,035 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 0, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:32:04,035 DEBUG [main] scheduler.ActiveUsersManager (ActiveUsersManager.java:activateApplication(68)) - User user2 added to activeUsers, currently: 1 2016-08-11 02:32:04,035 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(988)) - allocate: post-update 2016-08-11 02:32:04,036 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:32:04,036 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=4096 2016-08-11 02:32:04,036 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:32:04,037 DEBUG [main] security.BaseContainerTokenSecretManager (BaseContainerTokenSecretManager.java:createPassword(90)) - Creating password for container_100_0002_02_000004 for user container_100_0002_02_000004 (auth:SIMPLE) to be run on NM test:123 2016-08-11 02:32:04,038 DEBUG [main] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 2 cluster_timestamp: 100 } attemptId: 2 } id: 4 } nmHostAddr: "test:123" appSubmitter: "user2" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1470863524037 masterKeyId: 100511640 rmIdentifier: 1470862883498 priority { priority: 1 } creationTime: 1470862924017 nodeLabelExpression: "" containerType: TASK executionType: GUARANTEED 2016-08-11 02:32:24,058 DEBUG [main] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 2 cluster_timestamp: 100 } attemptId: 2 } id: 4 } nmHostAddr: "test:123" appSubmitter: "user2" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1470863524037 masterKeyId: 100511640 rmIdentifier: 1470862883498 priority { priority: 1 } creationTime: 1470862924017 nodeLabelExpression: "" containerType: TASK executionType: GUARANTEED 2016-08-11 02:32:24,059 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0002_02_000004 of type ACQUIRED 2016-08-11 02:32:24,060 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppRunningOnNodeEvent.EventType: APP_RUNNING_ON_NODE 2016-08-11 02:32:24,060 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0002_02_000004 Container Transitioned from ALLOCATED to ACQUIRED 2016-08-11 02:32:24,061 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:32:24,063 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1282)) - Trying to schedule on node: test, available: 2016-08-11 02:32:24,063 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(437)) - Trying to assign containers to child-queue of root 2016-08-11 02:32:24,064 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.15625238), label=(*)root.busedCapacity=(1.2503053), label=(*) 2016-08-11 02:32:24,064 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 2016-08-11 02:32:24,065 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=test #applications=1 2016-08-11 02:32:24,065 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(909)) - Skip this queue=root.a, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 02:32:24,065 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 --> , NODE_LOCAL 2016-08-11 02:32:24,066 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=1.2503053, absoluteUsedCapacity=0.25, numApps=1, numContainers=4 2016-08-11 02:32:24,066 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=test #applications=1 2016-08-11 02:32:24,067 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:32:24,067 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimitAndSetHeadroom(1120)) - Headroom calculation for user user2: userLimit= queueMaxAvailRes= consumed= headroom= 2016-08-11 02:32:24,067 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:assignContainers(511)) - pre-assignContainers for application application_100_0002 2016-08-11 02:32:24,068 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:32:24,068 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=4096 2016-08-11 02:32:24,069 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:32:24,069 DEBUG [main] allocator.IncreaseContainerAllocator (IncreaseContainerAllocator.java:assignContainers(187)) - Skip allocating increase request since we don't have any increase request on this node=test:123 2016-08-11 02:32:24,070 DEBUG [main] allocator.RegularContainerAllocator (RegularContainerAllocator.java:assignContainer(473)) - assignContainers: node=test application=application_100_0002 priority=1 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } type=OFF_SWITCH 2016-08-11 02:32:24,070 DEBUG [main] scheduler.ActiveUsersManager (ActiveUsersManager.java:deactivateApplication(94)) - User user2 removed from activeUsers, currently: 0 2016-08-11 02:32:24,071 DEBUG [main] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:allocate(631)) - allocate: applicationId=application_100_0002 container=container_100_0002_02_000005 host=test:123 user=user2 resource= type=OFF_SWITCH 2016-08-11 02:32:24,072 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0002_02_000005 of type START 2016-08-11 02:32:24,073 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: CONTAINER_ALLOCATED 2016-08-11 02:32:24,074 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0002_02_000005 Container Transitioned from NEW to ALLOCATED 2016-08-11 02:32:24,075 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:allocate(237)) - allocate: applicationAttemptId=appattempt_100_0002_000002 container=container_100_0002_02_000005 host=test type=OFF_SWITCH 2016-08-11 02:32:24,076 INFO [main] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(156)) - USER=user2 OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_100_0002 CONTAINERID=container_100_0002_02_000005 RESOURCE= 2016-08-11 02:32:24,076 DEBUG [main] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(157)) - Assigned container container_100_0002_02_000005 of capacity on host test:2, which has 5 containers, used and available after allocation 2016-08-11 02:32:24,078 INFO [main] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(120)) - assignedContainer application attempt=appattempt_100_0002_000002 container=container_100_0002_02_000005 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@22356acd clusterResource= type=OFF_SWITCH 2016-08-11 02:32:24,079 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(962)) - post-assignContainers for application application_100_0002 2016-08-11 02:32:24,079 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:32:24,080 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=5120 2016-08-11 02:32:24,080 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 0, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:32:24,081 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:32:24,082 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:32:24,082 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:32:24,083 DEBUG [main] capacity.LeafQueue (LeafQueue.java:allocateResource(1443)) - b user=user2 used= numContainers=5 headroom = user-resources= 2016-08-11 02:32:24,083 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=1.5628816, absoluteUsedCapacity=0.3125, numApps=1, numContainers=5 --> , OFF_SWITCH 2016-08-11 02:32:24,084 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(670)) - Re-sorting assigned queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=1.5628816, absoluteUsedCapacity=0.3125, numApps=1, numContainers=5 2016-08-11 02:32:24,084 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.15625238), label=(*)root.busedCapacity=(1.5628816), label=(*) 2016-08-11 02:32:24,085 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainers(522)) - assignedContainer queue=root usedCapacity=0.4375 absoluteUsedCapacity=0.4375 used= cluster= 2016-08-11 02:32:24,086 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(544)) - ParentQ=root assignedSoFarInThisIteration= usedCapacity=0.4375 absoluteUsedCapacity=0.4375 2016-08-11 02:32:24,086 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(555)) - Not assigning more than one off-switch container, assignments so far: resource:; type:OFF_SWITCH; excessReservation:null; applicationid:null; skipped:NONE; fulfilled reservation:false; allocations(count/resource):1/; reservations(count/resource):0/ 2016-08-11 02:32:24,087 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1282)) - Trying to schedule on node: 127.0.0.1, available: 2016-08-11 02:32:24,087 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(416)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 02:32:24,088 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1282)) - Trying to schedule on node: test, available: 2016-08-11 02:32:24,088 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(416)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 02:32:24,093 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(976)) - allocate: pre-update appattempt_100_0002_000002 ask size =1 2016-08-11 02:32:24,094 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:32:24,094 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=5120 2016-08-11 02:32:24,094 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 0, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:32:24,095 DEBUG [main] scheduler.ActiveUsersManager (ActiveUsersManager.java:activateApplication(68)) - User user2 added to activeUsers, currently: 1 2016-08-11 02:32:24,095 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(988)) - allocate: post-update 2016-08-11 02:32:24,096 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:32:24,096 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=5120 2016-08-11 02:32:24,096 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:32:24,097 DEBUG [main] security.BaseContainerTokenSecretManager (BaseContainerTokenSecretManager.java:createPassword(90)) - Creating password for container_100_0002_02_000005 for user container_100_0002_02_000005 (auth:SIMPLE) to be run on NM test:123 2016-08-11 02:32:24,098 DEBUG [main] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 2 cluster_timestamp: 100 } attemptId: 2 } id: 5 } nmHostAddr: "test:123" appSubmitter: "user2" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1470863544096 masterKeyId: 100511640 rmIdentifier: 1470862883498 priority { priority: 1 } creationTime: 1470862944070 nodeLabelExpression: "" containerType: TASK executionType: GUARANTEED 2016-08-11 02:32:44,121 DEBUG [main] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 2 cluster_timestamp: 100 } attemptId: 2 } id: 5 } nmHostAddr: "test:123" appSubmitter: "user2" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1470863544096 masterKeyId: 100511640 rmIdentifier: 1470862883498 priority { priority: 1 } creationTime: 1470862944070 nodeLabelExpression: "" containerType: TASK executionType: GUARANTEED 2016-08-11 02:32:44,122 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0002_02_000005 of type ACQUIRED 2016-08-11 02:32:44,123 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0002_02_000005 Container Transitioned from ALLOCATED to ACQUIRED 2016-08-11 02:32:44,123 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppRunningOnNodeEvent.EventType: APP_RUNNING_ON_NODE 2016-08-11 02:32:44,124 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:32:44,126 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1282)) - Trying to schedule on node: test, available: 2016-08-11 02:32:44,126 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(437)) - Trying to assign containers to child-queue of root 2016-08-11 02:32:44,127 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.15625238), label=(*)root.busedCapacity=(1.5628816), label=(*) 2016-08-11 02:32:44,127 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 2016-08-11 02:32:44,127 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=test #applications=1 2016-08-11 02:32:44,128 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(909)) - Skip this queue=root.a, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 02:32:44,129 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 --> , NODE_LOCAL 2016-08-11 02:32:44,129 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=1.5628816, absoluteUsedCapacity=0.3125, numApps=1, numContainers=5 2016-08-11 02:32:44,130 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=test #applications=1 2016-08-11 02:32:44,130 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:32:44,130 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimitAndSetHeadroom(1120)) - Headroom calculation for user user2: userLimit= queueMaxAvailRes= consumed= headroom= 2016-08-11 02:32:44,131 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:assignContainers(511)) - pre-assignContainers for application application_100_0002 2016-08-11 02:32:44,131 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:32:44,132 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=5120 2016-08-11 02:32:44,132 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:32:44,132 DEBUG [main] allocator.IncreaseContainerAllocator (IncreaseContainerAllocator.java:assignContainers(187)) - Skip allocating increase request since we don't have any increase request on this node=test:123 2016-08-11 02:32:44,133 DEBUG [main] allocator.RegularContainerAllocator (RegularContainerAllocator.java:assignContainer(473)) - assignContainers: node=test application=application_100_0002 priority=1 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } type=OFF_SWITCH 2016-08-11 02:32:44,133 DEBUG [main] scheduler.ActiveUsersManager (ActiveUsersManager.java:deactivateApplication(94)) - User user2 removed from activeUsers, currently: 0 2016-08-11 02:32:44,134 DEBUG [main] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:allocate(631)) - allocate: applicationId=application_100_0002 container=container_100_0002_02_000006 host=test:123 user=user2 resource= type=OFF_SWITCH 2016-08-11 02:32:44,134 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0002_02_000006 of type START 2016-08-11 02:32:44,135 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0002_02_000006 Container Transitioned from NEW to ALLOCATED 2016-08-11 02:32:44,135 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: CONTAINER_ALLOCATED 2016-08-11 02:32:44,135 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:allocate(237)) - allocate: applicationAttemptId=appattempt_100_0002_000002 container=container_100_0002_02_000006 host=test type=OFF_SWITCH 2016-08-11 02:32:44,136 INFO [main] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(156)) - USER=user2 OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_100_0002 CONTAINERID=container_100_0002_02_000006 RESOURCE= 2016-08-11 02:32:44,136 DEBUG [main] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(157)) - Assigned container container_100_0002_02_000006 of capacity on host test:2, which has 6 containers, used and available after allocation 2016-08-11 02:32:44,137 INFO [main] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(120)) - assignedContainer application attempt=appattempt_100_0002_000002 container=container_100_0002_02_000006 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@22356acd clusterResource= type=OFF_SWITCH 2016-08-11 02:32:44,138 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(962)) - post-assignContainers for application application_100_0002 2016-08-11 02:32:44,138 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:32:44,139 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=6144 2016-08-11 02:32:44,139 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 0, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:32:44,140 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:32:44,140 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:32:44,140 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:32:44,141 DEBUG [main] capacity.LeafQueue (LeafQueue.java:allocateResource(1443)) - b user=user2 used= numContainers=6 headroom = user-resources= 2016-08-11 02:32:44,141 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=1.8754579, absoluteUsedCapacity=0.375, numApps=1, numContainers=6 --> , OFF_SWITCH 2016-08-11 02:32:44,141 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(670)) - Re-sorting assigned queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=1.8754579, absoluteUsedCapacity=0.375, numApps=1, numContainers=6 2016-08-11 02:32:44,142 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.15625238), label=(*)root.busedCapacity=(1.8754579), label=(*) 2016-08-11 02:32:44,142 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainers(522)) - assignedContainer queue=root usedCapacity=0.5 absoluteUsedCapacity=0.5 used= cluster= 2016-08-11 02:32:44,143 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(544)) - ParentQ=root assignedSoFarInThisIteration= usedCapacity=0.5 absoluteUsedCapacity=0.5 2016-08-11 02:32:44,143 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(555)) - Not assigning more than one off-switch container, assignments so far: resource:; type:OFF_SWITCH; excessReservation:null; applicationid:null; skipped:NONE; fulfilled reservation:false; allocations(count/resource):1/; reservations(count/resource):0/ 2016-08-11 02:32:44,143 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1282)) - Trying to schedule on node: 127.0.0.1, available: 2016-08-11 02:32:44,144 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(416)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 02:32:44,144 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1282)) - Trying to schedule on node: test, available: 2016-08-11 02:32:44,144 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(416)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 02:32:44,150 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(976)) - allocate: pre-update appattempt_100_0002_000002 ask size =1 2016-08-11 02:32:44,150 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:32:44,151 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=6144 2016-08-11 02:32:44,151 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 0, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:32:44,151 DEBUG [main] scheduler.ActiveUsersManager (ActiveUsersManager.java:activateApplication(68)) - User user2 added to activeUsers, currently: 1 2016-08-11 02:32:44,152 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(988)) - allocate: post-update 2016-08-11 02:32:44,152 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:32:44,152 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=6144 2016-08-11 02:32:44,153 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:32:44,153 DEBUG [main] security.BaseContainerTokenSecretManager (BaseContainerTokenSecretManager.java:createPassword(90)) - Creating password for container_100_0002_02_000006 for user container_100_0002_02_000006 (auth:SIMPLE) to be run on NM test:123 2016-08-11 02:32:44,154 DEBUG [main] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 2 cluster_timestamp: 100 } attemptId: 2 } id: 6 } nmHostAddr: "test:123" appSubmitter: "user2" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1470863564153 masterKeyId: 100511640 rmIdentifier: 1470862883498 priority { priority: 1 } creationTime: 1470862964133 nodeLabelExpression: "" containerType: TASK executionType: GUARANTEED 2016-08-11 02:33:04,176 DEBUG [main] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 2 cluster_timestamp: 100 } attemptId: 2 } id: 6 } nmHostAddr: "test:123" appSubmitter: "user2" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1470863564153 masterKeyId: 100511640 rmIdentifier: 1470862883498 priority { priority: 1 } creationTime: 1470862964133 nodeLabelExpression: "" containerType: TASK executionType: GUARANTEED 2016-08-11 02:33:04,177 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0002_02_000006 of type ACQUIRED 2016-08-11 02:33:04,178 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppRunningOnNodeEvent.EventType: APP_RUNNING_ON_NODE 2016-08-11 02:33:04,178 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0002_02_000006 Container Transitioned from ALLOCATED to ACQUIRED 2016-08-11 02:33:04,179 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:33:04,181 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1282)) - Trying to schedule on node: 127.0.0.1, available: 2016-08-11 02:33:04,181 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(437)) - Trying to assign containers to child-queue of root 2016-08-11 02:33:04,181 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.15625238), label=(*)root.busedCapacity=(1.8754579), label=(*) 2016-08-11 02:33:04,182 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 2016-08-11 02:33:04,182 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=127.0.0.1 #applications=1 2016-08-11 02:33:04,183 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(909)) - Skip this queue=root.a, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 02:33:04,184 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 --> , NODE_LOCAL 2016-08-11 02:33:04,185 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=1.8754579, absoluteUsedCapacity=0.375, numApps=1, numContainers=6 2016-08-11 02:33:04,185 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=127.0.0.1 #applications=1 2016-08-11 02:33:04,186 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:33:04,186 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimitAndSetHeadroom(1120)) - Headroom calculation for user user2: userLimit= queueMaxAvailRes= consumed= headroom= 2016-08-11 02:33:04,187 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:assignContainers(511)) - pre-assignContainers for application application_100_0002 2016-08-11 02:33:04,188 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:33:04,188 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=6144 2016-08-11 02:33:04,189 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:33:04,189 DEBUG [main] allocator.IncreaseContainerAllocator (IncreaseContainerAllocator.java:assignContainers(187)) - Skip allocating increase request since we don't have any increase request on this node=127.0.0.1:123 2016-08-11 02:33:04,190 DEBUG [main] allocator.RegularContainerAllocator (RegularContainerAllocator.java:assignContainer(473)) - assignContainers: node=127.0.0.1 application=application_100_0002 priority=1 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } type=OFF_SWITCH 2016-08-11 02:33:04,191 DEBUG [main] scheduler.ActiveUsersManager (ActiveUsersManager.java:deactivateApplication(94)) - User user2 removed from activeUsers, currently: 0 2016-08-11 02:33:04,192 DEBUG [main] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:allocate(631)) - allocate: applicationId=application_100_0002 container=container_100_0002_02_000007 host=127.0.0.1:123 user=user2 resource= type=OFF_SWITCH 2016-08-11 02:33:04,193 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0002_02_000007 of type START 2016-08-11 02:33:04,193 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: CONTAINER_ALLOCATED 2016-08-11 02:33:04,194 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0002_02_000007 Container Transitioned from NEW to ALLOCATED 2016-08-11 02:33:04,194 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:allocate(237)) - allocate: applicationAttemptId=appattempt_100_0002_000002 container=container_100_0002_02_000007 host=127.0.0.1 type=OFF_SWITCH 2016-08-11 02:33:04,195 INFO [main] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(156)) - USER=user2 OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_100_0002 CONTAINERID=container_100_0002_02_000007 RESOURCE= 2016-08-11 02:33:04,195 DEBUG [main] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(157)) - Assigned container container_100_0002_02_000007 of capacity on host 127.0.0.1:1, which has 2 containers, used and available after allocation 2016-08-11 02:33:04,197 INFO [main] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(120)) - assignedContainer application attempt=appattempt_100_0002_000002 container=container_100_0002_02_000007 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@22356acd clusterResource= type=OFF_SWITCH 2016-08-11 02:33:04,197 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(962)) - post-assignContainers for application application_100_0002 2016-08-11 02:33:04,198 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:33:04,198 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=7168 2016-08-11 02:33:04,198 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 0, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:33:04,199 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:33:04,199 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:33:04,200 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:33:04,200 DEBUG [main] capacity.LeafQueue (LeafQueue.java:allocateResource(1443)) - b user=user2 used= numContainers=7 headroom = user-resources= 2016-08-11 02:33:04,200 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=2.1880343, absoluteUsedCapacity=0.4375, numApps=1, numContainers=7 --> , OFF_SWITCH 2016-08-11 02:33:04,201 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(670)) - Re-sorting assigned queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=2.1880343, absoluteUsedCapacity=0.4375, numApps=1, numContainers=7 2016-08-11 02:33:04,201 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.15625238), label=(*)root.busedCapacity=(2.1880343), label=(*) 2016-08-11 02:33:04,202 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainers(522)) - assignedContainer queue=root usedCapacity=0.5625 absoluteUsedCapacity=0.5625 used= cluster= 2016-08-11 02:33:04,202 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(544)) - ParentQ=root assignedSoFarInThisIteration= usedCapacity=0.5625 absoluteUsedCapacity=0.5625 2016-08-11 02:33:04,202 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(555)) - Not assigning more than one off-switch container, assignments so far: resource:; type:OFF_SWITCH; excessReservation:null; applicationid:null; skipped:NONE; fulfilled reservation:false; allocations(count/resource):1/; reservations(count/resource):0/ 2016-08-11 02:33:04,202 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1282)) - Trying to schedule on node: test, available: 2016-08-11 02:33:04,202 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(416)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 02:33:04,203 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1282)) - Trying to schedule on node: 127.0.0.1, available: 2016-08-11 02:33:04,203 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(416)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 02:33:04,203 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1282)) - Trying to schedule on node: test, available: 2016-08-11 02:33:04,203 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(416)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 02:33:04,212 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(976)) - allocate: pre-update appattempt_100_0002_000002 ask size =1 2016-08-11 02:33:04,213 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:33:04,213 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=7168 2016-08-11 02:33:04,213 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 0, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:33:04,214 DEBUG [main] scheduler.ActiveUsersManager (ActiveUsersManager.java:activateApplication(68)) - User user2 added to activeUsers, currently: 1 2016-08-11 02:33:04,214 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(988)) - allocate: post-update 2016-08-11 02:33:04,215 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:33:04,215 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=7168 2016-08-11 02:33:04,215 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:33:04,216 DEBUG [main] security.BaseContainerTokenSecretManager (BaseContainerTokenSecretManager.java:createPassword(90)) - Creating password for container_100_0002_02_000007 for user container_100_0002_02_000007 (auth:SIMPLE) to be run on NM 127.0.0.1:123 2016-08-11 02:33:04,217 DEBUG [main] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 2 cluster_timestamp: 100 } attemptId: 2 } id: 7 } nmHostAddr: "127.0.0.1:123" appSubmitter: "user2" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1470863584215 masterKeyId: 100511640 rmIdentifier: 1470862883498 priority { priority: 1 } creationTime: 1470862984190 nodeLabelExpression: "" containerType: TASK executionType: GUARANTEED 2016-08-11 02:33:04,218 DEBUG [main] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 2 cluster_timestamp: 100 } attemptId: 2 } id: 7 } nmHostAddr: "127.0.0.1:123" appSubmitter: "user2" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1470863584215 masterKeyId: 100511640 rmIdentifier: 1470862883498 priority { priority: 1 } creationTime: 1470862984190 nodeLabelExpression: "" containerType: TASK executionType: GUARANTEED 2016-08-11 02:33:04,219 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0002_02_000007 of type ACQUIRED 2016-08-11 02:33:04,220 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0002_02_000007 Container Transitioned from ALLOCATED to ACQUIRED 2016-08-11 02:33:04,220 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppRunningOnNodeEvent.EventType: APP_RUNNING_ON_NODE 2016-08-11 02:33:04,220 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:33:04,222 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1282)) - Trying to schedule on node: test, available: 2016-08-11 02:33:04,222 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(437)) - Trying to assign containers to child-queue of root 2016-08-11 02:33:04,223 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.15625238), label=(*)root.busedCapacity=(2.1880343), label=(*) 2016-08-11 02:33:04,223 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 2016-08-11 02:33:04,224 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=test #applications=1 2016-08-11 02:33:04,224 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(909)) - Skip this queue=root.a, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 02:33:04,224 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 --> , NODE_LOCAL 2016-08-11 02:33:04,224 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=2.1880343, absoluteUsedCapacity=0.4375, numApps=1, numContainers=7 2016-08-11 02:33:04,225 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=test #applications=1 2016-08-11 02:33:04,225 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:33:04,225 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimitAndSetHeadroom(1120)) - Headroom calculation for user user2: userLimit= queueMaxAvailRes= consumed= headroom= 2016-08-11 02:33:04,226 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:assignContainers(511)) - pre-assignContainers for application application_100_0002 2016-08-11 02:33:04,226 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:33:04,226 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=7168 2016-08-11 02:33:04,227 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:33:04,227 DEBUG [main] allocator.IncreaseContainerAllocator (IncreaseContainerAllocator.java:assignContainers(187)) - Skip allocating increase request since we don't have any increase request on this node=test:123 2016-08-11 02:33:04,227 DEBUG [main] allocator.RegularContainerAllocator (RegularContainerAllocator.java:assignContainer(473)) - assignContainers: node=test application=application_100_0002 priority=1 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } type=OFF_SWITCH 2016-08-11 02:33:04,228 DEBUG [main] scheduler.ActiveUsersManager (ActiveUsersManager.java:deactivateApplication(94)) - User user2 removed from activeUsers, currently: 0 2016-08-11 02:33:04,228 DEBUG [main] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:allocate(631)) - allocate: applicationId=application_100_0002 container=container_100_0002_02_000008 host=test:123 user=user2 resource= type=OFF_SWITCH 2016-08-11 02:33:04,228 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0002_02_000008 of type START 2016-08-11 02:33:04,228 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0002_02_000008 Container Transitioned from NEW to ALLOCATED 2016-08-11 02:33:04,228 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: CONTAINER_ALLOCATED 2016-08-11 02:33:04,229 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:allocate(237)) - allocate: applicationAttemptId=appattempt_100_0002_000002 container=container_100_0002_02_000008 host=test type=OFF_SWITCH 2016-08-11 02:33:04,229 INFO [main] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(156)) - USER=user2 OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_100_0002 CONTAINERID=container_100_0002_02_000008 RESOURCE= 2016-08-11 02:33:04,229 DEBUG [main] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(157)) - Assigned container container_100_0002_02_000008 of capacity on host test:2, which has 7 containers, used and available after allocation 2016-08-11 02:33:04,230 INFO [main] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(120)) - assignedContainer application attempt=appattempt_100_0002_000002 container=container_100_0002_02_000008 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@22356acd clusterResource= type=OFF_SWITCH 2016-08-11 02:33:04,230 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(962)) - post-assignContainers for application application_100_0002 2016-08-11 02:33:04,230 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:33:04,231 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=8192 2016-08-11 02:33:04,231 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 0, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:33:04,231 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:33:04,232 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:33:04,232 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:33:04,232 DEBUG [main] capacity.LeafQueue (LeafQueue.java:allocateResource(1443)) - b user=user2 used= numContainers=8 headroom = user-resources= 2016-08-11 02:33:04,232 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=2.5006106, absoluteUsedCapacity=0.5, numApps=1, numContainers=8 --> , OFF_SWITCH 2016-08-11 02:33:04,233 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(670)) - Re-sorting assigned queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=2.5006106, absoluteUsedCapacity=0.5, numApps=1, numContainers=8 2016-08-11 02:33:04,233 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.15625238), label=(*)root.busedCapacity=(2.5006106), label=(*) 2016-08-11 02:33:04,233 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainers(522)) - assignedContainer queue=root usedCapacity=0.625 absoluteUsedCapacity=0.625 used= cluster= 2016-08-11 02:33:04,234 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(544)) - ParentQ=root assignedSoFarInThisIteration= usedCapacity=0.625 absoluteUsedCapacity=0.625 2016-08-11 02:33:04,234 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(555)) - Not assigning more than one off-switch container, assignments so far: resource:; type:OFF_SWITCH; excessReservation:null; applicationid:null; skipped:NONE; fulfilled reservation:false; allocations(count/resource):1/; reservations(count/resource):0/ 2016-08-11 02:33:04,234 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1282)) - Trying to schedule on node: 127.0.0.1, available: 2016-08-11 02:33:04,234 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(416)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 02:33:04,240 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(976)) - allocate: pre-update appattempt_100_0002_000002 ask size =1 2016-08-11 02:33:04,240 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:33:04,240 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=8192 2016-08-11 02:33:04,241 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 0, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:33:04,241 DEBUG [main] scheduler.ActiveUsersManager (ActiveUsersManager.java:activateApplication(68)) - User user2 added to activeUsers, currently: 1 2016-08-11 02:33:04,241 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(988)) - allocate: post-update 2016-08-11 02:33:04,242 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:33:04,242 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=8192 2016-08-11 02:33:04,242 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:33:04,243 DEBUG [main] security.BaseContainerTokenSecretManager (BaseContainerTokenSecretManager.java:createPassword(90)) - Creating password for container_100_0002_02_000008 for user container_100_0002_02_000008 (auth:SIMPLE) to be run on NM test:123 2016-08-11 02:33:04,243 DEBUG [main] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 2 cluster_timestamp: 100 } attemptId: 2 } id: 8 } nmHostAddr: "test:123" appSubmitter: "user2" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1470863584242 masterKeyId: 100511640 rmIdentifier: 1470862883498 priority { priority: 1 } creationTime: 1470862984228 nodeLabelExpression: "" containerType: TASK executionType: GUARANTEED 2016-08-11 02:33:24,293 DEBUG [main] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 2 cluster_timestamp: 100 } attemptId: 2 } id: 8 } nmHostAddr: "test:123" appSubmitter: "user2" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1470863584242 masterKeyId: 100511640 rmIdentifier: 1470862883498 priority { priority: 1 } creationTime: 1470862984228 nodeLabelExpression: "" containerType: TASK executionType: GUARANTEED 2016-08-11 02:33:24,294 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0002_02_000008 of type ACQUIRED 2016-08-11 02:33:24,295 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0002_02_000008 Container Transitioned from ALLOCATED to ACQUIRED 2016-08-11 02:33:24,295 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppRunningOnNodeEvent.EventType: APP_RUNNING_ON_NODE 2016-08-11 02:33:24,296 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:33:24,298 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1282)) - Trying to schedule on node: 127.0.0.1, available: 2016-08-11 02:33:24,299 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(437)) - Trying to assign containers to child-queue of root 2016-08-11 02:33:24,299 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.15625238), label=(*)root.busedCapacity=(2.5006106), label=(*) 2016-08-11 02:33:24,300 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 2016-08-11 02:33:24,300 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=127.0.0.1 #applications=1 2016-08-11 02:33:24,301 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(909)) - Skip this queue=root.a, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 02:33:24,301 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 --> , NODE_LOCAL 2016-08-11 02:33:24,302 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=2.5006106, absoluteUsedCapacity=0.5, numApps=1, numContainers=8 2016-08-11 02:33:24,302 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=127.0.0.1 #applications=1 2016-08-11 02:33:24,303 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:33:24,304 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimitAndSetHeadroom(1120)) - Headroom calculation for user user2: userLimit= queueMaxAvailRes= consumed= headroom= 2016-08-11 02:33:24,304 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:assignContainers(511)) - pre-assignContainers for application application_100_0002 2016-08-11 02:33:24,305 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:33:24,305 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=8192 2016-08-11 02:33:24,306 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:33:24,306 DEBUG [main] allocator.IncreaseContainerAllocator (IncreaseContainerAllocator.java:assignContainers(187)) - Skip allocating increase request since we don't have any increase request on this node=127.0.0.1:123 2016-08-11 02:33:24,307 DEBUG [main] allocator.RegularContainerAllocator (RegularContainerAllocator.java:assignContainer(473)) - assignContainers: node=127.0.0.1 application=application_100_0002 priority=1 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } type=OFF_SWITCH 2016-08-11 02:33:24,308 DEBUG [main] scheduler.ActiveUsersManager (ActiveUsersManager.java:deactivateApplication(94)) - User user2 removed from activeUsers, currently: 0 2016-08-11 02:33:24,308 DEBUG [main] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:allocate(631)) - allocate: applicationId=application_100_0002 container=container_100_0002_02_000009 host=127.0.0.1:123 user=user2 resource= type=OFF_SWITCH 2016-08-11 02:33:24,309 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0002_02_000009 of type START 2016-08-11 02:33:24,309 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0002_02_000009 Container Transitioned from NEW to ALLOCATED 2016-08-11 02:33:24,309 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: CONTAINER_ALLOCATED 2016-08-11 02:33:24,310 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:allocate(237)) - allocate: applicationAttemptId=appattempt_100_0002_000002 container=container_100_0002_02_000009 host=127.0.0.1 type=OFF_SWITCH 2016-08-11 02:33:24,311 INFO [main] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(156)) - USER=user2 OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_100_0002 CONTAINERID=container_100_0002_02_000009 RESOURCE= 2016-08-11 02:33:24,311 DEBUG [main] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(157)) - Assigned container container_100_0002_02_000009 of capacity on host 127.0.0.1:1, which has 3 containers, used and available after allocation 2016-08-11 02:33:24,313 INFO [main] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(120)) - assignedContainer application attempt=appattempt_100_0002_000002 container=container_100_0002_02_000009 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@22356acd clusterResource= type=OFF_SWITCH 2016-08-11 02:33:24,314 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(962)) - post-assignContainers for application application_100_0002 2016-08-11 02:33:24,314 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:33:24,315 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=9216 2016-08-11 02:33:24,315 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 0, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:33:24,316 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:33:24,316 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:33:24,316 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:33:24,317 DEBUG [main] capacity.LeafQueue (LeafQueue.java:allocateResource(1443)) - b user=user2 used= numContainers=9 headroom = user-resources= 2016-08-11 02:33:24,317 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=2.813187, absoluteUsedCapacity=0.5625, numApps=1, numContainers=9 --> , OFF_SWITCH 2016-08-11 02:33:24,318 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(670)) - Re-sorting assigned queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=2.813187, absoluteUsedCapacity=0.5625, numApps=1, numContainers=9 2016-08-11 02:33:24,318 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.15625238), label=(*)root.busedCapacity=(2.813187), label=(*) 2016-08-11 02:33:24,319 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainers(522)) - assignedContainer queue=root usedCapacity=0.6875 absoluteUsedCapacity=0.6875 used= cluster= 2016-08-11 02:33:24,319 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(544)) - ParentQ=root assignedSoFarInThisIteration= usedCapacity=0.6875 absoluteUsedCapacity=0.6875 2016-08-11 02:33:24,319 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(555)) - Not assigning more than one off-switch container, assignments so far: resource:; type:OFF_SWITCH; excessReservation:null; applicationid:null; skipped:NONE; fulfilled reservation:false; allocations(count/resource):1/; reservations(count/resource):0/ 2016-08-11 02:33:24,325 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(976)) - allocate: pre-update appattempt_100_0002_000002 ask size =1 2016-08-11 02:33:24,325 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:33:24,325 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=9216 2016-08-11 02:33:24,326 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 0, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:33:24,326 DEBUG [main] scheduler.ActiveUsersManager (ActiveUsersManager.java:activateApplication(68)) - User user2 added to activeUsers, currently: 1 2016-08-11 02:33:24,326 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(988)) - allocate: post-update 2016-08-11 02:33:24,326 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:33:24,327 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=9216 2016-08-11 02:33:24,327 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:33:24,327 DEBUG [main] security.BaseContainerTokenSecretManager (BaseContainerTokenSecretManager.java:createPassword(90)) - Creating password for container_100_0002_02_000009 for user container_100_0002_02_000009 (auth:SIMPLE) to be run on NM 127.0.0.1:123 2016-08-11 02:33:24,329 DEBUG [main] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 2 cluster_timestamp: 100 } attemptId: 2 } id: 9 } nmHostAddr: "127.0.0.1:123" appSubmitter: "user2" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1470863604327 masterKeyId: 100511640 rmIdentifier: 1470862883498 priority { priority: 1 } creationTime: 1470863004308 nodeLabelExpression: "" containerType: TASK executionType: GUARANTEED 2016-08-11 02:33:24,330 DEBUG [main] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 2 cluster_timestamp: 100 } attemptId: 2 } id: 9 } nmHostAddr: "127.0.0.1:123" appSubmitter: "user2" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1470863604327 masterKeyId: 100511640 rmIdentifier: 1470862883498 priority { priority: 1 } creationTime: 1470863004308 nodeLabelExpression: "" containerType: TASK executionType: GUARANTEED 2016-08-11 02:33:24,330 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0002_02_000009 of type ACQUIRED 2016-08-11 02:33:24,330 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0002_02_000009 Container Transitioned from ALLOCATED to ACQUIRED 2016-08-11 02:33:24,330 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppRunningOnNodeEvent.EventType: APP_RUNNING_ON_NODE 2016-08-11 02:33:24,331 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:33:24,332 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1282)) - Trying to schedule on node: 127.0.0.1, available: 2016-08-11 02:33:24,332 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(437)) - Trying to assign containers to child-queue of root 2016-08-11 02:33:24,332 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.15625238), label=(*)root.busedCapacity=(2.813187), label=(*) 2016-08-11 02:33:24,332 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 2016-08-11 02:33:24,333 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=127.0.0.1 #applications=1 2016-08-11 02:33:24,333 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(909)) - Skip this queue=root.a, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 02:33:24,333 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 --> , NODE_LOCAL 2016-08-11 02:33:24,333 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=2.813187, absoluteUsedCapacity=0.5625, numApps=1, numContainers=9 2016-08-11 02:33:24,333 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=127.0.0.1 #applications=1 2016-08-11 02:33:24,334 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:33:24,334 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimitAndSetHeadroom(1120)) - Headroom calculation for user user2: userLimit= queueMaxAvailRes= consumed= headroom= 2016-08-11 02:33:24,334 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:assignContainers(511)) - pre-assignContainers for application application_100_0002 2016-08-11 02:33:24,334 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:33:24,334 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=9216 2016-08-11 02:33:24,335 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:33:24,335 DEBUG [main] allocator.IncreaseContainerAllocator (IncreaseContainerAllocator.java:assignContainers(187)) - Skip allocating increase request since we don't have any increase request on this node=127.0.0.1:123 2016-08-11 02:33:24,335 DEBUG [main] allocator.RegularContainerAllocator (RegularContainerAllocator.java:assignContainer(473)) - assignContainers: node=127.0.0.1 application=application_100_0002 priority=1 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } type=OFF_SWITCH 2016-08-11 02:33:24,335 DEBUG [main] scheduler.ActiveUsersManager (ActiveUsersManager.java:deactivateApplication(94)) - User user2 removed from activeUsers, currently: 0 2016-08-11 02:33:24,336 DEBUG [main] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:allocate(631)) - allocate: applicationId=application_100_0002 container=container_100_0002_02_000010 host=127.0.0.1:123 user=user2 resource= type=OFF_SWITCH 2016-08-11 02:33:24,336 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0002_02_000010 of type START 2016-08-11 02:33:24,336 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0002_02_000010 Container Transitioned from NEW to ALLOCATED 2016-08-11 02:33:24,336 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:allocate(237)) - allocate: applicationAttemptId=appattempt_100_0002_000002 container=container_100_0002_02_000010 host=127.0.0.1 type=OFF_SWITCH 2016-08-11 02:33:24,336 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: CONTAINER_ALLOCATED 2016-08-11 02:33:24,337 INFO [main] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(156)) - USER=user2 OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_100_0002 CONTAINERID=container_100_0002_02_000010 RESOURCE= 2016-08-11 02:33:24,337 DEBUG [main] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(157)) - Assigned container container_100_0002_02_000010 of capacity on host 127.0.0.1:1, which has 4 containers, used and available after allocation 2016-08-11 02:33:24,338 INFO [main] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(120)) - assignedContainer application attempt=appattempt_100_0002_000002 container=container_100_0002_02_000010 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@22356acd clusterResource= type=OFF_SWITCH 2016-08-11 02:33:24,338 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(962)) - post-assignContainers for application application_100_0002 2016-08-11 02:33:24,338 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:33:24,339 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=10240 2016-08-11 02:33:24,339 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 0, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:33:24,339 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:33:24,340 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:33:24,340 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:33:24,340 DEBUG [main] capacity.LeafQueue (LeafQueue.java:allocateResource(1443)) - b user=user2 used= numContainers=10 headroom = user-resources= 2016-08-11 02:33:24,340 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=3.1257632, absoluteUsedCapacity=0.625, numApps=1, numContainers=10 --> , OFF_SWITCH 2016-08-11 02:33:24,340 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(670)) - Re-sorting assigned queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=3.1257632, absoluteUsedCapacity=0.625, numApps=1, numContainers=10 2016-08-11 02:33:24,341 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.15625238), label=(*)root.busedCapacity=(3.1257632), label=(*) 2016-08-11 02:33:24,341 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainers(522)) - assignedContainer queue=root usedCapacity=0.75 absoluteUsedCapacity=0.75 used= cluster= 2016-08-11 02:33:24,341 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(544)) - ParentQ=root assignedSoFarInThisIteration= usedCapacity=0.75 absoluteUsedCapacity=0.75 2016-08-11 02:33:24,341 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(555)) - Not assigning more than one off-switch container, assignments so far: resource:; type:OFF_SWITCH; excessReservation:null; applicationid:null; skipped:NONE; fulfilled reservation:false; allocations(count/resource):1/; reservations(count/resource):0/ 2016-08-11 02:33:24,347 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(976)) - allocate: pre-update appattempt_100_0002_000002 ask size =1 2016-08-11 02:33:24,348 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:33:24,348 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=10240 2016-08-11 02:33:24,348 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 0, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:33:24,348 DEBUG [main] scheduler.ActiveUsersManager (ActiveUsersManager.java:activateApplication(68)) - User user2 added to activeUsers, currently: 1 2016-08-11 02:33:24,349 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(988)) - allocate: post-update 2016-08-11 02:33:24,349 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:33:24,349 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=10240 2016-08-11 02:33:24,349 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:33:24,350 DEBUG [main] security.BaseContainerTokenSecretManager (BaseContainerTokenSecretManager.java:createPassword(90)) - Creating password for container_100_0002_02_000010 for user container_100_0002_02_000010 (auth:SIMPLE) to be run on NM 127.0.0.1:123 2016-08-11 02:33:24,351 DEBUG [main] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 2 cluster_timestamp: 100 } attemptId: 2 } id: 10 } nmHostAddr: "127.0.0.1:123" appSubmitter: "user2" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1470863604350 masterKeyId: 100511640 rmIdentifier: 1470862883498 priority { priority: 1 } creationTime: 1470863004335 nodeLabelExpression: "" containerType: TASK executionType: GUARANTEED 2016-08-11 02:33:24,352 DEBUG [main] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 2 cluster_timestamp: 100 } attemptId: 2 } id: 10 } nmHostAddr: "127.0.0.1:123" appSubmitter: "user2" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1470863604350 masterKeyId: 100511640 rmIdentifier: 1470862883498 priority { priority: 1 } creationTime: 1470863004335 nodeLabelExpression: "" containerType: TASK executionType: GUARANTEED 2016-08-11 02:33:24,353 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0002_02_000010 of type ACQUIRED 2016-08-11 02:33:24,353 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0002_02_000010 Container Transitioned from ALLOCATED to ACQUIRED 2016-08-11 02:33:24,353 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppRunningOnNodeEvent.EventType: APP_RUNNING_ON_NODE 2016-08-11 02:33:24,358 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:33:24,359 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1282)) - Trying to schedule on node: 127.0.0.1, available: 2016-08-11 02:33:24,359 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(437)) - Trying to assign containers to child-queue of root 2016-08-11 02:33:24,360 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.15625238), label=(*)root.busedCapacity=(3.1257632), label=(*) 2016-08-11 02:33:24,360 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 2016-08-11 02:33:24,360 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=127.0.0.1 #applications=1 2016-08-11 02:33:24,360 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(909)) - Skip this queue=root.a, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 02:33:24,360 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 --> , NODE_LOCAL 2016-08-11 02:33:24,360 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=3.1257632, absoluteUsedCapacity=0.625, numApps=1, numContainers=10 2016-08-11 02:33:24,360 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=127.0.0.1 #applications=1 2016-08-11 02:33:24,361 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:33:24,361 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimitAndSetHeadroom(1120)) - Headroom calculation for user user2: userLimit= queueMaxAvailRes= consumed= headroom= 2016-08-11 02:33:24,361 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:assignContainers(511)) - pre-assignContainers for application application_100_0002 2016-08-11 02:33:24,361 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:33:24,361 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=10240 2016-08-11 02:33:24,361 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:33:24,361 DEBUG [main] allocator.IncreaseContainerAllocator (IncreaseContainerAllocator.java:assignContainers(187)) - Skip allocating increase request since we don't have any increase request on this node=127.0.0.1:123 2016-08-11 02:33:24,362 DEBUG [main] allocator.RegularContainerAllocator (RegularContainerAllocator.java:assignContainer(473)) - assignContainers: node=127.0.0.1 application=application_100_0002 priority=1 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } type=OFF_SWITCH 2016-08-11 02:33:24,362 DEBUG [main] scheduler.ActiveUsersManager (ActiveUsersManager.java:deactivateApplication(94)) - User user2 removed from activeUsers, currently: 0 2016-08-11 02:33:24,362 DEBUG [main] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:allocate(631)) - allocate: applicationId=application_100_0002 container=container_100_0002_02_000011 host=127.0.0.1:123 user=user2 resource= type=OFF_SWITCH 2016-08-11 02:33:24,362 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0002_02_000011 of type START 2016-08-11 02:33:24,362 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0002_02_000011 Container Transitioned from NEW to ALLOCATED 2016-08-11 02:33:24,362 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: CONTAINER_ALLOCATED 2016-08-11 02:33:24,363 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:allocate(237)) - allocate: applicationAttemptId=appattempt_100_0002_000002 container=container_100_0002_02_000011 host=127.0.0.1 type=OFF_SWITCH 2016-08-11 02:33:24,363 INFO [main] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(156)) - USER=user2 OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_100_0002 CONTAINERID=container_100_0002_02_000011 RESOURCE= 2016-08-11 02:33:24,363 DEBUG [main] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(157)) - Assigned container container_100_0002_02_000011 of capacity on host 127.0.0.1:1, which has 5 containers, used and available after allocation 2016-08-11 02:33:24,363 INFO [main] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(120)) - assignedContainer application attempt=appattempt_100_0002_000002 container=container_100_0002_02_000011 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@22356acd clusterResource= type=OFF_SWITCH 2016-08-11 02:33:24,364 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(962)) - post-assignContainers for application application_100_0002 2016-08-11 02:33:24,364 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:33:24,364 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=11264 2016-08-11 02:33:24,364 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 0, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:33:24,364 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:33:24,364 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:33:24,365 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:33:24,365 DEBUG [main] capacity.LeafQueue (LeafQueue.java:allocateResource(1443)) - b user=user2 used= numContainers=11 headroom = user-resources= 2016-08-11 02:33:24,365 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=3.4383395, absoluteUsedCapacity=0.6875, numApps=1, numContainers=11 --> , OFF_SWITCH 2016-08-11 02:33:24,365 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(670)) - Re-sorting assigned queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=3.4383395, absoluteUsedCapacity=0.6875, numApps=1, numContainers=11 2016-08-11 02:33:24,365 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.15625238), label=(*)root.busedCapacity=(3.4383395), label=(*) 2016-08-11 02:33:24,365 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainers(522)) - assignedContainer queue=root usedCapacity=0.8125 absoluteUsedCapacity=0.8125 used= cluster= 2016-08-11 02:33:24,366 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(544)) - ParentQ=root assignedSoFarInThisIteration= usedCapacity=0.8125 absoluteUsedCapacity=0.8125 2016-08-11 02:33:24,366 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(555)) - Not assigning more than one off-switch container, assignments so far: resource:; type:OFF_SWITCH; excessReservation:null; applicationid:null; skipped:NONE; fulfilled reservation:false; allocations(count/resource):1/; reservations(count/resource):0/ 2016-08-11 02:33:24,371 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(976)) - allocate: pre-update appattempt_100_0002_000002 ask size =1 2016-08-11 02:33:24,371 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:33:24,371 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=11264 2016-08-11 02:33:24,372 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 0, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:33:24,372 DEBUG [main] scheduler.ActiveUsersManager (ActiveUsersManager.java:activateApplication(68)) - User user2 added to activeUsers, currently: 1 2016-08-11 02:33:24,372 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(988)) - allocate: post-update 2016-08-11 02:33:24,372 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:33:24,372 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=11264 2016-08-11 02:33:24,372 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:33:24,372 DEBUG [main] security.BaseContainerTokenSecretManager (BaseContainerTokenSecretManager.java:createPassword(90)) - Creating password for container_100_0002_02_000011 for user container_100_0002_02_000011 (auth:SIMPLE) to be run on NM 127.0.0.1:123 2016-08-11 02:33:24,373 DEBUG [main] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 2 cluster_timestamp: 100 } attemptId: 2 } id: 11 } nmHostAddr: "127.0.0.1:123" appSubmitter: "user2" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1470863604372 masterKeyId: 100511640 rmIdentifier: 1470862883498 priority { priority: 1 } creationTime: 1470863004362 nodeLabelExpression: "" containerType: TASK executionType: GUARANTEED 2016-08-11 02:33:24,373 DEBUG [main] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 2 cluster_timestamp: 100 } attemptId: 2 } id: 11 } nmHostAddr: "127.0.0.1:123" appSubmitter: "user2" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1470863604372 masterKeyId: 100511640 rmIdentifier: 1470862883498 priority { priority: 1 } creationTime: 1470863004362 nodeLabelExpression: "" containerType: TASK executionType: GUARANTEED 2016-08-11 02:33:24,374 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0002_02_000011 of type ACQUIRED 2016-08-11 02:33:24,374 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0002_02_000011 Container Transitioned from ALLOCATED to ACQUIRED 2016-08-11 02:33:24,374 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppRunningOnNodeEvent.EventType: APP_RUNNING_ON_NODE 2016-08-11 02:33:24,374 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:33:24,375 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1282)) - Trying to schedule on node: 127.0.0.1, available: 2016-08-11 02:33:24,375 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(437)) - Trying to assign containers to child-queue of root 2016-08-11 02:33:24,375 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.15625238), label=(*)root.busedCapacity=(3.4383395), label=(*) 2016-08-11 02:33:24,375 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 2016-08-11 02:33:24,375 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=127.0.0.1 #applications=1 2016-08-11 02:33:24,376 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(909)) - Skip this queue=root.a, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 02:33:24,376 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 --> , NODE_LOCAL 2016-08-11 02:33:24,376 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=3.4383395, absoluteUsedCapacity=0.6875, numApps=1, numContainers=11 2016-08-11 02:33:24,376 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=127.0.0.1 #applications=1 2016-08-11 02:33:24,376 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:33:24,376 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimitAndSetHeadroom(1120)) - Headroom calculation for user user2: userLimit= queueMaxAvailRes= consumed= headroom= 2016-08-11 02:33:24,376 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:assignContainers(511)) - pre-assignContainers for application application_100_0002 2016-08-11 02:33:24,376 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:33:24,376 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=11264 2016-08-11 02:33:24,377 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:33:24,377 DEBUG [main] allocator.IncreaseContainerAllocator (IncreaseContainerAllocator.java:assignContainers(187)) - Skip allocating increase request since we don't have any increase request on this node=127.0.0.1:123 2016-08-11 02:33:24,377 DEBUG [main] allocator.RegularContainerAllocator (RegularContainerAllocator.java:assignContainer(473)) - assignContainers: node=127.0.0.1 application=application_100_0002 priority=1 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } type=OFF_SWITCH 2016-08-11 02:33:24,377 DEBUG [main] scheduler.ActiveUsersManager (ActiveUsersManager.java:deactivateApplication(94)) - User user2 removed from activeUsers, currently: 0 2016-08-11 02:33:24,377 DEBUG [main] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:allocate(631)) - allocate: applicationId=application_100_0002 container=container_100_0002_02_000012 host=127.0.0.1:123 user=user2 resource= type=OFF_SWITCH 2016-08-11 02:33:24,377 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0002_02_000012 of type START 2016-08-11 02:33:24,378 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0002_02_000012 Container Transitioned from NEW to ALLOCATED 2016-08-11 02:33:24,378 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: CONTAINER_ALLOCATED 2016-08-11 02:33:24,378 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:allocate(237)) - allocate: applicationAttemptId=appattempt_100_0002_000002 container=container_100_0002_02_000012 host=127.0.0.1 type=OFF_SWITCH 2016-08-11 02:33:24,378 INFO [main] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(156)) - USER=user2 OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_100_0002 CONTAINERID=container_100_0002_02_000012 RESOURCE= 2016-08-11 02:33:24,378 DEBUG [main] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(157)) - Assigned container container_100_0002_02_000012 of capacity on host 127.0.0.1:1, which has 6 containers, used and available after allocation 2016-08-11 02:33:24,379 INFO [main] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(120)) - assignedContainer application attempt=appattempt_100_0002_000002 container=container_100_0002_02_000012 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@22356acd clusterResource= type=OFF_SWITCH 2016-08-11 02:33:24,379 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(962)) - post-assignContainers for application application_100_0002 2016-08-11 02:33:24,379 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:33:24,379 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=12288 2016-08-11 02:33:24,380 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 0, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:33:24,380 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:33:24,380 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:33:24,380 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:33:24,380 DEBUG [main] capacity.LeafQueue (LeafQueue.java:allocateResource(1443)) - b user=user2 used= numContainers=12 headroom = user-resources= 2016-08-11 02:33:24,381 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=3.7509158, absoluteUsedCapacity=0.75, numApps=1, numContainers=12 --> , OFF_SWITCH 2016-08-11 02:33:24,381 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(670)) - Re-sorting assigned queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=3.7509158, absoluteUsedCapacity=0.75, numApps=1, numContainers=12 2016-08-11 02:33:24,381 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.15625238), label=(*)root.busedCapacity=(3.7509158), label=(*) 2016-08-11 02:33:24,381 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainers(522)) - assignedContainer queue=root usedCapacity=0.875 absoluteUsedCapacity=0.875 used= cluster= 2016-08-11 02:33:24,381 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(544)) - ParentQ=root assignedSoFarInThisIteration= usedCapacity=0.875 absoluteUsedCapacity=0.875 2016-08-11 02:33:24,381 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(555)) - Not assigning more than one off-switch container, assignments so far: resource:; type:OFF_SWITCH; excessReservation:null; applicationid:null; skipped:NONE; fulfilled reservation:false; allocations(count/resource):1/; reservations(count/resource):0/ 2016-08-11 02:33:24,387 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(976)) - allocate: pre-update appattempt_100_0002_000002 ask size =1 2016-08-11 02:33:24,387 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:33:24,387 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=12288 2016-08-11 02:33:24,388 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 0, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:33:24,388 DEBUG [main] scheduler.ActiveUsersManager (ActiveUsersManager.java:activateApplication(68)) - User user2 added to activeUsers, currently: 1 2016-08-11 02:33:24,388 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(988)) - allocate: post-update 2016-08-11 02:33:24,388 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:33:24,388 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=12288 2016-08-11 02:33:24,389 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:33:24,389 DEBUG [main] security.BaseContainerTokenSecretManager (BaseContainerTokenSecretManager.java:createPassword(90)) - Creating password for container_100_0002_02_000012 for user container_100_0002_02_000012 (auth:SIMPLE) to be run on NM 127.0.0.1:123 2016-08-11 02:33:24,389 DEBUG [main] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 2 cluster_timestamp: 100 } attemptId: 2 } id: 12 } nmHostAddr: "127.0.0.1:123" appSubmitter: "user2" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1470863604389 masterKeyId: 100511640 rmIdentifier: 1470862883498 priority { priority: 1 } creationTime: 1470863004377 nodeLabelExpression: "" containerType: TASK executionType: GUARANTEED 2016-08-11 02:33:24,390 DEBUG [main] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 2 cluster_timestamp: 100 } attemptId: 2 } id: 12 } nmHostAddr: "127.0.0.1:123" appSubmitter: "user2" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1470863604389 masterKeyId: 100511640 rmIdentifier: 1470862883498 priority { priority: 1 } creationTime: 1470863004377 nodeLabelExpression: "" containerType: TASK executionType: GUARANTEED 2016-08-11 02:33:24,390 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0002_02_000012 of type ACQUIRED 2016-08-11 02:33:24,391 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0002_02_000012 Container Transitioned from ALLOCATED to ACQUIRED 2016-08-11 02:33:24,391 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppRunningOnNodeEvent.EventType: APP_RUNNING_ON_NODE 2016-08-11 02:33:24,391 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:33:24,392 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1282)) - Trying to schedule on node: 127.0.0.1, available: 2016-08-11 02:33:24,392 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(437)) - Trying to assign containers to child-queue of root 2016-08-11 02:33:24,392 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.15625238), label=(*)root.busedCapacity=(3.7509158), label=(*) 2016-08-11 02:33:24,392 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 2016-08-11 02:33:24,392 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=127.0.0.1 #applications=1 2016-08-11 02:33:24,392 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(909)) - Skip this queue=root.a, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 02:33:24,393 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 --> , NODE_LOCAL 2016-08-11 02:33:24,393 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=3.7509158, absoluteUsedCapacity=0.75, numApps=1, numContainers=12 2016-08-11 02:33:24,393 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=127.0.0.1 #applications=1 2016-08-11 02:33:24,393 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:33:24,393 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimitAndSetHeadroom(1120)) - Headroom calculation for user user2: userLimit= queueMaxAvailRes= consumed= headroom= 2016-08-11 02:33:24,394 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:assignContainers(511)) - pre-assignContainers for application application_100_0002 2016-08-11 02:33:24,394 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:33:24,394 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=12288 2016-08-11 02:33:24,394 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:33:24,394 DEBUG [main] allocator.IncreaseContainerAllocator (IncreaseContainerAllocator.java:assignContainers(187)) - Skip allocating increase request since we don't have any increase request on this node=127.0.0.1:123 2016-08-11 02:33:24,395 DEBUG [main] allocator.RegularContainerAllocator (RegularContainerAllocator.java:assignContainer(473)) - assignContainers: node=127.0.0.1 application=application_100_0002 priority=1 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } type=OFF_SWITCH 2016-08-11 02:33:24,395 DEBUG [main] scheduler.ActiveUsersManager (ActiveUsersManager.java:deactivateApplication(94)) - User user2 removed from activeUsers, currently: 0 2016-08-11 02:33:24,395 DEBUG [main] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:allocate(631)) - allocate: applicationId=application_100_0002 container=container_100_0002_02_000013 host=127.0.0.1:123 user=user2 resource= type=OFF_SWITCH 2016-08-11 02:33:24,395 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0002_02_000013 of type START 2016-08-11 02:33:24,396 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0002_02_000013 Container Transitioned from NEW to ALLOCATED 2016-08-11 02:33:24,396 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: CONTAINER_ALLOCATED 2016-08-11 02:33:24,396 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:allocate(237)) - allocate: applicationAttemptId=appattempt_100_0002_000002 container=container_100_0002_02_000013 host=127.0.0.1 type=OFF_SWITCH 2016-08-11 02:33:24,396 INFO [main] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(156)) - USER=user2 OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_100_0002 CONTAINERID=container_100_0002_02_000013 RESOURCE= 2016-08-11 02:33:24,396 DEBUG [main] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(157)) - Assigned container container_100_0002_02_000013 of capacity on host 127.0.0.1:1, which has 7 containers, used and available after allocation 2016-08-11 02:33:24,397 INFO [main] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(120)) - assignedContainer application attempt=appattempt_100_0002_000002 container=container_100_0002_02_000013 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@22356acd clusterResource= type=OFF_SWITCH 2016-08-11 02:33:24,397 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(962)) - post-assignContainers for application application_100_0002 2016-08-11 02:33:24,397 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:33:24,397 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=13312 2016-08-11 02:33:24,398 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 0, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:33:24,398 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:33:24,398 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:33:24,398 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:33:24,398 DEBUG [main] capacity.LeafQueue (LeafQueue.java:allocateResource(1443)) - b user=user2 used= numContainers=13 headroom = user-resources= 2016-08-11 02:33:24,399 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=4.0634923, absoluteUsedCapacity=0.8125, numApps=1, numContainers=13 --> , OFF_SWITCH 2016-08-11 02:33:24,399 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(670)) - Re-sorting assigned queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=4.0634923, absoluteUsedCapacity=0.8125, numApps=1, numContainers=13 2016-08-11 02:33:24,399 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.15625238), label=(*)root.busedCapacity=(4.0634923), label=(*) 2016-08-11 02:33:24,399 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainers(522)) - assignedContainer queue=root usedCapacity=0.9375 absoluteUsedCapacity=0.9375 used= cluster= 2016-08-11 02:33:24,399 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(544)) - ParentQ=root assignedSoFarInThisIteration= usedCapacity=0.9375 absoluteUsedCapacity=0.9375 2016-08-11 02:33:24,400 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(555)) - Not assigning more than one off-switch container, assignments so far: resource:; type:OFF_SWITCH; excessReservation:null; applicationid:null; skipped:NONE; fulfilled reservation:false; allocations(count/resource):1/; reservations(count/resource):0/ 2016-08-11 02:33:24,405 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(976)) - allocate: pre-update appattempt_100_0001_000001 ask size =1 2016-08-11 02:33:24,405 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user1 in queue a userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:33:24,406 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0001 headRoom= currentConsumption=2048 2016-08-11 02:33:24,406 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0001 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 0, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:33:24,406 DEBUG [main] scheduler.ActiveUsersManager (ActiveUsersManager.java:activateApplication(68)) - User user1 added to activeUsers, currently: 1 2016-08-11 02:33:24,406 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(988)) - allocate: post-update 2016-08-11 02:33:24,406 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user1 in queue a userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:33:24,407 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0001 headRoom= currentConsumption=2048 2016-08-11 02:33:24,407 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0001 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:33:24,407 DEBUG [main] security.BaseContainerTokenSecretManager (BaseContainerTokenSecretManager.java:createPassword(90)) - Creating password for container_100_0001_01_000001 for user container_100_0001_01_000001 (auth:SIMPLE) to be run on NM test:123 2016-08-11 02:33:24,408 DEBUG [main] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 1 cluster_timestamp: 100 } attemptId: 1 } id: 1 } nmHostAddr: "test:123" appSubmitter: "user1" resource { memory: 2048 virtual_cores: 1 } expiryTimeStamp: 1470863604407 masterKeyId: 100511640 rmIdentifier: 1470862883498 priority { priority: 1 } creationTime: 1470862883716 nodeLabelExpression: "" containerType: TASK executionType: GUARANTEED 2016-08-11 02:33:44,427 DEBUG [main] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 1 cluster_timestamp: 100 } attemptId: 1 } id: 1 } nmHostAddr: "test:123" appSubmitter: "user1" resource { memory: 2048 virtual_cores: 1 } expiryTimeStamp: 1470863604407 masterKeyId: 100511640 rmIdentifier: 1470862883498 priority { priority: 1 } creationTime: 1470862883716 nodeLabelExpression: "" containerType: TASK executionType: GUARANTEED 2016-08-11 02:33:44,428 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0001_01_000001 of type ACQUIRED 2016-08-11 02:33:44,429 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0001_01_000001 Container Transitioned from ALLOCATED to ACQUIRED 2016-08-11 02:33:44,429 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppRunningOnNodeEvent.EventType: APP_RUNNING_ON_NODE 2016-08-11 02:33:44,430 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user1 in queue a userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:33:44,431 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1282)) - Trying to schedule on node: 127.0.0.1, available: 2016-08-11 02:33:44,431 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(437)) - Trying to assign containers to child-queue of root 2016-08-11 02:33:44,432 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.15625238), label=(*)root.busedCapacity=(4.0634923), label=(*) 2016-08-11 02:33:44,432 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 2016-08-11 02:33:44,433 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=127.0.0.1 #applications=1 2016-08-11 02:33:44,433 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user1 in queue a userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:33:44,433 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimitAndSetHeadroom(1120)) - Headroom calculation for user user1: userLimit= queueMaxAvailRes= consumed= headroom= 2016-08-11 02:33:44,434 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:assignContainers(511)) - pre-assignContainers for application application_100_0001 2016-08-11 02:33:44,434 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user1 in queue a userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:33:44,434 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0001 headRoom= currentConsumption=2048 2016-08-11 02:33:44,435 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0001 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:33:44,435 DEBUG [main] allocator.IncreaseContainerAllocator (IncreaseContainerAllocator.java:assignContainers(187)) - Skip allocating increase request since we don't have any increase request on this node=127.0.0.1:123 2016-08-11 02:33:44,435 DEBUG [main] allocator.RegularContainerAllocator (RegularContainerAllocator.java:preCheckForNewContainer(170)) - cannot allocate required resource= because of headroom 2016-08-11 02:33:44,436 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(962)) - post-assignContainers for application application_100_0001 2016-08-11 02:33:44,436 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user1 in queue a userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:33:44,437 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0001 headRoom= currentConsumption=2048 2016-08-11 02:33:44,437 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0001 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:33:44,437 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 --> , NODE_LOCAL 2016-08-11 02:33:44,438 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(688)) - Decrease parentLimits for root by as childQueue=a is blocked 2016-08-11 02:33:44,438 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=4.0634923, absoluteUsedCapacity=0.8125, numApps=1, numContainers=13 2016-08-11 02:33:44,438 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=127.0.0.1 #applications=1 2016-08-11 02:33:44,439 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(909)) - Skip this queue=root.b, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 02:33:44,439 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=4.0634923, absoluteUsedCapacity=0.8125, numApps=1, numContainers=13 --> , NODE_LOCAL 2016-08-11 02:33:44,439 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1282)) - Trying to schedule on node: 127.0.0.1, available: 2016-08-11 02:33:44,440 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(437)) - Trying to assign containers to child-queue of root 2016-08-11 02:33:44,440 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.15625238), label=(*)root.busedCapacity=(4.0634923), label=(*) 2016-08-11 02:33:44,440 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 2016-08-11 02:33:44,441 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=127.0.0.1 #applications=1 2016-08-11 02:33:44,441 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user1 in queue a userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:33:44,441 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimitAndSetHeadroom(1120)) - Headroom calculation for user user1: userLimit= queueMaxAvailRes= consumed= headroom= 2016-08-11 02:33:44,442 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:assignContainers(511)) - pre-assignContainers for application application_100_0001 2016-08-11 02:33:44,442 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user1 in queue a userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:33:44,442 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0001 headRoom= currentConsumption=2048 2016-08-11 02:33:44,443 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0001 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:33:44,443 DEBUG [main] allocator.IncreaseContainerAllocator (IncreaseContainerAllocator.java:assignContainers(187)) - Skip allocating increase request since we don't have any increase request on this node=127.0.0.1:123 2016-08-11 02:33:44,443 DEBUG [main] allocator.RegularContainerAllocator (RegularContainerAllocator.java:preCheckForNewContainer(170)) - cannot allocate required resource= because of headroom 2016-08-11 02:33:44,444 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(962)) - post-assignContainers for application application_100_0001 2016-08-11 02:33:44,444 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user1 in queue a userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:33:44,445 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0001 headRoom= currentConsumption=2048 2016-08-11 02:33:44,445 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0001 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:33:44,445 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 --> , NODE_LOCAL 2016-08-11 02:33:44,445 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(688)) - Decrease parentLimits for root by as childQueue=a is blocked 2016-08-11 02:33:44,446 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=4.0634923, absoluteUsedCapacity=0.8125, numApps=1, numContainers=13 2016-08-11 02:33:44,446 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=127.0.0.1 #applications=1 2016-08-11 02:33:44,446 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(909)) - Skip this queue=root.b, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-11 02:33:44,447 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=4.0634923, absoluteUsedCapacity=0.8125, numApps=1, numContainers=13 --> , NODE_LOCAL 2016-08-11 02:33:44,452 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(976)) - allocate: pre-update appattempt_100_0002_000002 ask size =1 2016-08-11 02:33:44,452 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:33:44,453 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=13312 2016-08-11 02:33:44,453 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 0, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:33:44,453 DEBUG [main] scheduler.ActiveUsersManager (ActiveUsersManager.java:activateApplication(68)) - User user2 added to activeUsers, currently: 1 2016-08-11 02:33:44,454 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocate(988)) - allocate: post-update 2016-08-11 02:33:44,454 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:33:44,455 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0002 headRoom= currentConsumption=13312 2016-08-11 02:33:44,455 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0002 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:33:44,456 DEBUG [main] security.BaseContainerTokenSecretManager (BaseContainerTokenSecretManager.java:createPassword(90)) - Creating password for container_100_0002_02_000013 for user container_100_0002_02_000013 (auth:SIMPLE) to be run on NM 127.0.0.1:123 2016-08-11 02:33:44,457 DEBUG [main] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 2 cluster_timestamp: 100 } attemptId: 2 } id: 13 } nmHostAddr: "127.0.0.1:123" appSubmitter: "user2" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1470863624455 masterKeyId: 100511640 rmIdentifier: 1470862883498 priority { priority: 1 } creationTime: 1470863004395 nodeLabelExpression: "" containerType: TASK executionType: GUARANTEED 2016-08-11 02:33:44,458 DEBUG [main] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 2 cluster_timestamp: 100 } attemptId: 2 } id: 13 } nmHostAddr: "127.0.0.1:123" appSubmitter: "user2" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1470863624455 masterKeyId: 100511640 rmIdentifier: 1470862883498 priority { priority: 1 } creationTime: 1470863004395 nodeLabelExpression: "" containerType: TASK executionType: GUARANTEED 2016-08-11 02:33:44,459 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0002_02_000013 of type ACQUIRED 2016-08-11 02:33:44,460 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0002_02_000013 Container Transitioned from ALLOCATED to ACQUIRED 2016-08-11 02:33:44,460 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppRunningOnNodeEvent.EventType: APP_RUNNING_ON_NODE 2016-08-11 02:33:44,461 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:33:44,463 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1282)) - Trying to schedule on node: 127.0.0.1, available: 2016-08-11 02:33:44,463 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(437)) - Trying to assign containers to child-queue of root 2016-08-11 02:33:44,464 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.15625238), label=(*)root.busedCapacity=(4.0634923), label=(*) 2016-08-11 02:33:44,464 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 2016-08-11 02:33:44,465 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=127.0.0.1 #applications=1 2016-08-11 02:33:44,465 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user1 in queue a userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:33:44,466 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimitAndSetHeadroom(1120)) - Headroom calculation for user user1: userLimit= queueMaxAvailRes= consumed= headroom= 2016-08-11 02:33:44,466 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:assignContainers(511)) - pre-assignContainers for application application_100_0001 2016-08-11 02:33:44,466 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user1 in queue a userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:33:44,467 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0001 headRoom= currentConsumption=2048 2016-08-11 02:33:44,467 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0001 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:33:44,468 DEBUG [main] allocator.IncreaseContainerAllocator (IncreaseContainerAllocator.java:assignContainers(187)) - Skip allocating increase request since we don't have any increase request on this node=127.0.0.1:123 2016-08-11 02:33:44,468 DEBUG [main] allocator.RegularContainerAllocator (RegularContainerAllocator.java:preCheckForNewContainer(170)) - cannot allocate required resource= because of headroom 2016-08-11 02:33:44,469 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(962)) - post-assignContainers for application application_100_0001 2016-08-11 02:33:44,469 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user1 in queue a userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:33:44,470 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0001 headRoom= currentConsumption=2048 2016-08-11 02:33:44,470 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0001 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:33:44,470 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 --> , NODE_LOCAL 2016-08-11 02:33:44,471 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(688)) - Decrease parentLimits for root by as childQueue=a is blocked 2016-08-11 02:33:44,471 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=4.0634923, absoluteUsedCapacity=0.8125, numApps=1, numContainers=13 2016-08-11 02:33:44,471 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=127.0.0.1 #applications=1 2016-08-11 02:33:44,472 DEBUG [main] capacity.AbstractCSQueue (AbstractCSQueue.java:canAssignToThisQueue(530)) - bCheck assign to queue, nodePartition= usedResources: clusterResources: currentUsedCapacity 0.8125 max-capacity: 1.0) 2016-08-11 02:33:44,473 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.b stats: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=4.0634923, absoluteUsedCapacity=0.8125, numApps=1, numContainers=13 --> , NODE_LOCAL 2016-08-11 02:33:44,491 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0002_02_000010 of type EXPIRE 2016-08-11 02:33:44,494 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0002_02_000010 Container Transitioned from ACQUIRED to EXPIRED 2016-08-11 02:33:44,495 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeCleanContainerEvent.EventType: CLEANUP_CONTAINER 2016-08-11 02:33:44,495 INFO [main] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(156)) - USER=user2 OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_100_0002 CONTAINERID=container_100_0002_02_000010 RESOURCE= 2016-08-11 02:33:44,495 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptContainerFinishedEvent.EventType: CONTAINER_FINISHED 2016-08-11 02:33:44,496 DEBUG [main] scheduler.SchedulerNode (SchedulerNode.java:releaseContainer(269)) - Released container container_100_0002_02_000010 of capacity on host 127.0.0.1:1, which currently has 6 containers, used and available, release resources=true 2016-08-11 02:33:44,496 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:33:44,497 DEBUG [main] capacity.LeafQueue (LeafQueue.java:releaseResource(1479)) - b used= numContainers=12 user=user2 user-resources= 2016-08-11 02:33:44,497 DEBUG [main] capacity.ParentQueue (ParentQueue.java:internalReleaseResource(728)) - completedContainer root: numChildQueue= 2, capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.875, numApps=2, numContainers=13, cluster= 2016-08-11 02:33:44,498 DEBUG [main] capacity.ParentQueue (ParentQueue.java:internalReleaseResource(741)) - Re-sorting completed queue: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=3.7509158, absoluteUsedCapacity=0.75, numApps=1, numContainers=12 2016-08-11 02:33:44,499 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0002_02_000011 of type EXPIRE 2016-08-11 02:33:44,499 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeCleanContainerEvent.EventType: CLEANUP_CONTAINER 2016-08-11 02:33:44,500 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptContainerFinishedEvent.EventType: CONTAINER_FINISHED 2016-08-11 02:33:44,500 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0002_02_000011 Container Transitioned from ACQUIRED to EXPIRED 2016-08-11 02:33:44,500 INFO [main] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(156)) - USER=user2 OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_100_0002 CONTAINERID=container_100_0002_02_000011 RESOURCE= 2016-08-11 02:33:44,501 DEBUG [main] scheduler.SchedulerNode (SchedulerNode.java:releaseContainer(269)) - Released container container_100_0002_02_000011 of capacity on host 127.0.0.1:1, which currently has 5 containers, used and available, release resources=true 2016-08-11 02:33:44,501 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user2 in queue b userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:33:44,502 DEBUG [main] capacity.LeafQueue (LeafQueue.java:releaseResource(1479)) - b used= numContainers=11 user=user2 user-resources= 2016-08-11 02:33:44,502 DEBUG [main] capacity.ParentQueue (ParentQueue.java:internalReleaseResource(728)) - completedContainer root: numChildQueue= 2, capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.8125, numApps=2, numContainers=12, cluster= 2016-08-11 02:33:44,502 DEBUG [main] capacity.ParentQueue (ParentQueue.java:internalReleaseResource(741)) - Re-sorting completed queue: b: capacity=0.2, absoluteCapacity=0.2, usedResources=, usedCapacity=3.4383395, absoluteUsedCapacity=0.6875, numApps=1, numContainers=11 2016-08-11 02:33:44,503 DEBUG [main] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1282)) - Trying to schedule on node: 127.0.0.1, available: 2016-08-11 02:33:44,503 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(437)) - Trying to assign containers to child-queue of root 2016-08-11 02:33:44,503 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.15625238), label=(*)root.busedCapacity=(3.4383395), label=(*) 2016-08-11 02:33:44,503 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(644)) - Trying to assign to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.15625238, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 2016-08-11 02:33:44,504 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(868)) - assignContainers: node=127.0.0.1 #applications=1 2016-08-11 02:33:44,504 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user1 in queue a userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:33:44,505 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimitAndSetHeadroom(1120)) - Headroom calculation for user user1: userLimit= queueMaxAvailRes= consumed= headroom= 2016-08-11 02:33:44,505 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:assignContainers(511)) - pre-assignContainers for application application_100_0001 2016-08-11 02:33:44,505 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user1 in queue a userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-08-11 02:33:44,505 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0001 headRoom= currentConsumption=2048 2016-08-11 02:33:44,506 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0001 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:33:44,506 DEBUG [main] allocator.IncreaseContainerAllocator (IncreaseContainerAllocator.java:assignContainers(187)) - Skip allocating increase request since we don't have any increase request on this node=127.0.0.1:123 2016-08-11 02:33:44,506 DEBUG [main] allocator.RegularContainerAllocator (RegularContainerAllocator.java:assignContainer(473)) - assignContainers: node=127.0.0.1 application=application_100_0001 priority=1 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } type=OFF_SWITCH 2016-08-11 02:33:44,507 DEBUG [main] scheduler.ActiveUsersManager (ActiveUsersManager.java:deactivateApplication(94)) - User user1 removed from activeUsers, currently: 0 2016-08-11 02:33:44,507 DEBUG [main] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:allocate(631)) - allocate: applicationId=application_100_0001 container=container_100_0001_01_000002 host=127.0.0.1:123 user=user1 resource= type=OFF_SWITCH 2016-08-11 02:33:44,508 DEBUG [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(468)) - Processing container_100_0001_01_000002 of type START 2016-08-11 02:33:44,508 INFO [main] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(482)) - container_100_0001_01_000002 Container Transitioned from NEW to ALLOCATED 2016-08-11 02:33:44,508 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(178)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: CONTAINER_ALLOCATED 2016-08-11 02:33:44,508 DEBUG [main] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:allocate(237)) - allocate: applicationAttemptId=appattempt_100_0001_000001 container=container_100_0001_01_000002 host=127.0.0.1 type=OFF_SWITCH 2016-08-11 02:33:44,508 INFO [main] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(156)) - USER=user1 OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_100_0001 CONTAINERID=container_100_0001_01_000002 RESOURCE= 2016-08-11 02:33:44,509 DEBUG [main] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(157)) - Assigned container container_100_0001_01_000002 of capacity on host 127.0.0.1:1, which has 6 containers, used and available after allocation 2016-08-11 02:33:44,509 INFO [main] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(120)) - assignedContainer application attempt=appattempt_100_0001_000001 container=container_100_0001_01_000002 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@2e1ef60 clusterResource= type=OFF_SWITCH 2016-08-11 02:33:44,509 DEBUG [main] capacity.LeafQueue (LeafQueue.java:assignContainers(962)) - post-assignContainers for application application_100_0001 2016-08-11 02:33:44,510 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user1 in queue a userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:33:44,510 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(513)) - showRequests: application=application_100_0001 headRoom= currentConsumption=4096 2016-08-11 02:33:44,510 DEBUG [main] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(517)) - showRequests: application=application_100_0001 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 0, Location: *, Relax Locality: true, Execution Type Request: null, Node Label Expression: } 2016-08-11 02:33:44,511 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user1 in queue a userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:33:44,511 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user1 in queue a userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:33:44,511 DEBUG [main] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1231)) - User limit computation for user1 in queue a userLimitPercent=100 userLimitFactor=100.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-08-11 02:33:44,511 DEBUG [main] capacity.LeafQueue (LeafQueue.java:allocateResource(1443)) - a user=user1 used= numContainers=2 headroom = user-resources= 2016-08-11 02:33:44,512 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(656)) - Assigned to queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.31250477, absoluteUsedCapacity=0.25, numApps=1, numContainers=2 --> , OFF_SWITCH 2016-08-11 02:33:44,512 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(670)) - Re-sorting assigned queue: root.a stats: a: capacity=0.8, absoluteCapacity=0.8, usedResources=, usedCapacity=0.31250477, absoluteUsedCapacity=0.25, numApps=1, numContainers=2 2016-08-11 02:33:44,512 DEBUG [main] capacity.ParentQueue (ParentQueue.java:printChildQueues(715)) - printChildQueues - queue: root child-queues: root.ausedCapacity=(0.31250477), label=(*)root.busedCapacity=(3.4383395), label=(*) 2016-08-11 02:33:44,512 INFO [main] capacity.ParentQueue (ParentQueue.java:assignContainers(522)) - assignedContainer queue=root usedCapacity=0.9375 absoluteUsedCapacity=0.9375 used= cluster= 2016-08-11 02:33:44,513 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(544)) - ParentQ=root assignedSoFarInThisIteration= usedCapacity=0.9375 absoluteUsedCapacity=0.9375 2016-08-11 02:33:44,513 DEBUG [main] capacity.ParentQueue (ParentQueue.java:assignContainers(555)) - Not assigning more than one off-switch container, assignments so far: resource:; type:OFF_SWITCH; excessReservation:null; applicationid:null; skipped:NONE; fulfilled reservation:false; allocations(count/resource):1/; reservations(count/resource):0/ 2016-08-11 02:33:44,519 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: ResourceManager entered state STOPPED 2016-08-11 02:33:44,519 DEBUG [main] service.CompositeService (CompositeService.java:serviceStop(129)) - ResourceManager: stopping services, size=3 2016-08-11 02:33:44,519 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #2: Service org.apache.hadoop.yarn.server.resourcemanager.ahs.RMApplicationHistoryWriter in state org.apache.hadoop.yarn.server.resourcemanager.ahs.RMApplicationHistoryWriter: INITED 2016-08-11 02:33:44,520 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.ahs.RMApplicationHistoryWriter entered state STOPPED 2016-08-11 02:33:44,520 DEBUG [main] service.CompositeService (CompositeService.java:serviceStop(129)) - org.apache.hadoop.yarn.server.resourcemanager.ahs.RMApplicationHistoryWriter: stopping services, size=0 2016-08-11 02:33:44,520 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #1: Service org.apache.hadoop.yarn.server.resourcemanager.AdminService in state org.apache.hadoop.yarn.server.resourcemanager.AdminService: INITED 2016-08-11 02:33:44,520 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.AdminService entered state STOPPED 2016-08-11 02:33:44,521 DEBUG [main] service.CompositeService (CompositeService.java:serviceStop(129)) - org.apache.hadoop.yarn.server.resourcemanager.AdminService: stopping services, size=0 2016-08-11 02:33:44,521 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #0: Service Dispatcher in state Dispatcher: STARTED 2016-08-11 02:33:44,521 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: Dispatcher entered state STOPPED 2016-08-11 02:33:44,521 INFO [main] resourcemanager.ResourceManager (ResourceManager.java:transitionToStandby(1108)) - Transitioning to standby state 2016-08-11 02:33:44,521 INFO [main] resourcemanager.ResourceManager (ResourceManager.java:transitionToStandby(1115)) - Transitioned to standby state