scheme is : hdfs 2017-01-30 10:24:00,235 INFO [main] security.TestDelegationTokenRenewer (TestDelegationTokenRenewer.java:setUpClass(194)) - filesystem uri = hdfs://localhost:0 2017-01-30 10:24:00,650 INFO [main] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:(94)) - AMRMTokenKeyRollingInterval: 86400000ms and AMRMTokenKeyActivationDelay: 900000 ms 2017-01-30 10:24:00,653 INFO [main] security.RMContainerTokenSecretManager (RMContainerTokenSecretManager.java:(79)) - ContainerTokenKeyRollingInterval: 86400000ms and ContainerTokenKeyActivationDelay: 900000ms 2017-01-30 10:24:00,655 INFO [main] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:(75)) - NMTokenKeyRollingInterval: 86400000ms and NMTokenKeyActivationDelay: 900000ms 2017-01-30 10:24:00,690 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(227)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStoreEventType for class org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$ForwardingEventHandler 2017-01-30 10:24:00,836 INFO [Thread-6] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2460)) - found resource core-site.xml at file:/opt/apacheprojects/hadoop/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/target/test-classes/core-site.xml 2017-01-30 10:24:00,854 INFO [Thread-6] security.Groups (Groups.java:refresh(402)) - clearing userToGroupsMap cache 2017-01-30 10:24:00,890 INFO [Thread-6] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2460)) - found resource yarn-site.xml at file:/opt/apacheprojects/hadoop/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/target/test-classes/yarn-site.xml 2017-01-30 10:24:00,912 INFO [Thread-6] event.AsyncDispatcher (AsyncDispatcher.java:register(227)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.RMFatalEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMFatalEventDispatcher 2017-01-30 10:24:00,915 INFO [Thread-6] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:(75)) - NMTokenKeyRollingInterval: 86400000ms and NMTokenKeyActivationDelay: 900000ms 2017-01-30 10:24:00,916 INFO [Thread-6] security.RMContainerTokenSecretManager (RMContainerTokenSecretManager.java:(79)) - ContainerTokenKeyRollingInterval: 86400000ms and ContainerTokenKeyActivationDelay: 900000ms 2017-01-30 10:24:00,916 INFO [Thread-6] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:(94)) - AMRMTokenKeyRollingInterval: 86400000ms and AMRMTokenKeyActivationDelay: 900000 ms 2017-01-30 10:24:00,924 INFO [Thread-6] security.YarnAuthorizationProvider (YarnAuthorizationProvider.java:getInstance(58)) - org.apache.hadoop.yarn.security.ConfiguredYarnAuthorizer is instantiated. 2017-01-30 10:24:00,924 INFO [Thread-6] event.AsyncDispatcher (AsyncDispatcher.java:register(227)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStoreEventType for class org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$ForwardingEventHandler 2017-01-30 10:24:00,926 INFO [Thread-6] event.AsyncDispatcher (AsyncDispatcher.java:register(227)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.NodesListManagerEventType for class org.apache.hadoop.yarn.server.resourcemanager.NodesListManager 2017-01-30 10:24:00,926 INFO [Thread-6] resourcemanager.ResourceManager (ResourceManager.java:createScheduler(431)) - Using Scheduler: org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler 2017-01-30 10:24:00,936 INFO [Thread-6] event.AsyncDispatcher (AsyncDispatcher.java:register(227)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.SchedulerEventType for class org.apache.hadoop.yarn.server.resourcemanager.MockRM$1 2017-01-30 10:24:00,936 INFO [Thread-6] event.AsyncDispatcher (AsyncDispatcher.java:register(227)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationEventDispatcher 2017-01-30 10:24:00,937 INFO [Thread-6] event.AsyncDispatcher (AsyncDispatcher.java:register(227)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationAttemptEventDispatcher 2017-01-30 10:24:00,937 INFO [Thread-6] event.AsyncDispatcher (AsyncDispatcher.java:register(227)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$NodeEventDispatcher 2017-01-30 10:24:00,938 INFO [Thread-6] security.RMContainerTokenSecretManager (RMContainerTokenSecretManager.java:rollMasterKey(107)) - Rolling master-key for container-tokens 2017-01-30 10:24:00,943 INFO [Thread-6] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:rollMasterKey(95)) - Rolling master-key for nm-tokens 2017-01-30 10:24:01,048 INFO [Thread-6] beanutils.FluentPropertyBeanIntrospector (FluentPropertyBeanIntrospector.java:introspect(147)) - Error when creating PropertyDescriptor for public final void org.apache.commons.configuration2.AbstractConfiguration.setProperty(java.lang.String,java.lang.Object)! Ignoring this property. 2017-01-30 10:24:01,064 WARN [Thread-6] impl.MetricsConfig (MetricsConfig.java:loadFirst(134)) - Cannot locate configuration: tried hadoop-metrics2-resourcemanager.properties,hadoop-metrics2.properties 2017-01-30 10:24:01,124 INFO [Thread-6] impl.MetricsSystemImpl (MetricsSystemImpl.java:startTimer(374)) - Scheduled Metric snapshot period at 10 second(s). 2017-01-30 10:24:01,124 INFO [Thread-6] impl.MetricsSystemImpl (MetricsSystemImpl.java:start(191)) - ResourceManager metrics system started 2017-01-30 10:24:01,137 INFO [Thread-6] event.AsyncDispatcher (AsyncDispatcher.java:register(227)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.RMAppManagerEventType for class org.apache.hadoop.yarn.server.resourcemanager.RMAppManager 2017-01-30 10:24:01,139 INFO [Thread-6] event.AsyncDispatcher (AsyncDispatcher.java:register(227)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncherEventType for class org.apache.hadoop.yarn.server.resourcemanager.MockRM$7 2017-01-30 10:24:01,141 INFO [Thread-6] resourcemanager.RMNMInfo (RMNMInfo.java:(63)) - Registered RMNMInfo MBean 2017-01-30 10:24:01,141 INFO [Thread-6] monitor.RMAppLifetimeMonitor (RMAppLifetimeMonitor.java:serviceInit(66)) - Application lifelime monitor interval set to 3000 ms. 2017-01-30 10:24:01,142 INFO [Thread-6] util.HostsFileReader (HostsFileReader.java:refresh(240)) - Refreshing hosts (include/exclude) list 2017-01-30 10:24:01,146 INFO [Thread-6] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2460)) - found resource capacity-scheduler.xml at file:/opt/apacheprojects/hadoop/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/target/test-classes/capacity-scheduler.xml 2017-01-30 10:24:01,197 INFO [Thread-6] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(790)) - max alloc mb per queue for root is undefined 2017-01-30 10:24:01,197 INFO [Thread-6] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(794)) - max alloc vcore per queue for root is undefined 2017-01-30 10:24:01,207 INFO [Thread-6] capacity.ParentQueue (ParentQueue.java:setupQueueConfigs(145)) - root, capacity=1.0, absoluteCapacity=1.0, maxCapacity=1.0, absoluteMaxCapacity=1.0, state=RUNNING, acls=SUBMIT_APP:*ADMINISTER_QUEUE:*, labels=*, , reservationsContinueLooking=true, orderingPolicy=utilization, priority=0 2017-01-30 10:24:01,207 INFO [Thread-6] capacity.ParentQueue (ParentQueue.java:(108)) - Initialized parent-queue root name=root, fullname=root 2017-01-30 10:24:01,219 INFO [Thread-6] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(790)) - max alloc mb per queue for root.default is undefined 2017-01-30 10:24:01,219 INFO [Thread-6] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(794)) - max alloc vcore per queue for root.default is undefined 2017-01-30 10:24:01,221 INFO [Thread-6] capacity.LeafQueue (LeafQueue.java:setupQueueConfigs(254)) - Initializing default capacity = 1.0 [= (float) configuredCapacity / 100 ] absoluteCapacity = 1.0 [= parentAbsoluteCapacity * capacity ] maxCapacity = 1.0 [= configuredMaxCapacity ] absoluteMaxCapacity = 1.0 [= 1.0 maximumCapacity undefined, (parentAbsoluteMaxCapacity * maximumCapacity) / 100 otherwise ] userLimit = 100 [= configuredUserLimit ] userLimitFactor = 1.0 [= configuredUserLimitFactor ] maxApplications = 10000 [= configuredMaximumSystemApplicationsPerQueue or (int)(configuredMaximumSystemApplications * absoluteCapacity)] maxApplicationsPerUser = 10000 [= (int)(maxApplications * (userLimit / 100.0f) * userLimitFactor) ] usedCapacity = 0.0 [= usedResourcesMemory / (clusterResourceMemory * absoluteCapacity)] absoluteUsedCapacity = 0.0 [= usedResourcesMemory / clusterResourceMemory] maxAMResourcePerQueuePercent = 0.1 [= configuredMaximumAMResourcePercent ] minimumAllocationFactor = 0.875 [= (float)(maximumAllocationMemory - minimumAllocationMemory) / maximumAllocationMemory ] maximumAllocation = [= configuredMaxAllocation ] numContainers = 0 [= currentNumContainers ] state = RUNNING [= configuredState ] acls = SUBMIT_APP:*ADMINISTER_QUEUE:* [= configuredAcls ] nodeLocalityDelay = 40 labels=*, reservationsContinueLooking = true preemptionDisabled = true defaultAppPriorityPerQueue = 0 priority = 0 2017-01-30 10:24:01,221 INFO [Thread-6] capacity.CapacitySchedulerQueueManager (CapacitySchedulerQueueManager.java:parseQueue(259)) - Initialized queue: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=0, numContainers=0 2017-01-30 10:24:01,222 INFO [Thread-6] capacity.CapacitySchedulerQueueManager (CapacitySchedulerQueueManager.java:parseQueue(259)) - Initialized queue: root: numChildQueue= 1, capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.0, numApps=0, numContainers=0 2017-01-30 10:24:01,222 INFO [Thread-6] capacity.CapacitySchedulerQueueManager (CapacitySchedulerQueueManager.java:initializeQueues(156)) - Initialized root queue root: numChildQueue= 1, capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.0, numApps=0, numContainers=0 2017-01-30 10:24:01,222 INFO [Thread-6] capacity.CapacityScheduler (CapacityScheduler.java:getUserGroupMappingPlacementRule(539)) - Initialized queue mappings, override: false 2017-01-30 10:24:01,223 INFO [Thread-6] capacity.CapacityScheduler (CapacityScheduler.java:initScheduler(332)) - Initialized CapacityScheduler with calculator=class org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator, minimumAllocation=<>, maximumAllocation=<>, asynchronousScheduling=false, asyncScheduleInterval=5ms 2017-01-30 10:24:01,228 INFO [Thread-6] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2457)) - dynamic-resources.xml not found 2017-01-30 10:24:01,233 INFO [Thread-6] resourcemanager.ResourceManager (ResourceManager.java:createSystemMetricsPublisher(521)) - TimelineServicePublisher is not configured 2017-01-30 10:24:01,234 INFO [Thread-6] event.AsyncDispatcher (AsyncDispatcher.java:register(227)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStoreEventType for class org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$ForwardingEventHandler 2017-01-30 10:24:01,234 INFO [Thread-6] resourcemanager.ResourceManager (ResourceManager.java:transitionToActive(1173)) - Transitioning to active state 2017-01-30 10:24:01,235 DEBUG [Thread-6] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1919)) - PrivilegedAction as:root (auth:KERBEROS) from:org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.transitionToActive(ResourceManager.java:1175) 2017-01-30 10:24:01,237 DEBUG [Thread-6] service.AbstractService (AbstractService.java:start(197)) - Service Dispatcher is started 2017-01-30 10:24:01,237 DEBUG [Thread-6] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore is started 2017-01-30 10:24:01,237 DEBUG [Thread-6] service.CompositeService (CompositeService.java:serviceStart(115)) - RMActiveServices: starting services, size=15 2017-01-30 10:24:01,241 DEBUG [Thread-6] recovery.RMStateStore (RMStateStore.java:handleStoreEvent(1094)) - Processing event of type UPDATE_AMRM_TOKEN 2017-01-30 10:24:01,241 INFO [Thread-6] recovery.RMStateStore (RMStateStore.java:transition(502)) - Updating AMRMToken 2017-01-30 10:24:01,241 INFO [Thread-6] security.RMContainerTokenSecretManager (RMContainerTokenSecretManager.java:rollMasterKey(107)) - Rolling master-key for container-tokens 2017-01-30 10:24:01,242 INFO [Thread-6] security.RMContainerTokenSecretManager (RMContainerTokenSecretManager.java:rollMasterKey(112)) - Going to activate master-key with key-id 313252935 in 900000ms 2017-01-30 10:24:01,242 INFO [Thread-6] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:rollMasterKey(95)) - Rolling master-key for nm-tokens 2017-01-30 10:24:01,242 INFO [Thread-6] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:rollMasterKey(100)) - Going to activate master-key with key-id -671765033 in 900000ms 2017-01-30 10:24:01,242 INFO [Thread-6] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(342)) - Updating the current master key for generating delegation tokens 2017-01-30 10:24:01,243 INFO [Thread-6] security.RMDelegationTokenSecretManager (RMDelegationTokenSecretManager.java:storeNewMasterKey(87)) - storing master key with keyID 1 2017-01-30 10:24:01,243 DEBUG [Thread-6] recovery.RMStateStore (RMStateStore.java:handleStoreEvent(1094)) - Processing event of type STORE_MASTERKEY 2017-01-30 10:24:01,244 INFO [Thread-6] recovery.RMStateStore (RMStateStore.java:transition(453)) - Storing RMDTMasterKey. 2017-01-30 10:24:01,245 DEBUG [Thread-6] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.RMSecretManagerService is started 2017-01-30 10:24:01,245 INFO [Thread[Thread-17,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(668)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2017-01-30 10:24:01,246 INFO [Thread[Thread-17,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(342)) - Updating the current master key for generating delegation tokens 2017-01-30 10:24:01,246 INFO [Thread[Thread-17,5,main]] security.RMDelegationTokenSecretManager (RMDelegationTokenSecretManager.java:storeNewMasterKey(87)) - storing master key with keyID 2 2017-01-30 10:24:01,246 DEBUG [Thread[Thread-17,5,main]] recovery.RMStateStore (RMStateStore.java:handleStoreEvent(1094)) - Processing event of type STORE_MASTERKEY 2017-01-30 10:24:01,246 INFO [Thread[Thread-17,5,main]] recovery.RMStateStore (RMStateStore.java:transition(453)) - Storing RMDTMasterKey. 2017-01-30 10:24:01,246 DEBUG [Thread-6] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.ContainerAllocationExpirer is started 2017-01-30 10:24:01,247 DEBUG [Thread-6] service.AbstractService (AbstractService.java:start(197)) - Service AMLivelinessMonitor is started 2017-01-30 10:24:01,247 DEBUG [Thread-6] service.AbstractService (AbstractService.java:start(197)) - Service AMLivelinessMonitor is started 2017-01-30 10:24:01,247 DEBUG [Thread-6] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.rmapp.monitor.RMAppLifetimeMonitor is started 2017-01-30 10:24:01,247 DEBUG [Thread-6] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.nodelabels.CommonNodeLabelsManager is started 2017-01-30 10:24:01,248 DEBUG [Thread-6] service.CompositeService (CompositeService.java:serviceStart(115)) - org.apache.hadoop.yarn.server.resourcemanager.NodesListManager: starting services, size=0 2017-01-30 10:24:01,248 DEBUG [Thread-6] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.NodesListManager is started 2017-01-30 10:24:01,248 DEBUG [Thread-6] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.scheduler.activities.ActivitiesManager is started 2017-01-30 10:24:01,249 DEBUG [Thread-6] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler is started 2017-01-30 10:24:01,249 DEBUG [Thread-6] service.AbstractService (AbstractService.java:start(197)) - Service NMLivelinessMonitor is started 2017-01-30 10:24:01,249 DEBUG [Thread-6] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.ResourceTrackerService is started 2017-01-30 10:24:01,250 DEBUG [Thread-6] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.util.JvmPauseMonitor is started 2017-01-30 10:24:01,250 DEBUG [Thread-6] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService is started 2017-01-30 10:24:01,250 DEBUG [Thread-6] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.ClientRMService is started 2017-01-30 10:24:01,250 DEBUG [Thread-6] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher is started 2017-01-30 10:24:01,250 INFO [org.apache.hadoop.util.JvmPauseMonitor$Monitor@687fcf5] util.JvmPauseMonitor (JvmPauseMonitor.java:run(188)) - Starting JVM pause monitor 2017-01-30 10:24:01,251 DEBUG [Thread-6] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.security.DelegationTokenRenewer is started 2017-01-30 10:24:01,251 DEBUG [Thread-6] service.AbstractService (AbstractService.java:start(197)) - Service RMActiveServices is started 2017-01-30 10:24:01,251 INFO [Thread-6] resourcemanager.ResourceManager (ResourceManager.java:transitionToActive(1189)) - Transitioned to active state 2017-01-30 10:24:01,251 DEBUG [Thread-6] service.CompositeService (CompositeService.java:serviceStart(115)) - ResourceManager: starting services, size=3 2017-01-30 10:24:01,252 DEBUG [Thread-6] service.AbstractService (AbstractService.java:start(197)) - Service Dispatcher is started 2017-01-30 10:24:01,252 DEBUG [Thread-6] service.CompositeService (CompositeService.java:serviceStart(115)) - org.apache.hadoop.yarn.server.resourcemanager.AdminService: starting services, size=0 2017-01-30 10:24:01,252 DEBUG [Thread-6] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.AdminService is started 2017-01-30 10:24:01,252 DEBUG [Thread-6] service.CompositeService (CompositeService.java:serviceStart(115)) - org.apache.hadoop.yarn.server.resourcemanager.ahs.RMApplicationHistoryWriter: starting services, size=0 2017-01-30 10:24:01,255 DEBUG [Thread-6] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.ahs.RMApplicationHistoryWriter is started 2017-01-30 10:24:01,256 DEBUG [Thread-6] service.AbstractService (AbstractService.java:start(197)) - Service ResourceManager is started 2017-01-30 10:24:01,307 DEBUG [Thread-6] util.RackResolver (RackResolver.java:coreResolve(112)) - Resolved 127.0.0.1 to /default-rack 2017-01-30 10:24:01,325 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStartedEvent.EventType: STARTED 2017-01-30 10:24:01,325 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(657)) - Processing 127.0.0.1:1234 of type STARTED 2017-01-30 10:24:01,326 INFO [Thread-6] resourcemanager.ResourceTrackerService (ResourceTrackerService.java:registerNodeManager(454)) - NodeManager from node 127.0.0.1(cmPort: 1234 httpPort: 2) registered with capability: , assigned nodeId 127.0.0.1:1234 2017-01-30 10:24:01,327 DEBUG [AsyncDispatcher event handler] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.resourcemanager.ClusterMetrics.numActiveNMs with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, about=, always=false, type=DEFAULT, valueName=Time, value=[# of active NMs]) 2017-01-30 10:24:01,327 DEBUG [AsyncDispatcher event handler] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.resourcemanager.ClusterMetrics.numDecommissioningNMs with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, about=, always=false, type=DEFAULT, valueName=Time, value=[# of decommissioning NMs]) 2017-01-30 10:24:01,328 DEBUG [AsyncDispatcher event handler] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.resourcemanager.ClusterMetrics.numDecommissionedNMs with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, about=, always=false, type=DEFAULT, valueName=Time, value=[# of decommissioned NMs]) 2017-01-30 10:24:01,328 DEBUG [AsyncDispatcher event handler] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.resourcemanager.ClusterMetrics.numLostNMs with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, about=, always=false, type=DEFAULT, valueName=Time, value=[# of lost NMs]) 2017-01-30 10:24:01,328 DEBUG [AsyncDispatcher event handler] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.resourcemanager.ClusterMetrics.numUnhealthyNMs with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, about=, always=false, type=DEFAULT, valueName=Time, value=[# of unhealthy NMs]) 2017-01-30 10:24:01,328 DEBUG [AsyncDispatcher event handler] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.resourcemanager.ClusterMetrics.numRebootedNMs with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, about=, always=false, type=DEFAULT, valueName=Time, value=[# of Rebooted NMs]) 2017-01-30 10:24:01,329 DEBUG [AsyncDispatcher event handler] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.resourcemanager.ClusterMetrics.numShutdownNMs with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, about=, always=false, type=DEFAULT, valueName=Time, value=[# of Shutdown NMs]) 2017-01-30 10:24:01,329 DEBUG [AsyncDispatcher event handler] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.yarn.server.resourcemanager.ClusterMetrics.aMLaunchDelay with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, about=, always=false, type=DEFAULT, valueName=Time, value=[AM container launch delay]) 2017-01-30 10:24:01,329 DEBUG [AsyncDispatcher event handler] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.yarn.server.resourcemanager.ClusterMetrics.aMRegisterDelay with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, about=, always=false, type=DEFAULT, valueName=Time, value=[AM register delay]) 2017-01-30 10:24:01,330 DEBUG [AsyncDispatcher event handler] impl.MetricsSystemImpl (MetricsSystemImpl.java:register(231)) - ClusterMetrics, Metrics for the Yarn Cluster 2017-01-30 10:24:01,330 DEBUG [AsyncDispatcher event handler] impl.MetricsConfig (MetricsConfig.java:getPropertyInternal(189)) - poking parent 'PropertiesConfiguration' for key: source.source.start_mbeans 2017-01-30 10:24:01,330 DEBUG [AsyncDispatcher event handler] impl.MetricsConfig (MetricsConfig.java:getPropertyInternal(189)) - poking parent 'MetricsConfig' for key: source.start_mbeans 2017-01-30 10:24:01,330 DEBUG [AsyncDispatcher event handler] impl.MetricsConfig (MetricsConfig.java:getPropertyInternal(189)) - poking parent 'PropertiesConfiguration' for key: *.source.start_mbeans 2017-01-30 10:24:01,330 DEBUG [AsyncDispatcher event handler] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateAttrCache(251)) - Updating attr cache... 2017-01-30 10:24:01,331 DEBUG [AsyncDispatcher event handler] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateAttrCache(265)) - Done. # tags & metrics=14 2017-01-30 10:24:01,331 DEBUG [AsyncDispatcher event handler] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateInfoCache(244)) - Updating info cache... 2017-01-30 10:24:01,331 DEBUG [AsyncDispatcher event handler] impl.MetricsSystemImpl (MBeanInfoBuilder.java:get(109)) - [javax.management.MBeanAttributeInfo[description=Metrics for the Yarn Cluster, name=tag.ClusterMetrics, type=java.lang.String, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Metrics context, name=tag.Context, type=java.lang.String, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Local hostname, name=tag.Hostname, type=java.lang.String, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=# of active NMs, name=NumActiveNMs, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=# of decommissioning NMs, name=NumDecommissioningNMs, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=# of decommissioned NMs, name=NumDecommissionedNMs, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=# of lost NMs, name=NumLostNMs, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=# of unhealthy NMs, name=NumUnhealthyNMs, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=# of Rebooted NMs, name=NumRebootedNMs, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=# of Shutdown NMs, name=NumShutdownNMs, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of ops for aM container launch delay, name=AMLaunchDelayNumOps, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Average time for aM container launch delay, name=AMLaunchDelayAvgTime, type=java.lang.Double, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of ops for aM register delay, name=AMRegisterDelayNumOps, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Average time for aM register delay, name=AMRegisterDelayAvgTime, type=java.lang.Double, read-only, descriptor={}]] 2017-01-30 10:24:01,370 DEBUG [AsyncDispatcher event handler] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateInfoCache(246)) - Done 2017-01-30 10:24:01,370 DEBUG [AsyncDispatcher event handler] util.MBeans (MBeans.java:register(68)) - Registered Hadoop:service=ResourceManager,name=ClusterMetrics 2017-01-30 10:24:01,370 DEBUG [AsyncDispatcher event handler] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:startMBeans(222)) - MBean for source ClusterMetrics registered. 2017-01-30 10:24:01,370 DEBUG [AsyncDispatcher event handler] impl.MetricsSystemImpl (MetricsSystemImpl.java:registerSource(269)) - Registered source ClusterMetrics 2017-01-30 10:24:01,372 INFO [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(669)) - 127.0.0.1:1234 Node Transitioned from NEW to RUNNING 2017-01-30 10:24:01,373 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeAddedSchedulerEvent.EventType: NODE_ADDED 2017-01-30 10:24:01,380 INFO [AsyncDispatcher event handler] capacity.CapacityScheduler (CapacityScheduler.java:addNode(1556)) - Added node 127.0.0.1:1234 clusterResource: 2017-01-30 10:24:01,381 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.NodesListManagerEvent.EventType: NODE_USABLE 2017-01-30 10:24:01,381 DEBUG [AsyncDispatcher event handler] resourcemanager.NodesListManager (NodesListManager.java:handle(488)) - 127.0.0.1:1234 reported usable 2017-01-30 10:24:01,382 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeLabelsUpdateSchedulerEvent.EventType: NODE_LABELS_UPDATE 2017-01-30 10:24:01,390 INFO [Thread-6] resourcemanager.ClientRMService (ClientRMService.java:getNewApplicationId(320)) - Allocated new applicationId: 1 2017-01-30 10:24:01,443 DEBUG [Thread-6] security.JniBasedUnixGroupsMappingWithFallback (JniBasedUnixGroupsMappingWithFallback.java:(45)) - Group mapping impl=org.apache.hadoop.security.JniBasedUnixGroupsMapping 2017-01-30 10:24:01,443 DEBUG [Thread-6] security.Groups (Groups.java:(151)) - Group mapping impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback; cacheTimeout=300000; warningDeltaMs=5000 2017-01-30 10:24:01,444 DEBUG [Thread-6] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1919)) - PrivilegedAction as:user (auth:SIMPLE) from:org.apache.hadoop.yarn.server.resourcemanager.MockRM.submitApp(MockRM.java:765) 2017-01-30 10:24:01,462 INFO [Thread-6] capacity.CapacityScheduler (CapacityScheduler.java:checkAndGetApplicationPriority(2205)) - Application 'application_1485752041236_0001' is submitted without priority hence considering default queue/cluster priority: 0 2017-01-30 10:24:01,462 INFO [Thread-6] capacity.CapacityScheduler (CapacityScheduler.java:checkAndGetApplicationPriority(2226)) - Priority '0' is acceptable in queue : default for application: application_1485752041236_0001 2017-01-30 10:24:01,474 INFO [Thread-6] resourcemanager.ClientRMService (ClientRMService.java:submitApplication(662)) - Application with id 1 submitted by user user 2017-01-30 10:24:01,474 DEBUG [DelegationTokenRenewer #0] security.DelegationTokenRenewer (DelegationTokenRenewer.java:handleAppSubmitEvent(432)) - Registering tokens for renewal for: appId = application_1485752041236_0001 2017-01-30 10:24:01,477 INFO [Thread-6] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(194)) - USER=user OPERATION=Submit Application Request TARGET=ClientRMService RESULT=SUCCESS APPID=application_1485752041236_0001 2017-01-30 10:24:01,480 INFO [Thread-6] resourcemanager.MockRM (MockRM.java:waitForState(280)) - App : application_1485752041236_0001 State is : NEW Waiting for state : ACCEPTED 2017-01-30 10:24:01,488 INFO [DelegationTokenRenewer #0] security.DelegationTokenRenewer (DelegationTokenRenewer.java:handleAppSubmitEvent(450)) - application_1485752041236_0001 found existing hdfs token Kind: HDFS_DELEGATION_TOKEN, Service: service1, Ident: (token for user: HDFS_DELEGATION_TOKEN owner=user, renewer=renewer1, realUser=user, issueDate=0, maxDate=0, sequenceNumber=0, masterKeyId=0) 2017-01-30 10:24:01,489 DEBUG [DelegationTokenRenewer #0] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1919)) - PrivilegedAction as:root (auth:KERBEROS) from:org.apache.hadoop.yarn.server.resourcemanager.security.DelegationTokenRenewer.renewToken(DelegationTokenRenewer.java:613) 2017-01-30 10:24:01,490 INFO [DelegationTokenRenewer #0] security.DelegationTokenRenewer (DelegationTokenRenewer.java:renewToken(623)) - Renewed delegation-token= [Kind: HDFS_DELEGATION_TOKEN, Service: service1, Ident: (token for user: HDFS_DELEGATION_TOKEN owner=user, renewer=renewer1, realUser=user, issueDate=0, maxDate=0, sequenceNumber=0, masterKeyId=0);exp=1485752044490; apps=[application_1485752041236_0001]] 2017-01-30 10:24:01,490 INFO [Thread-6] resourcemanager.MockRM (MockRM.java:waitForState(280)) - App : application_1485752041236_0001 State is : NEW Waiting for state : ACCEPTED 2017-01-30 10:24:01,490 INFO [DelegationTokenRenewer #0] security.DelegationTokenRenewer (DelegationTokenRenewer.java:setTimerForTokenRenewal(601)) - Renew Kind: HDFS_DELEGATION_TOKEN, Service: service1, Ident: (token for user: HDFS_DELEGATION_TOKEN owner=user, renewer=renewer1, realUser=user, issueDate=0, maxDate=0, sequenceNumber=0, masterKeyId=0);exp=1485752044490; apps=[application_1485752041236_0001] in 3000 ms, appId = [application_1485752041236_0001] 2017-01-30 10:24:01,491 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEvent.EventType: START 2017-01-30 10:24:01,491 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(901)) - Processing event for application_1485752041236_0001 of type START 2017-01-30 10:24:01,491 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:transition(1284)) - Storing application with id application_1485752041236_0001 2017-01-30 10:24:01,497 DEBUG [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:handleStoreEvent(1094)) - Processing event of type STORE_APP 2017-01-30 10:24:01,497 INFO [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:transition(217)) - Storing info for app: application_1485752041236_0001 2017-01-30 10:24:01,497 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(917)) - application_1485752041236_0001 State change from NEW to NEW_SAVING on event = START 2017-01-30 10:24:01,497 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEvent.EventType: APP_NEW_SAVED 2017-01-30 10:24:01,498 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(901)) - Processing event for application_1485752041236_0001 of type APP_NEW_SAVED 2017-01-30 10:24:01,498 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(917)) - application_1485752041236_0001 State change from NEW_SAVING to SUBMITTED on event = APP_NEW_SAVED 2017-01-30 10:24:01,498 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.AppAddedSchedulerEvent.EventType: APP_ADDED 2017-01-30 10:24:01,500 INFO [AsyncDispatcher event handler] capacity.ParentQueue (ParentQueue.java:addApplication(427)) - Application added - appId: application_1485752041236_0001 user: user leaf-queue of parent: root #applications: 1 2017-01-30 10:24:01,501 INFO [AsyncDispatcher event handler] capacity.CapacityScheduler (CapacityScheduler.java:addApplication(729)) - Accepted application application_1485752041236_0001 from user: user, in queue: default 2017-01-30 10:24:01,501 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEvent.EventType: APP_ACCEPTED 2017-01-30 10:24:01,501 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(901)) - Processing event for application_1485752041236_0001 of type APP_ACCEPTED 2017-01-30 10:24:01,501 INFO [Thread-6] resourcemanager.MockRM (MockRM.java:waitForState(280)) - App : application_1485752041236_0001 State is : SUBMITTED Waiting for state : ACCEPTED 2017-01-30 10:24:01,519 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(917)) - application_1485752041236_0001 State change from SUBMITTED to ACCEPTED on event = APP_ACCEPTED 2017-01-30 10:24:01,519 INFO [Thread-6] resourcemanager.MockRM (MockRM.java:waitForState(286)) - App State is : ACCEPTED 2017-01-30 10:24:01,519 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppStartAttemptEvent.EventType: START 2017-01-30 10:24:01,519 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(864)) - Processing event for appattempt_1485752041236_0001_000001 of type START 2017-01-30 10:24:01,536 INFO [AsyncDispatcher event handler] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:registerAppAttempt(776)) - Registering app attempt : appattempt_1485752041236_0001_000001 2017-01-30 10:24:01,537 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(880)) - appattempt_1485752041236_0001_000001 State change from NEW to SUBMITTED on event = START 2017-01-30 10:24:01,537 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.AppAttemptAddedSchedulerEvent.EventType: APP_ATTEMPT_ADDED 2017-01-30 10:24:01,552 DEBUG [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:activateApplications(777)) - application application_1485752041236_0001 AMResource maxAMResourcePerQueuePercent 0.1 amLimit lastClusterResource amIfStarted AM node-partition name 2017-01-30 10:24:01,553 INFO [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:activateApplications(853)) - Application application_1485752041236_0001 from user: user activated in queue: default 2017-01-30 10:24:01,554 INFO [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:addApplicationAttempt(883)) - Application added - appId: application_1485752041236_0001 user: user, leaf-queue: default #user-pending-applications: 0 #user-active-applications: 1 #queue-pending-applications: 0 #queue-active-applications: 1 2017-01-30 10:24:01,554 INFO [AsyncDispatcher event handler] capacity.CapacityScheduler (CapacityScheduler.java:addApplicationAttempt(775)) - Added Application Attempt appattempt_1485752041236_0001_000001 to scheduler from user user in queue default 2017-01-30 10:24:01,554 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: ATTEMPT_ADDED 2017-01-30 10:24:01,554 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(864)) - Processing event for appattempt_1485752041236_0001_000001 of type ATTEMPT_ADDED 2017-01-30 10:24:01,554 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:transition(1065)) - Using blacklist for AM: additions([]) and removals([]) 2017-01-30 10:24:01,555 DEBUG [AsyncDispatcher event handler] capacity.CapacityScheduler (CapacityScheduler.java:allocate(956)) - allocate: pre-update appattempt_1485752041236_0001_000001 ask size =1 2017-01-30 10:24:01,557 DEBUG [AsyncDispatcher event handler] scheduler.ActiveUsersManager (ActiveUsersManager.java:activateApplication(68)) - User user added to activeUsers, currently: 1 2017-01-30 10:24:01,557 DEBUG [AsyncDispatcher event handler] capacity.CapacityScheduler (CapacityScheduler.java:allocate(969)) - allocate: post-update 2017-01-30 10:24:01,558 DEBUG [AsyncDispatcher event handler] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(620)) - showRequests: application=application_1485752041236_0001 headRoom= currentConsumption=0 2017-01-30 10:24:01,558 DEBUG [AsyncDispatcher event handler] placement.LocalitySchedulingPlacementSet (LocalitySchedulingPlacementSet.java:showRequests(369)) - Request={AllocationRequestId: 0, Priority: 0, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: {Execution Type: GUARANTEED, Enforce Execution Type: false}, Node Label Expression: } 2017-01-30 10:24:01,559 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(880)) - appattempt_1485752041236_0001_000001 State change from SUBMITTED to SCHEDULED on event = ATTEMPT_ADDED 2017-01-30 10:24:01,560 INFO [Thread-6] resourcemanager.MockRM (MockRM.java:waitForState(360)) - Attempt State is : SCHEDULED 2017-01-30 10:24:01,560 INFO [Thread-6] resourcemanager.MockRM (MockRM.java:waitForState(286)) - App State is : ACCEPTED 2017-01-30 10:24:01,560 INFO [Thread-6] resourcemanager.MockRM (MockRM.java:waitForState(360)) - Attempt State is : SCHEDULED 2017-01-30 10:24:01,560 INFO [Thread-6] resourcemanager.MockRM (MockRM.java:launchAM(1110)) - Launch AM appattempt_1485752041236_0001_000001 2017-01-30 10:24:01,582 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2017-01-30 10:24:01,582 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(657)) - Processing 127.0.0.1:1234 of type STATUS_UPDATE 2017-01-30 10:24:01,582 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2017-01-30 10:24:01,583 DEBUG [AsyncDispatcher event handler] scheduler.AbstractYarnScheduler (AbstractYarnScheduler.java:nodeUpdate(1015)) - nodeUpdate: 127.0.0.1:1234 cluster capacity: 2017-01-30 10:24:01,583 DEBUG [AsyncDispatcher event handler] scheduler.AbstractYarnScheduler (AbstractYarnScheduler.java:nodeUpdate(1048)) - Node being looked for scheduling 127.0.0.1:1234 availableResource: 2017-01-30 10:24:01,588 DEBUG [AsyncDispatcher event handler] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainerOnSingleNode(1283)) - Trying to schedule on node: 127.0.0.1, available: 2017-01-30 10:24:01,588 DEBUG [AsyncDispatcher event handler] capacity.ParentQueue (ParentQueue.java:assignContainers(521)) - Trying to assign containers to child-queue of root 2017-01-30 10:24:01,589 DEBUG [AsyncDispatcher event handler] capacity.AbstractCSQueue (AbstractCSQueue.java:canAssignToThisQueue(665)) - Check assign to queue: root nodePartition: , usedResources: , clusterResources: , currentUsedCapacity: 0.0, max-capacity: 1.0 2017-01-30 10:24:01,590 DEBUG [AsyncDispatcher event handler] capacity.ParentQueue (ParentQueue.java:printChildQueues(770)) - printChildQueues - queue: root child-queues: root.defaultusedCapacity=(0.0), label=(*) 2017-01-30 10:24:01,591 DEBUG [AsyncDispatcher event handler] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(712)) - Trying to assign to queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=1, numContainers=0 2017-01-30 10:24:01,591 DEBUG [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:assignContainers(1003)) - assignContainers: partition= #applications=1 2017-01-30 10:24:01,592 DEBUG [AsyncDispatcher event handler] capacity.AbstractCSQueue (AbstractCSQueue.java:canAssignToThisQueue(665)) - Check assign to queue: default nodePartition: , usedResources: , clusterResources: , currentUsedCapacity: 0.0, max-capacity: 1.0 2017-01-30 10:24:01,592 DEBUG [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1498)) - User limit computation for user in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: consumedRatio: 0.0 currentCapacity: activeUsers: 1 clusterCapacity: resourceByLabel: usageratio: 0.0 Partition: 2017-01-30 10:24:01,593 DEBUG [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:computeUserLimitAndSetHeadroom(1370)) - Headroom calculation for user user: userLimit= queueMaxAvailRes= consumed= headroom= 2017-01-30 10:24:01,595 DEBUG [AsyncDispatcher event handler] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:assignContainers(955)) - pre-assignContainers for application application_1485752041236_0001 2017-01-30 10:24:01,596 DEBUG [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1498)) - User limit computation for user in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: consumedRatio: 0.0 currentCapacity: activeUsers: 1 clusterCapacity: resourceByLabel: usageratio: 0.0 Partition: 2017-01-30 10:24:01,596 DEBUG [AsyncDispatcher event handler] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(620)) - showRequests: application=application_1485752041236_0001 headRoom= currentConsumption=0 2017-01-30 10:24:01,596 DEBUG [AsyncDispatcher event handler] placement.LocalitySchedulingPlacementSet (LocalitySchedulingPlacementSet.java:showRequests(369)) - Request={AllocationRequestId: 0, Priority: 0, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: {Execution Type: GUARANTEED, Enforce Execution Type: false}, Node Label Expression: } 2017-01-30 10:24:01,599 DEBUG [AsyncDispatcher event handler] allocator.IncreaseContainerAllocator (IncreaseContainerAllocator.java:assignContainers(163)) - Skip allocating increase request since we don't have any increase request on this node=127.0.0.1:1234 2017-01-30 10:24:01,604 DEBUG [AsyncDispatcher event handler] allocator.RegularContainerAllocator (RegularContainerAllocator.java:assignContainer(478)) - assignContainers: node=127.0.0.1 application=application_1485752041236_0001 priority=0 pendingAsk=,repeat=1> type=OFF_SWITCH 2017-01-30 10:24:01,616 INFO [AsyncDispatcher event handler] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(127)) - assignedContainer application attempt=appattempt_1485752041236_0001_000001 container=null queue=default clusterResource= type=OFF_SWITCH 2017-01-30 10:24:01,617 DEBUG [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:assignContainers(1081)) - post-assignContainers for application application_1485752041236_0001 2017-01-30 10:24:01,617 DEBUG [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1498)) - User limit computation for user in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: consumedRatio: 0.0 currentCapacity: activeUsers: 1 clusterCapacity: resourceByLabel: usageratio: 0.0 Partition: 2017-01-30 10:24:01,617 DEBUG [AsyncDispatcher event handler] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(620)) - showRequests: application=application_1485752041236_0001 headRoom= currentConsumption=0 2017-01-30 10:24:01,617 DEBUG [AsyncDispatcher event handler] placement.LocalitySchedulingPlacementSet (LocalitySchedulingPlacementSet.java:showRequests(369)) - Request={AllocationRequestId: 0, Priority: 0, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: {Execution Type: GUARANTEED, Enforce Execution Type: false}, Node Label Expression: } 2017-01-30 10:24:01,618 DEBUG [AsyncDispatcher event handler] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(724)) - Assigned to queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=1, numContainers=0 --> , OFF_SWITCH 2017-01-30 10:24:01,618 INFO [AsyncDispatcher event handler] capacity.ParentQueue (ParentQueue.java:assignContainers(603)) - assignedContainer queue=root usedCapacity=0.0 absoluteUsedCapacity=0.0 used= cluster= 2017-01-30 10:24:01,618 DEBUG [AsyncDispatcher event handler] capacity.ParentQueue (ParentQueue.java:assignContainers(609)) - ParentQ=root assignedSoFarInThisIteration= usedCapacity=0.0 absoluteUsedCapacity=0.0 2017-01-30 10:24:01,619 DEBUG [AsyncDispatcher event handler] capacity.CapacityScheduler (CapacityScheduler.java:tryCommit(2473)) - Try to commit allocation proposal=New org.apache.hadoop.yarn.server.resourcemanager.scheduler.common.ResourceCommitRequest: ALLOCATED=[(Application=appattempt_1485752041236_0001_000001; Node=127.0.0.1:1234; Resource=)] 2017-01-30 10:24:01,620 DEBUG [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1498)) - User limit computation for user in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: consumedRatio: 0.0 currentCapacity: activeUsers: 1 clusterCapacity: resourceByLabel: usageratio: 0.0 Partition: 2017-01-30 10:24:01,620 DEBUG [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:computeUserLimitAndSetHeadroom(1370)) - Headroom calculation for user user: userLimit= queueMaxAvailRes= consumed= headroom= 2017-01-30 10:24:01,620 DEBUG [AsyncDispatcher event handler] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:updateMetricsForAllocatedContainer(880)) - allocate: applicationId=application_1485752041236_0001 container=container_1485752041236_0001_01_000001 host=127.0.0.1:1234 user=user resource= type=OFF_SWITCH 2017-01-30 10:24:01,621 DEBUG [AsyncDispatcher event handler] scheduler.ActiveUsersManager (ActiveUsersManager.java:deactivateApplication(94)) - User user removed from activeUsers, currently: 0 2017-01-30 10:24:01,621 DEBUG [AsyncDispatcher event handler] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(476)) - Processing container_1485752041236_0001_01_000001 of type START 2017-01-30 10:24:01,621 INFO [AsyncDispatcher event handler] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(490)) - container_1485752041236_0001_01_000001 Container Transitioned from NEW to ALLOCATED 2017-01-30 10:24:01,622 DEBUG [AsyncDispatcher event handler] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(174)) - Assigned container container_1485752041236_0001_01_000001 of capacity on host 127.0.0.1:1234, which has 1 containers, used and available after allocation 2017-01-30 10:24:01,622 DEBUG [AsyncDispatcher event handler] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:apply(595)) - allocate: applicationAttemptId=appattempt_1485752041236_0001_000001 container=container_1485752041236_0001_01_000001 host=127.0.0.1 type=OFF_SWITCH 2017-01-30 10:24:01,622 INFO [AsyncDispatcher event handler] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(165)) - USER=user OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1485752041236_0001 CONTAINERID=container_1485752041236_0001_01_000001 RESOURCE= 2017-01-30 10:24:01,622 DEBUG [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1498)) - User limit computation for user in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: consumedRatio: 0.0 currentCapacity: activeUsers: 0 clusterCapacity: resourceByLabel: usageratio: 0.06772487 Partition: 2017-01-30 10:24:01,623 DEBUG [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1498)) - User limit computation for user in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: consumedRatio: 0.0 currentCapacity: activeUsers: 0 clusterCapacity: resourceByLabel: usageratio: 0.06772487 Partition: 2017-01-30 10:24:01,623 DEBUG [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1498)) - User limit computation for user in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: consumedRatio: 0.0 currentCapacity: activeUsers: 0 clusterCapacity: resourceByLabel: usageratio: 0.06772487 Partition: 2017-01-30 10:24:01,623 DEBUG [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:allocateResource(1785)) - default user=user used= numContainers=1 headroom = user-resources= 2017-01-30 10:24:01,623 INFO [AsyncDispatcher event handler] capacity.ParentQueue (ParentQueue.java:apply(1060)) - assignedContainer queue=root usedCapacity=0.06772487 absoluteUsedCapacity=0.06772487 used= cluster= 2017-01-30 10:24:01,623 INFO [AsyncDispatcher event handler] capacity.CapacityScheduler (CapacityScheduler.java:tryCommit(2481)) - Allocation proposal accepted 2017-01-30 10:24:01,623 DEBUG [AsyncDispatcher event handler] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1185)) - Assigned maximum number of off-switch containers: 1, assignments so far: resource:; type:OFF_SWITCH; excessReservation:null; applicationid:null; skipped:NONE; fulfilled reservation:false; allocations(count/resource):1/; reservations(count/resource):0/ 2017-01-30 10:24:01,624 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: CONTAINER_ALLOCATED 2017-01-30 10:24:01,624 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(864)) - Processing event for appattempt_1485752041236_0001_000001 of type CONTAINER_ALLOCATED 2017-01-30 10:24:01,628 DEBUG [AsyncDispatcher event handler] security.BaseContainerTokenSecretManager (BaseContainerTokenSecretManager.java:createPassword(90)) - Creating password for container_1485752041236_0001_01_000001 for user container_1485752041236_0001_01_000001 (auth:SIMPLE) to be run on NM 127.0.0.1:1234 2017-01-30 10:24:01,682 DEBUG [AsyncDispatcher event handler] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(248)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 1 cluster_timestamp: 1485752041236 } attemptId: 1 } id: 1 } nmHostAddr: "127.0.0.1:1234" appSubmitter: "user" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1485752641624 masterKeyId: 313252934 rmIdentifier: 1485752041236 priority { priority: 0 } creationTime: 1485752041612 nodeLabelExpression: "" containerType: APPLICATION_MASTER executionType: GUARANTEED version: 0 2017-01-30 10:24:01,684 DEBUG [AsyncDispatcher event handler] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(248)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 1 cluster_timestamp: 1485752041236 } attemptId: 1 } id: 1 } nmHostAddr: "127.0.0.1:1234" appSubmitter: "user" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1485752641624 masterKeyId: 313252934 rmIdentifier: 1485752041236 priority { priority: 0 } creationTime: 1485752041612 nodeLabelExpression: "" containerType: APPLICATION_MASTER executionType: GUARANTEED version: 0 2017-01-30 10:24:01,685 INFO [AsyncDispatcher event handler] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:createAndGetNMToken(200)) - Sending NMToken for nodeId : 127.0.0.1:1234 for container : container_1485752041236_0001_01_000001 2017-01-30 10:24:01,687 DEBUG [AsyncDispatcher event handler] security.BaseNMTokenSecretManager (BaseNMTokenSecretManager.java:createPassword(75)) - creating password for appattempt_1485752041236_0001_000001 for user user to run on NM 127.0.0.1:1234 2017-01-30 10:24:01,688 DEBUG [AsyncDispatcher event handler] security.NMTokenIdentifier (NMTokenIdentifier.java:write(96)) - Writing NMTokenIdentifier to RPC layer: appAttemptId { application_id { id: 1 cluster_timestamp: 1485752041236 } attemptId: 1 } nodeId { host: "127.0.0.1" port: 1234 } appSubmitter: "user" keyId: -671765034 2017-01-30 10:24:01,689 DEBUG [AsyncDispatcher event handler] security.NMTokenIdentifier (NMTokenIdentifier.java:write(96)) - Writing NMTokenIdentifier to RPC layer: appAttemptId { application_id { id: 1 cluster_timestamp: 1485752041236 } attemptId: 1 } nodeId { host: "127.0.0.1" port: 1234 } appSubmitter: "user" keyId: -671765034 2017-01-30 10:24:01,691 DEBUG [AsyncDispatcher event handler] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(476)) - Processing container_1485752041236_0001_01_000001 of type ACQUIRED 2017-01-30 10:24:01,692 INFO [AsyncDispatcher event handler] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(490)) - container_1485752041236_0001_01_000001 Container Transitioned from ALLOCATED to ACQUIRED 2017-01-30 10:24:01,692 DEBUG [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1498)) - User limit computation for user in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: consumedRatio: 0.0 currentCapacity: activeUsers: 0 clusterCapacity: resourceByLabel: usageratio: 0.06772487 Partition: 2017-01-30 10:24:01,692 INFO [AsyncDispatcher event handler] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:clearNodeSetForAttempt(146)) - Clear node set for appattempt_1485752041236_0001_000001 2017-01-30 10:24:01,692 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:storeAttempt(2134)) - Storing attempt: AppId: application_1485752041236_0001 AttemptId: appattempt_1485752041236_0001_000001 MasterContainer: Container: [ContainerId: container_1485752041236_0001_01_000001, AllocationRequestId: 0, Version: 0, NodeId: 127.0.0.1:1234, NodeHttpAddress: 127.0.0.1:2, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 127.0.0.1:1234 }, ExecutionType: GUARANTEED, ] 2017-01-30 10:24:01,698 DEBUG [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:handleStoreEvent(1094)) - Processing event of type STORE_APP_ATTEMPT 2017-01-30 10:24:01,698 DEBUG [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:transition(315)) - Storing info for attempt: appattempt_1485752041236_0001_000001 2017-01-30 10:24:01,698 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(880)) - appattempt_1485752041236_0001_000001 State change from SCHEDULED to ALLOCATED_SAVING on event = CONTAINER_ALLOCATED 2017-01-30 10:24:01,698 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppRunningOnNodeEvent.EventType: APP_RUNNING_ON_NODE 2017-01-30 10:24:01,698 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(901)) - Processing event for application_1485752041236_0001 of type APP_RUNNING_ON_NODE 2017-01-30 10:24:01,701 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: ATTEMPT_NEW_SAVED 2017-01-30 10:24:01,701 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(864)) - Processing event for appattempt_1485752041236_0001_000001 of type ATTEMPT_NEW_SAVED 2017-01-30 10:24:01,702 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(880)) - appattempt_1485752041236_0001_000001 State change from ALLOCATED_SAVING to ALLOCATED on event = ATTEMPT_NEW_SAVED 2017-01-30 10:24:01,702 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncherEvent.EventType: LAUNCH 2017-01-30 10:24:01,702 INFO [Thread-6] resourcemanager.MockRM (MockRM.java:waitForState(360)) - Attempt State is : ALLOCATED 2017-01-30 10:24:01,702 INFO [Thread-6] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:createAndGetAMRMToken(195)) - Create AMRMToken for ApplicationAttempt: appattempt_1485752041236_0001_000001 2017-01-30 10:24:01,704 INFO [Thread-6] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:createPassword(307)) - Creating password for appattempt_1485752041236_0001_000001 2017-01-30 10:24:01,705 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: LAUNCHED 2017-01-30 10:24:01,705 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(864)) - Processing event for appattempt_1485752041236_0001_000001 of type LAUNCHED 2017-01-30 10:24:01,705 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(880)) - appattempt_1485752041236_0001_000001 State change from ALLOCATED to LAUNCHED on event = LAUNCHED 2017-01-30 10:24:01,705 INFO [Thread-6] resourcemanager.MockRM (MockRM.java:waitForState(360)) - Attempt State is : LAUNCHED 2017-01-30 10:24:01,705 INFO [Thread-6] resourcemanager.MockRM (MockRM.java:waitForState(360)) - Attempt State is : LAUNCHED 2017-01-30 10:24:01,709 DEBUG [Thread-6] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1919)) - PrivilegedAction as:appattempt_1485752041236_0001_000001 (auth:SIMPLE) from:org.apache.hadoop.yarn.server.resourcemanager.MockAM.registerAppAttempt(MockAM.java:117) 2017-01-30 10:24:01,710 INFO [Thread-6] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:registerApplicationMaster(245)) - AM registration appattempt_1485752041236_0001_000001 2017-01-30 10:24:01,711 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptRegistrationEvent.EventType: REGISTERED 2017-01-30 10:24:01,711 INFO [Thread-6] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(186)) - USER=user OPERATION=Register App Master TARGET=ApplicationMasterService RESULT=SUCCESS APPID=application_1485752041236_0001 APPATTEMPTID=appattempt_1485752041236_0001_000001 2017-01-30 10:24:01,711 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(864)) - Processing event for appattempt_1485752041236_0001_000001 of type REGISTERED 2017-01-30 10:24:01,711 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(880)) - appattempt_1485752041236_0001_000001 State change from LAUNCHED to RUNNING on event = REGISTERED 2017-01-30 10:24:01,711 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEvent.EventType: ATTEMPT_REGISTERED 2017-01-30 10:24:01,711 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(901)) - Processing event for application_1485752041236_0001 of type ATTEMPT_REGISTERED 2017-01-30 10:24:01,712 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(917)) - application_1485752041236_0001 State change from ACCEPTED to RUNNING on event = ATTEMPT_REGISTERED 2017-01-30 10:24:01,715 INFO [Thread-6] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:registerApplicationMaster(264)) - Setting client token master key 2017-01-30 10:24:01,716 INFO [Thread-6] resourcemanager.MockRM (MockRM.java:waitForState(286)) - App State is : RUNNING 2017-01-30 10:24:01,716 INFO [Thread-6] resourcemanager.MockRM (MockRM.java:waitForState(286)) - App State is : RUNNING 2017-01-30 10:24:01,716 INFO [Thread-6] resourcemanager.ClientRMService (ClientRMService.java:getNewApplicationId(320)) - Allocated new applicationId: 2 2017-01-30 10:24:01,717 DEBUG [Thread-6] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1919)) - PrivilegedAction as:user (auth:SIMPLE) from:org.apache.hadoop.yarn.server.resourcemanager.MockRM.submitApp(MockRM.java:765) 2017-01-30 10:24:01,717 INFO [Thread-6] capacity.CapacityScheduler (CapacityScheduler.java:checkAndGetApplicationPriority(2205)) - Application 'application_1485752041236_0002' is submitted without priority hence considering default queue/cluster priority: 0 2017-01-30 10:24:01,717 INFO [Thread-6] capacity.CapacityScheduler (CapacityScheduler.java:checkAndGetApplicationPriority(2226)) - Priority '0' is acceptable in queue : default for application: application_1485752041236_0002 2017-01-30 10:24:01,718 INFO [Thread-6] resourcemanager.ClientRMService (ClientRMService.java:submitApplication(662)) - Application with id 2 submitted by user user 2017-01-30 10:24:01,718 INFO [Thread-6] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(194)) - USER=user OPERATION=Submit Application Request TARGET=ClientRMService RESULT=SUCCESS APPID=application_1485752041236_0002 2017-01-30 10:24:01,718 DEBUG [DelegationTokenRenewer #1] security.DelegationTokenRenewer (DelegationTokenRenewer.java:handleAppSubmitEvent(432)) - Registering tokens for renewal for: appId = application_1485752041236_0002 2017-01-30 10:24:01,718 INFO [Thread-6] resourcemanager.MockRM (MockRM.java:waitForState(280)) - App : application_1485752041236_0002 State is : NEW Waiting for state : ACCEPTED 2017-01-30 10:24:01,718 INFO [DelegationTokenRenewer #1] security.DelegationTokenRenewer (DelegationTokenRenewer.java:handleAppSubmitEvent(450)) - application_1485752041236_0002 found existing hdfs token Kind: HDFS_DELEGATION_TOKEN, Service: service1, Ident: (token for user: HDFS_DELEGATION_TOKEN owner=user, renewer=renewer1, realUser=user, issueDate=0, maxDate=0, sequenceNumber=0, masterKeyId=0) 2017-01-30 10:24:01,719 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEvent.EventType: START 2017-01-30 10:24:01,719 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(901)) - Processing event for application_1485752041236_0002 of type START 2017-01-30 10:24:01,719 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:transition(1284)) - Storing application with id application_1485752041236_0002 2017-01-30 10:24:01,719 DEBUG [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:handleStoreEvent(1094)) - Processing event of type STORE_APP 2017-01-30 10:24:01,720 INFO [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:transition(217)) - Storing info for app: application_1485752041236_0002 2017-01-30 10:24:01,720 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(917)) - application_1485752041236_0002 State change from NEW to NEW_SAVING on event = START 2017-01-30 10:24:01,720 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEvent.EventType: APP_NEW_SAVED 2017-01-30 10:24:01,720 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(901)) - Processing event for application_1485752041236_0002 of type APP_NEW_SAVED 2017-01-30 10:24:01,720 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(917)) - application_1485752041236_0002 State change from NEW_SAVING to SUBMITTED on event = APP_NEW_SAVED 2017-01-30 10:24:01,720 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.AppAddedSchedulerEvent.EventType: APP_ADDED 2017-01-30 10:24:01,720 INFO [AsyncDispatcher event handler] capacity.ParentQueue (ParentQueue.java:addApplication(427)) - Application added - appId: application_1485752041236_0002 user: user leaf-queue of parent: root #applications: 2 2017-01-30 10:24:01,720 INFO [AsyncDispatcher event handler] capacity.CapacityScheduler (CapacityScheduler.java:addApplication(729)) - Accepted application application_1485752041236_0002 from user: user, in queue: default 2017-01-30 10:24:01,721 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEvent.EventType: APP_ACCEPTED 2017-01-30 10:24:01,721 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(901)) - Processing event for application_1485752041236_0002 of type APP_ACCEPTED 2017-01-30 10:24:01,721 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(917)) - application_1485752041236_0002 State change from SUBMITTED to ACCEPTED on event = APP_ACCEPTED 2017-01-30 10:24:01,721 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppStartAttemptEvent.EventType: START 2017-01-30 10:24:01,722 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(864)) - Processing event for appattempt_1485752041236_0002_000001 of type START 2017-01-30 10:24:01,722 INFO [AsyncDispatcher event handler] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:registerAppAttempt(776)) - Registering app attempt : appattempt_1485752041236_0002_000001 2017-01-30 10:24:01,722 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(880)) - appattempt_1485752041236_0002_000001 State change from NEW to SUBMITTED on event = START 2017-01-30 10:24:01,722 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.AppAttemptAddedSchedulerEvent.EventType: APP_ATTEMPT_ADDED 2017-01-30 10:24:01,722 DEBUG [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:activateApplications(777)) - application application_1485752041236_0002 AMResource maxAMResourcePerQueuePercent 0.1 amLimit lastClusterResource amIfStarted AM node-partition name 2017-01-30 10:24:01,723 INFO [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:activateApplications(853)) - Application application_1485752041236_0002 from user: user activated in queue: default 2017-01-30 10:24:01,723 INFO [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:addApplicationAttempt(883)) - Application added - appId: application_1485752041236_0002 user: user, leaf-queue: default #user-pending-applications: 0 #user-active-applications: 2 #queue-pending-applications: 0 #queue-active-applications: 2 2017-01-30 10:24:01,723 INFO [AsyncDispatcher event handler] capacity.CapacityScheduler (CapacityScheduler.java:addApplicationAttempt(775)) - Added Application Attempt appattempt_1485752041236_0002_000001 to scheduler from user user in queue default 2017-01-30 10:24:01,723 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: ATTEMPT_ADDED 2017-01-30 10:24:01,723 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(864)) - Processing event for appattempt_1485752041236_0002_000001 of type ATTEMPT_ADDED 2017-01-30 10:24:01,723 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:transition(1065)) - Using blacklist for AM: additions([]) and removals([]) 2017-01-30 10:24:01,723 DEBUG [AsyncDispatcher event handler] capacity.CapacityScheduler (CapacityScheduler.java:allocate(956)) - allocate: pre-update appattempt_1485752041236_0002_000001 ask size =1 2017-01-30 10:24:01,724 DEBUG [AsyncDispatcher event handler] scheduler.ActiveUsersManager (ActiveUsersManager.java:activateApplication(68)) - User user added to activeUsers, currently: 1 2017-01-30 10:24:01,724 DEBUG [AsyncDispatcher event handler] capacity.CapacityScheduler (CapacityScheduler.java:allocate(969)) - allocate: post-update 2017-01-30 10:24:01,724 DEBUG [AsyncDispatcher event handler] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(620)) - showRequests: application=application_1485752041236_0002 headRoom= currentConsumption=0 2017-01-30 10:24:01,724 DEBUG [AsyncDispatcher event handler] placement.LocalitySchedulingPlacementSet (LocalitySchedulingPlacementSet.java:showRequests(369)) - Request={AllocationRequestId: 0, Priority: 0, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: {Execution Type: GUARANTEED, Enforce Execution Type: false}, Node Label Expression: } 2017-01-30 10:24:01,724 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(880)) - appattempt_1485752041236_0002_000001 State change from SUBMITTED to SCHEDULED on event = ATTEMPT_ADDED 2017-01-30 10:24:01,729 INFO [Thread-6] resourcemanager.MockRM (MockRM.java:waitForState(286)) - App State is : ACCEPTED 2017-01-30 10:24:01,729 INFO [Thread-6] resourcemanager.MockRM (MockRM.java:waitForState(360)) - Attempt State is : SCHEDULED 2017-01-30 10:24:01,729 INFO [Thread-6] resourcemanager.MockRM (MockRM.java:waitForState(286)) - App State is : ACCEPTED 2017-01-30 10:24:01,729 INFO [Thread-6] resourcemanager.MockRM (MockRM.java:waitForState(360)) - Attempt State is : SCHEDULED 2017-01-30 10:24:01,729 INFO [Thread-6] resourcemanager.MockRM (MockRM.java:launchAM(1110)) - Launch AM appattempt_1485752041236_0002_000001 2017-01-30 10:24:01,729 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2017-01-30 10:24:01,729 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(657)) - Processing 127.0.0.1:1234 of type STATUS_UPDATE 2017-01-30 10:24:01,730 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2017-01-30 10:24:01,730 DEBUG [AsyncDispatcher event handler] scheduler.AbstractYarnScheduler (AbstractYarnScheduler.java:nodeUpdate(1015)) - nodeUpdate: 127.0.0.1:1234 cluster capacity: 2017-01-30 10:24:01,730 DEBUG [AsyncDispatcher event handler] scheduler.AbstractYarnScheduler (AbstractYarnScheduler.java:nodeUpdate(1048)) - Node being looked for scheduling 127.0.0.1:1234 availableResource: 2017-01-30 10:24:01,730 DEBUG [AsyncDispatcher event handler] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainerOnSingleNode(1283)) - Trying to schedule on node: 127.0.0.1, available: 2017-01-30 10:24:01,730 DEBUG [AsyncDispatcher event handler] capacity.ParentQueue (ParentQueue.java:assignContainers(521)) - Trying to assign containers to child-queue of root 2017-01-30 10:24:01,730 DEBUG [AsyncDispatcher event handler] capacity.AbstractCSQueue (AbstractCSQueue.java:canAssignToThisQueue(665)) - Check assign to queue: root nodePartition: , usedResources: , clusterResources: , currentUsedCapacity: 0.06772487, max-capacity: 1.0 2017-01-30 10:24:01,731 DEBUG [AsyncDispatcher event handler] capacity.ParentQueue (ParentQueue.java:printChildQueues(770)) - printChildQueues - queue: root child-queues: root.defaultusedCapacity=(0.06772487), label=(*) 2017-01-30 10:24:01,731 DEBUG [AsyncDispatcher event handler] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(712)) - Trying to assign to queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.06772487, absoluteUsedCapacity=0.06772487, numApps=2, numContainers=1 2017-01-30 10:24:01,731 DEBUG [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:assignContainers(1003)) - assignContainers: partition= #applications=2 2017-01-30 10:24:01,731 DEBUG [AsyncDispatcher event handler] capacity.AbstractCSQueue (AbstractCSQueue.java:canAssignToThisQueue(665)) - Check assign to queue: default nodePartition: , usedResources: , clusterResources: , currentUsedCapacity: 0.06772487, max-capacity: 1.0 2017-01-30 10:24:01,731 DEBUG [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1498)) - User limit computation for user in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: consumedRatio: 0.0 currentCapacity: activeUsers: 1 clusterCapacity: resourceByLabel: usageratio: 0.06772487 Partition: 2017-01-30 10:24:01,731 DEBUG [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:computeUserLimitAndSetHeadroom(1370)) - Headroom calculation for user user: userLimit= queueMaxAvailRes= consumed= headroom= 2017-01-30 10:24:01,731 DEBUG [AsyncDispatcher event handler] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:assignContainers(955)) - pre-assignContainers for application application_1485752041236_0001 2017-01-30 10:24:01,732 DEBUG [AsyncDispatcher event handler] allocator.IncreaseContainerAllocator (IncreaseContainerAllocator.java:assignContainers(163)) - Skip allocating increase request since we don't have any increase request on this node=127.0.0.1:1234 2017-01-30 10:24:01,732 DEBUG [AsyncDispatcher event handler] allocator.RegularContainerAllocator (RegularContainerAllocator.java:assignContainers(827)) - Skip app_attempt=appattempt_1485752041236_0001_000001, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-label= 2017-01-30 10:24:01,732 DEBUG [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:assignContainers(1081)) - post-assignContainers for application application_1485752041236_0001 2017-01-30 10:24:01,732 DEBUG [AsyncDispatcher event handler] capacity.AbstractCSQueue (AbstractCSQueue.java:canAssignToThisQueue(665)) - Check assign to queue: default nodePartition: , usedResources: , clusterResources: , currentUsedCapacity: 0.06772487, max-capacity: 1.0 2017-01-30 10:24:01,732 DEBUG [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1498)) - User limit computation for user in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: consumedRatio: 0.0 currentCapacity: activeUsers: 1 clusterCapacity: resourceByLabel: usageratio: 0.06772487 Partition: 2017-01-30 10:24:01,732 DEBUG [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:computeUserLimitAndSetHeadroom(1370)) - Headroom calculation for user user: userLimit= queueMaxAvailRes= consumed= headroom= 2017-01-30 10:24:01,732 DEBUG [AsyncDispatcher event handler] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:assignContainers(955)) - pre-assignContainers for application application_1485752041236_0002 2017-01-30 10:24:01,733 DEBUG [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1498)) - User limit computation for user in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: consumedRatio: 0.0 currentCapacity: activeUsers: 1 clusterCapacity: resourceByLabel: usageratio: 0.06772487 Partition: 2017-01-30 10:24:01,733 DEBUG [AsyncDispatcher event handler] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(620)) - showRequests: application=application_1485752041236_0002 headRoom= currentConsumption=0 2017-01-30 10:24:01,733 DEBUG [AsyncDispatcher event handler] placement.LocalitySchedulingPlacementSet (LocalitySchedulingPlacementSet.java:showRequests(369)) - Request={AllocationRequestId: 0, Priority: 0, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: {Execution Type: GUARANTEED, Enforce Execution Type: false}, Node Label Expression: } 2017-01-30 10:24:01,733 DEBUG [AsyncDispatcher event handler] allocator.IncreaseContainerAllocator (IncreaseContainerAllocator.java:assignContainers(163)) - Skip allocating increase request since we don't have any increase request on this node=127.0.0.1:1234 2017-01-30 10:24:01,733 DEBUG [AsyncDispatcher event handler] allocator.RegularContainerAllocator (RegularContainerAllocator.java:assignContainer(478)) - assignContainers: node=127.0.0.1 application=application_1485752041236_0002 priority=0 pendingAsk=,repeat=1> type=OFF_SWITCH 2017-01-30 10:24:01,733 INFO [AsyncDispatcher event handler] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(127)) - assignedContainer application attempt=appattempt_1485752041236_0002_000001 container=null queue=default clusterResource= type=OFF_SWITCH 2017-01-30 10:24:01,734 DEBUG [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:assignContainers(1081)) - post-assignContainers for application application_1485752041236_0002 2017-01-30 10:24:01,734 DEBUG [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1498)) - User limit computation for user in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: consumedRatio: 0.0 currentCapacity: activeUsers: 1 clusterCapacity: resourceByLabel: usageratio: 0.06772487 Partition: 2017-01-30 10:24:01,734 DEBUG [AsyncDispatcher event handler] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(620)) - showRequests: application=application_1485752041236_0002 headRoom= currentConsumption=0 2017-01-30 10:24:01,734 DEBUG [AsyncDispatcher event handler] placement.LocalitySchedulingPlacementSet (LocalitySchedulingPlacementSet.java:showRequests(369)) - Request={AllocationRequestId: 0, Priority: 0, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: {Execution Type: GUARANTEED, Enforce Execution Type: false}, Node Label Expression: } 2017-01-30 10:24:01,734 DEBUG [AsyncDispatcher event handler] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(724)) - Assigned to queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.06772487, absoluteUsedCapacity=0.06772487, numApps=2, numContainers=1 --> , OFF_SWITCH 2017-01-30 10:24:01,734 INFO [AsyncDispatcher event handler] capacity.ParentQueue (ParentQueue.java:assignContainers(603)) - assignedContainer queue=root usedCapacity=0.06772487 absoluteUsedCapacity=0.06772487 used= cluster= 2017-01-30 10:24:01,735 DEBUG [AsyncDispatcher event handler] capacity.ParentQueue (ParentQueue.java:assignContainers(609)) - ParentQ=root assignedSoFarInThisIteration= usedCapacity=0.06772487 absoluteUsedCapacity=0.06772487 2017-01-30 10:24:01,735 DEBUG [AsyncDispatcher event handler] capacity.CapacityScheduler (CapacityScheduler.java:tryCommit(2473)) - Try to commit allocation proposal=New org.apache.hadoop.yarn.server.resourcemanager.scheduler.common.ResourceCommitRequest: ALLOCATED=[(Application=appattempt_1485752041236_0002_000001; Node=127.0.0.1:1234; Resource=)] 2017-01-30 10:24:01,735 DEBUG [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1498)) - User limit computation for user in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: consumedRatio: 0.0 currentCapacity: activeUsers: 1 clusterCapacity: resourceByLabel: usageratio: 0.06772487 Partition: 2017-01-30 10:24:01,735 DEBUG [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:computeUserLimitAndSetHeadroom(1370)) - Headroom calculation for user user: userLimit= queueMaxAvailRes= consumed= headroom= 2017-01-30 10:24:01,735 DEBUG [AsyncDispatcher event handler] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:updateMetricsForAllocatedContainer(880)) - allocate: applicationId=application_1485752041236_0002 container=container_1485752041236_0002_01_000001 host=127.0.0.1:1234 user=user resource= type=OFF_SWITCH 2017-01-30 10:24:01,736 DEBUG [AsyncDispatcher event handler] scheduler.ActiveUsersManager (ActiveUsersManager.java:deactivateApplication(94)) - User user removed from activeUsers, currently: 0 2017-01-30 10:24:01,736 DEBUG [AsyncDispatcher event handler] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(476)) - Processing container_1485752041236_0002_01_000001 of type START 2017-01-30 10:24:01,736 INFO [AsyncDispatcher event handler] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(490)) - container_1485752041236_0002_01_000001 Container Transitioned from NEW to ALLOCATED 2017-01-30 10:24:01,736 DEBUG [AsyncDispatcher event handler] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(174)) - Assigned container container_1485752041236_0002_01_000001 of capacity on host 127.0.0.1:1234, which has 2 containers, used and available after allocation 2017-01-30 10:24:01,736 DEBUG [AsyncDispatcher event handler] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:apply(595)) - allocate: applicationAttemptId=appattempt_1485752041236_0002_000001 container=container_1485752041236_0002_01_000001 host=127.0.0.1 type=OFF_SWITCH 2017-01-30 10:24:01,736 INFO [AsyncDispatcher event handler] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(165)) - USER=user OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1485752041236_0002 CONTAINERID=container_1485752041236_0002_01_000001 RESOURCE= 2017-01-30 10:24:01,737 DEBUG [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1498)) - User limit computation for user in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: consumedRatio: 0.0 currentCapacity: activeUsers: 0 clusterCapacity: resourceByLabel: usageratio: 0.13544974 Partition: 2017-01-30 10:24:01,737 DEBUG [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1498)) - User limit computation for user in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: consumedRatio: 0.0 currentCapacity: activeUsers: 0 clusterCapacity: resourceByLabel: usageratio: 0.13544974 Partition: 2017-01-30 10:24:01,737 DEBUG [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1498)) - User limit computation for user in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: consumedRatio: 0.0 currentCapacity: activeUsers: 0 clusterCapacity: resourceByLabel: usageratio: 0.13544974 Partition: 2017-01-30 10:24:01,737 DEBUG [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:allocateResource(1785)) - default user=user used= numContainers=2 headroom = user-resources= 2017-01-30 10:24:01,737 INFO [AsyncDispatcher event handler] capacity.ParentQueue (ParentQueue.java:apply(1060)) - assignedContainer queue=root usedCapacity=0.13544974 absoluteUsedCapacity=0.13544974 used= cluster= 2017-01-30 10:24:01,737 INFO [AsyncDispatcher event handler] capacity.CapacityScheduler (CapacityScheduler.java:tryCommit(2481)) - Allocation proposal accepted 2017-01-30 10:24:01,738 DEBUG [AsyncDispatcher event handler] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1185)) - Assigned maximum number of off-switch containers: 1, assignments so far: resource:; type:OFF_SWITCH; excessReservation:null; applicationid:null; skipped:NONE; fulfilled reservation:false; allocations(count/resource):1/; reservations(count/resource):0/ 2017-01-30 10:24:01,738 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: CONTAINER_ALLOCATED 2017-01-30 10:24:01,738 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(864)) - Processing event for appattempt_1485752041236_0002_000001 of type CONTAINER_ALLOCATED 2017-01-30 10:24:01,738 DEBUG [AsyncDispatcher event handler] security.BaseContainerTokenSecretManager (BaseContainerTokenSecretManager.java:createPassword(90)) - Creating password for container_1485752041236_0002_01_000001 for user container_1485752041236_0002_01_000001 (auth:SIMPLE) to be run on NM 127.0.0.1:1234 2017-01-30 10:24:01,739 DEBUG [AsyncDispatcher event handler] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(248)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 2 cluster_timestamp: 1485752041236 } attemptId: 1 } id: 1 } nmHostAddr: "127.0.0.1:1234" appSubmitter: "user" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1485752641738 masterKeyId: 313252934 rmIdentifier: 1485752041236 priority { priority: 0 } creationTime: 1485752041733 nodeLabelExpression: "" containerType: APPLICATION_MASTER executionType: GUARANTEED version: 0 2017-01-30 10:24:01,739 DEBUG [AsyncDispatcher event handler] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(248)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 2 cluster_timestamp: 1485752041236 } attemptId: 1 } id: 1 } nmHostAddr: "127.0.0.1:1234" appSubmitter: "user" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1485752641738 masterKeyId: 313252934 rmIdentifier: 1485752041236 priority { priority: 0 } creationTime: 1485752041733 nodeLabelExpression: "" containerType: APPLICATION_MASTER executionType: GUARANTEED version: 0 2017-01-30 10:24:01,740 INFO [AsyncDispatcher event handler] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:createAndGetNMToken(200)) - Sending NMToken for nodeId : 127.0.0.1:1234 for container : container_1485752041236_0002_01_000001 2017-01-30 10:24:01,740 DEBUG [AsyncDispatcher event handler] security.BaseNMTokenSecretManager (BaseNMTokenSecretManager.java:createPassword(75)) - creating password for appattempt_1485752041236_0002_000001 for user user to run on NM 127.0.0.1:1234 2017-01-30 10:24:01,740 DEBUG [AsyncDispatcher event handler] security.NMTokenIdentifier (NMTokenIdentifier.java:write(96)) - Writing NMTokenIdentifier to RPC layer: appAttemptId { application_id { id: 2 cluster_timestamp: 1485752041236 } attemptId: 1 } nodeId { host: "127.0.0.1" port: 1234 } appSubmitter: "user" keyId: -671765034 2017-01-30 10:24:01,741 DEBUG [AsyncDispatcher event handler] security.NMTokenIdentifier (NMTokenIdentifier.java:write(96)) - Writing NMTokenIdentifier to RPC layer: appAttemptId { application_id { id: 2 cluster_timestamp: 1485752041236 } attemptId: 1 } nodeId { host: "127.0.0.1" port: 1234 } appSubmitter: "user" keyId: -671765034 2017-01-30 10:24:01,741 DEBUG [AsyncDispatcher event handler] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(476)) - Processing container_1485752041236_0002_01_000001 of type ACQUIRED 2017-01-30 10:24:01,741 INFO [AsyncDispatcher event handler] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(490)) - container_1485752041236_0002_01_000001 Container Transitioned from ALLOCATED to ACQUIRED 2017-01-30 10:24:01,742 DEBUG [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1498)) - User limit computation for user in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: consumedRatio: 0.0 currentCapacity: activeUsers: 0 clusterCapacity: resourceByLabel: usageratio: 0.13544974 Partition: 2017-01-30 10:24:01,742 INFO [AsyncDispatcher event handler] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:clearNodeSetForAttempt(146)) - Clear node set for appattempt_1485752041236_0002_000001 2017-01-30 10:24:01,742 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:storeAttempt(2134)) - Storing attempt: AppId: application_1485752041236_0002 AttemptId: appattempt_1485752041236_0002_000001 MasterContainer: Container: [ContainerId: container_1485752041236_0002_01_000001, AllocationRequestId: 0, Version: 0, NodeId: 127.0.0.1:1234, NodeHttpAddress: 127.0.0.1:2, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 127.0.0.1:1234 }, ExecutionType: GUARANTEED, ] 2017-01-30 10:24:01,742 DEBUG [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:handleStoreEvent(1094)) - Processing event of type STORE_APP_ATTEMPT 2017-01-30 10:24:01,742 DEBUG [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:transition(315)) - Storing info for attempt: appattempt_1485752041236_0002_000001 2017-01-30 10:24:01,743 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(880)) - appattempt_1485752041236_0002_000001 State change from SCHEDULED to ALLOCATED_SAVING on event = CONTAINER_ALLOCATED 2017-01-30 10:24:01,743 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppRunningOnNodeEvent.EventType: APP_RUNNING_ON_NODE 2017-01-30 10:24:01,743 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(901)) - Processing event for application_1485752041236_0002 of type APP_RUNNING_ON_NODE 2017-01-30 10:24:01,743 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: ATTEMPT_NEW_SAVED 2017-01-30 10:24:01,743 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(864)) - Processing event for appattempt_1485752041236_0002_000001 of type ATTEMPT_NEW_SAVED 2017-01-30 10:24:01,743 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(880)) - appattempt_1485752041236_0002_000001 State change from ALLOCATED_SAVING to ALLOCATED on event = ATTEMPT_NEW_SAVED 2017-01-30 10:24:01,743 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncherEvent.EventType: LAUNCH 2017-01-30 10:24:01,747 INFO [Thread-6] resourcemanager.MockRM (MockRM.java:waitForState(360)) - Attempt State is : ALLOCATED 2017-01-30 10:24:01,747 INFO [Thread-6] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:createAndGetAMRMToken(195)) - Create AMRMToken for ApplicationAttempt: appattempt_1485752041236_0002_000001 2017-01-30 10:24:01,748 INFO [Thread-6] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:createPassword(307)) - Creating password for appattempt_1485752041236_0002_000001 2017-01-30 10:24:01,748 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: LAUNCHED 2017-01-30 10:24:01,748 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(864)) - Processing event for appattempt_1485752041236_0002_000001 of type LAUNCHED 2017-01-30 10:24:01,748 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(880)) - appattempt_1485752041236_0002_000001 State change from ALLOCATED to LAUNCHED on event = LAUNCHED 2017-01-30 10:24:01,748 INFO [Thread-6] resourcemanager.MockRM (MockRM.java:waitForState(360)) - Attempt State is : LAUNCHED 2017-01-30 10:24:01,748 INFO [Thread-6] resourcemanager.MockRM (MockRM.java:waitForState(360)) - Attempt State is : LAUNCHED 2017-01-30 10:24:01,749 DEBUG [Thread-6] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1919)) - PrivilegedAction as:appattempt_1485752041236_0002_000001 (auth:SIMPLE) from:org.apache.hadoop.yarn.server.resourcemanager.MockAM.registerAppAttempt(MockAM.java:117) 2017-01-30 10:24:01,749 INFO [Thread-6] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:registerApplicationMaster(245)) - AM registration appattempt_1485752041236_0002_000001 2017-01-30 10:24:01,749 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptRegistrationEvent.EventType: REGISTERED 2017-01-30 10:24:01,749 INFO [Thread-6] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(186)) - USER=user OPERATION=Register App Master TARGET=ApplicationMasterService RESULT=SUCCESS APPID=application_1485752041236_0002 APPATTEMPTID=appattempt_1485752041236_0002_000001 2017-01-30 10:24:01,749 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(864)) - Processing event for appattempt_1485752041236_0002_000001 of type REGISTERED 2017-01-30 10:24:01,749 INFO [Thread-6] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:registerApplicationMaster(264)) - Setting client token master key 2017-01-30 10:24:01,750 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(880)) - appattempt_1485752041236_0002_000001 State change from LAUNCHED to RUNNING on event = REGISTERED 2017-01-30 10:24:01,750 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEvent.EventType: ATTEMPT_REGISTERED 2017-01-30 10:24:01,750 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(901)) - Processing event for application_1485752041236_0002 of type ATTEMPT_REGISTERED 2017-01-30 10:24:01,750 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(917)) - application_1485752041236_0002 State change from ACCEPTED to RUNNING on event = ATTEMPT_REGISTERED 2017-01-30 10:24:01,750 INFO [Thread-6] resourcemanager.MockRM (MockRM.java:waitForState(286)) - App State is : RUNNING 2017-01-30 10:24:01,750 INFO [Thread-6] resourcemanager.MockRM (MockRM.java:waitForState(286)) - App State is : RUNNING 2017-01-30 10:24:01,754 INFO [Thread-6] resourcemanager.MockRM (MockRM.java:waitForState(360)) - Attempt State is : RUNNING 2017-01-30 10:24:01,755 DEBUG [Thread-6] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1919)) - PrivilegedAction as:appattempt_1485752041236_0002_000001 (auth:SIMPLE) from:org.apache.hadoop.yarn.server.resourcemanager.MockAM.unregisterAppAttempt(MockAM.java:319) 2017-01-30 10:24:01,755 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptUnregistrationEvent.EventType: UNREGISTERED 2017-01-30 10:24:01,755 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(864)) - Processing event for appattempt_1485752041236_0002_000001 of type UNREGISTERED 2017-01-30 10:24:01,756 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:rememberTargetTransitionsAndStoreState(1328)) - Updating application attempt appattempt_1485752041236_0002_000001 with final state: FINISHING, and exit status: -1000 2017-01-30 10:24:01,756 DEBUG [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:handleStoreEvent(1094)) - Processing event of type UPDATE_APP_ATTEMPT 2017-01-30 10:24:01,757 DEBUG [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:transition(346)) - Updating info for attempt: appattempt_1485752041236_0002_000001 2017-01-30 10:24:01,757 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(880)) - appattempt_1485752041236_0002_000001 State change from RUNNING to FINAL_SAVING on event = UNREGISTERED 2017-01-30 10:24:01,757 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: ATTEMPT_UPDATE_SAVED 2017-01-30 10:24:01,757 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(864)) - Processing event for appattempt_1485752041236_0002_000001 of type ATTEMPT_UPDATE_SAVED 2017-01-30 10:24:01,757 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(880)) - appattempt_1485752041236_0002_000001 State change from FINAL_SAVING to FINISHING on event = ATTEMPT_UPDATE_SAVED 2017-01-30 10:24:01,757 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEvent.EventType: ATTEMPT_UNREGISTERED 2017-01-30 10:24:01,758 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(901)) - Processing event for application_1485752041236_0002 of type ATTEMPT_UNREGISTERED 2017-01-30 10:24:01,758 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:rememberTargetTransitionsAndStoreState(1303)) - Updating application application_1485752041236_0002 with final state: FINISHING 2017-01-30 10:24:01,758 DEBUG [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:handleStoreEvent(1094)) - Processing event of type UPDATE_APP 2017-01-30 10:24:01,758 INFO [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:transition(248)) - Updating info for app: application_1485752041236_0002 2017-01-30 10:24:01,758 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(917)) - application_1485752041236_0002 State change from RUNNING to FINAL_SAVING on event = ATTEMPT_UNREGISTERED 2017-01-30 10:24:01,758 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEvent.EventType: APP_UPDATE_SAVED 2017-01-30 10:24:01,758 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(901)) - Processing event for application_1485752041236_0002 of type APP_UPDATE_SAVED 2017-01-30 10:24:01,758 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(917)) - application_1485752041236_0002 State change from FINAL_SAVING to FINISHING on event = APP_UPDATE_SAVED 2017-01-30 10:24:01,758 INFO [Thread-6] resourcemanager.MockRM (MockRM.java:waitForState(360)) - Attempt State is : FINISHING 2017-01-30 10:24:01,767 DEBUG [Thread-6] util.log (Log.java:initialized(180)) - Logging to org.slf4j.impl.Log4jLoggerAdapter(org.eclipse.jetty.util.log) via org.eclipse.jetty.util.log.Slf4jLog 2017-01-30 10:24:01,768 INFO [Thread-6] util.log (Log.java:initialized(192)) - Logging initialized @2146ms 2017-01-30 10:24:01,769 INFO [Thread-6] util.log (MockNM.java:nodeHeartbeat(178)) - ContainerStatus: ContainerStatus: [ContainerId: container_1485752041236_0002_01_000001, ExecutionType: GUARANTEED, State: COMPLETE, Capability: , Diagnostics: Success, ExitStatus: 0, IP: null, Host: null] 2017-01-30 10:24:01,769 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2017-01-30 10:24:01,769 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(657)) - Processing 127.0.0.1:1234 of type STATUS_UPDATE 2017-01-30 10:24:01,769 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handleContainerStatus(1389)) - Container container_1485752041236_0002_01_000001 is the first container get launched for application application_1485752041236_0002 2017-01-30 10:24:01,770 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppRunningOnNodeEvent.EventType: APP_RUNNING_ON_NODE 2017-01-30 10:24:01,770 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(901)) - Processing event for application_1485752041236_0002 of type APP_RUNNING_ON_NODE 2017-01-30 10:24:01,770 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2017-01-30 10:24:01,770 DEBUG [AsyncDispatcher event handler] scheduler.AbstractYarnScheduler (AbstractYarnScheduler.java:nodeUpdate(1015)) - nodeUpdate: 127.0.0.1:1234 cluster capacity: 2017-01-30 10:24:01,770 DEBUG [AsyncDispatcher event handler] scheduler.AbstractYarnScheduler (AbstractYarnScheduler.java:updateCompletedContainers(949)) - Container FINISHED: container_1485752041236_0002_01_000001 2017-01-30 10:24:01,771 DEBUG [AsyncDispatcher event handler] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(476)) - Processing container_1485752041236_0002_01_000001 of type FINISHED 2017-01-30 10:24:01,771 INFO [AsyncDispatcher event handler] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(490)) - container_1485752041236_0002_01_000001 Container Transitioned from ACQUIRED to COMPLETED 2017-01-30 10:24:01,771 INFO [AsyncDispatcher event handler] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(165)) - USER=user OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1485752041236_0002 CONTAINERID=container_1485752041236_0002_01_000001 RESOURCE= 2017-01-30 10:24:01,771 DEBUG [AsyncDispatcher event handler] scheduler.SchedulerNode (SchedulerNode.java:releaseContainer(294)) - Released container container_1485752041236_0002_01_000001 of capacity on host 127.0.0.1:1234, which currently has 1 containers, used and available, release resources=true 2017-01-30 10:24:01,772 DEBUG [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1498)) - User limit computation for user in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: consumedRatio: 0.0 currentCapacity: activeUsers: 0 clusterCapacity: resourceByLabel: usageratio: 0.06772487 Partition: 2017-01-30 10:24:01,772 DEBUG [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:releaseResource(1831)) - default used= numContainers=1 user=user user-resources= 2017-01-30 10:24:01,772 DEBUG [AsyncDispatcher event handler] capacity.ParentQueue (ParentQueue.java:internalReleaseResource(784)) - completedContainer root: numChildQueue= 1, capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.06772487, numApps=2, numContainers=1, cluster= 2017-01-30 10:24:01,772 DEBUG [AsyncDispatcher event handler] scheduler.AbstractYarnScheduler (AbstractYarnScheduler.java:nodeUpdate(1048)) - Node being looked for scheduling 127.0.0.1:1234 availableResource: 2017-01-30 10:24:01,773 DEBUG [AsyncDispatcher event handler] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainerOnSingleNode(1283)) - Trying to schedule on node: 127.0.0.1, available: 2017-01-30 10:24:01,773 DEBUG [AsyncDispatcher event handler] capacity.ParentQueue (ParentQueue.java:assignContainers(499)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2017-01-30 10:24:01,773 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptContainerFinishedEvent.EventType: CONTAINER_FINISHED 2017-01-30 10:24:01,773 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(864)) - Processing event for appattempt_1485752041236_0002_000001 of type CONTAINER_FINISHED 2017-01-30 10:24:01,773 INFO [AsyncDispatcher event handler] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:unregisterAttempt(782)) - Unregistering app attempt : appattempt_1485752041236_0002_000001 2017-01-30 10:24:01,774 INFO [AsyncDispatcher event handler] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:applicationMasterFinished(124)) - Application finished, removing password for appattempt_1485752041236_0002_000001 2017-01-30 10:24:01,774 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(880)) - appattempt_1485752041236_0002_000001 State change from FINISHING to FINISHED on event = CONTAINER_FINISHED 2017-01-30 10:24:01,774 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEvent.EventType: ATTEMPT_FINISHED 2017-01-30 10:24:01,774 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(901)) - Processing event for application_1485752041236_0002 of type ATTEMPT_FINISHED 2017-01-30 10:24:01,775 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(917)) - application_1485752041236_0002 State change from FINISHING to FINISHED on event = ATTEMPT_FINISHED 2017-01-30 10:24:01,776 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.AppAttemptRemovedSchedulerEvent.EventType: APP_ATTEMPT_REMOVED 2017-01-30 10:24:01,776 INFO [AsyncDispatcher event handler] capacity.CapacityScheduler (CapacityScheduler.java:doneApplicationAttempt(824)) - Application Attempt appattempt_1485752041236_0002_000001 is done. finalState=FINISHED 2017-01-30 10:24:01,776 INFO [AsyncDispatcher event handler] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:clearRequests(158)) - Application application_1485752041236_0002 requests cleared 2017-01-30 10:24:01,776 INFO [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:removeApplicationAttempt(945)) - Application removed - appId: application_1485752041236_0002 user: user queue: default #user-pending-applications: 0 #user-active-applications: 1 #queue-pending-applications: 0 #queue-active-applications: 1 2017-01-30 10:24:01,776 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncherEvent.EventType: CLEANUP 2017-01-30 10:24:01,776 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2017-01-30 10:24:01,776 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(657)) - Processing 127.0.0.1:1234 of type FINISHED_CONTAINERS_PULLED_BY_AM 2017-01-30 10:24:01,777 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeCleanAppEvent.EventType: CLEANUP_APP 2017-01-30 10:24:01,777 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(657)) - Processing 127.0.0.1:1234 of type CLEANUP_APP 2017-01-30 10:24:01,777 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.AppRemovedSchedulerEvent.EventType: APP_REMOVED 2017-01-30 10:24:01,777 INFO [AsyncDispatcher event handler] capacity.ParentQueue (ParentQueue.java:removeApplication(455)) - Application removed - appId: application_1485752041236_0002 user: user leaf-queue of parent: root #applications: 1 2017-01-30 10:24:01,777 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.RMAppManagerEvent.EventType: APP_COMPLETED 2017-01-30 10:24:01,777 DEBUG [AsyncDispatcher event handler] resourcemanager.RMAppManager (RMAppManager.java:handle(539)) - RMAppManager processing event for application_1485752041236_0002 of type APP_COMPLETED 2017-01-30 10:24:01,778 INFO [AsyncDispatcher event handler] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(242)) - USER=user OPERATION=Application Finished - Succeeded TARGET=RMAppManager RESULT=SUCCESS APPID=application_1485752041236_0002 2017-01-30 10:24:01,778 DEBUG [AsyncDispatcher event handler] scheduler.AbstractYarnScheduler (AbstractYarnScheduler.java:getAppResourceUsageReport(343)) - Request for appInfo of unknown attempt appattempt_1485752041236_0002_000001 2017-01-30 10:24:01,779 INFO [AsyncDispatcher event handler] resourcemanager.RMAppManager$ApplicationSummary (RMAppManager.java:logAppSummary(199)) - appId=application_1485752041236_0002,name=name,user=user,queue=default,state=FINISHED,trackingUrl=http://bibinpc:8088/proxy/application_1485752041236_0002/,appMasterHost=,startTime=1485752041717,finishTime=1485752041758,finalStatus=SUCCEEDED,memorySeconds=38,vcoreSeconds=0,preemptedMemorySeconds=0,preemptedVcoreSeconds=0,preemptedAMContainers=0,preemptedNonAMContainers=0,preemptedResources=,applicationType=YARN 2017-01-30 10:24:01,780 INFO [Thread-6] resourcemanager.MockRM (MockRM.java:waitForState(360)) - Attempt State is : FINISHED 2017-01-30 10:24:01,780 INFO [Thread-6] resourcemanager.MockRM (MockRM.java:waitForState(286)) - App State is : FINISHED 2017-01-30 10:24:01,780 DEBUG [DelegationTokenRenewer #2] security.DelegationTokenRenewer (DelegationTokenRenewer.java:removeApplicationFromRenewal(804)) - Removing delegation token for appId=application_1485752041236_0002; token=service1 2017-01-30 10:24:01,781 INFO [Thread-6] resourcemanager.ClientRMService (ClientRMService.java:getNewApplicationId(320)) - Allocated new applicationId: 3 2017-01-30 10:24:01,782 DEBUG [Thread-6] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1919)) - PrivilegedAction as:user (auth:SIMPLE) from:org.apache.hadoop.yarn.server.resourcemanager.MockRM.submitApp(MockRM.java:765) 2017-01-30 10:24:01,782 INFO [Thread-6] capacity.CapacityScheduler (CapacityScheduler.java:checkAndGetApplicationPriority(2205)) - Application 'application_1485752041236_0003' is submitted without priority hence considering default queue/cluster priority: 0 2017-01-30 10:24:01,782 INFO [Thread-6] capacity.CapacityScheduler (CapacityScheduler.java:checkAndGetApplicationPriority(2226)) - Priority '0' is acceptable in queue : default for application: application_1485752041236_0003 2017-01-30 10:24:01,782 INFO [Thread-6] resourcemanager.ClientRMService (ClientRMService.java:submitApplication(662)) - Application with id 3 submitted by user user 2017-01-30 10:24:01,783 DEBUG [DelegationTokenRenewer #3] security.DelegationTokenRenewer (DelegationTokenRenewer.java:handleAppSubmitEvent(432)) - Registering tokens for renewal for: appId = application_1485752041236_0003 2017-01-30 10:24:01,783 INFO [Thread-6] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(194)) - USER=user OPERATION=Submit Application Request TARGET=ClientRMService RESULT=SUCCESS APPID=application_1485752041236_0003 2017-01-30 10:24:01,783 INFO [Thread-6] resourcemanager.MockRM (MockRM.java:waitForState(280)) - App : application_1485752041236_0003 State is : NEW Waiting for state : ACCEPTED 2017-01-30 10:24:01,783 INFO [DelegationTokenRenewer #3] security.DelegationTokenRenewer (DelegationTokenRenewer.java:handleAppSubmitEvent(450)) - application_1485752041236_0003 found existing hdfs token Kind: HDFS_DELEGATION_TOKEN, Service: service1, Ident: (token for user: HDFS_DELEGATION_TOKEN owner=user, renewer=renewer1, realUser=user, issueDate=0, maxDate=0, sequenceNumber=0, masterKeyId=0) 2017-01-30 10:24:01,783 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEvent.EventType: START 2017-01-30 10:24:01,783 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(901)) - Processing event for application_1485752041236_0003 of type START 2017-01-30 10:24:01,783 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:transition(1284)) - Storing application with id application_1485752041236_0003 2017-01-30 10:24:01,783 DEBUG [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:handleStoreEvent(1094)) - Processing event of type STORE_APP 2017-01-30 10:24:01,783 INFO [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:transition(217)) - Storing info for app: application_1485752041236_0003 2017-01-30 10:24:01,784 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(917)) - application_1485752041236_0003 State change from NEW to NEW_SAVING on event = START 2017-01-30 10:24:01,784 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEvent.EventType: APP_NEW_SAVED 2017-01-30 10:24:01,784 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(901)) - Processing event for application_1485752041236_0003 of type APP_NEW_SAVED 2017-01-30 10:24:01,784 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(917)) - application_1485752041236_0003 State change from NEW_SAVING to SUBMITTED on event = APP_NEW_SAVED 2017-01-30 10:24:01,784 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.AppAddedSchedulerEvent.EventType: APP_ADDED 2017-01-30 10:24:01,784 INFO [AsyncDispatcher event handler] capacity.ParentQueue (ParentQueue.java:addApplication(427)) - Application added - appId: application_1485752041236_0003 user: user leaf-queue of parent: root #applications: 2 2017-01-30 10:24:01,784 INFO [AsyncDispatcher event handler] capacity.CapacityScheduler (CapacityScheduler.java:addApplication(729)) - Accepted application application_1485752041236_0003 from user: user, in queue: default 2017-01-30 10:24:01,784 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEvent.EventType: APP_ACCEPTED 2017-01-30 10:24:01,784 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(901)) - Processing event for application_1485752041236_0003 of type APP_ACCEPTED 2017-01-30 10:24:01,785 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(917)) - application_1485752041236_0003 State change from SUBMITTED to ACCEPTED on event = APP_ACCEPTED 2017-01-30 10:24:01,785 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppStartAttemptEvent.EventType: START 2017-01-30 10:24:01,785 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(864)) - Processing event for appattempt_1485752041236_0003_000001 of type START 2017-01-30 10:24:01,785 INFO [AsyncDispatcher event handler] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:registerAppAttempt(776)) - Registering app attempt : appattempt_1485752041236_0003_000001 2017-01-30 10:24:01,785 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(880)) - appattempt_1485752041236_0003_000001 State change from NEW to SUBMITTED on event = START 2017-01-30 10:24:01,785 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.AppAttemptAddedSchedulerEvent.EventType: APP_ATTEMPT_ADDED 2017-01-30 10:24:01,785 DEBUG [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:activateApplications(777)) - application application_1485752041236_0003 AMResource maxAMResourcePerQueuePercent 0.1 amLimit lastClusterResource amIfStarted AM node-partition name 2017-01-30 10:24:01,786 INFO [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:activateApplications(853)) - Application application_1485752041236_0003 from user: user activated in queue: default 2017-01-30 10:24:01,786 INFO [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:addApplicationAttempt(883)) - Application added - appId: application_1485752041236_0003 user: user, leaf-queue: default #user-pending-applications: 0 #user-active-applications: 2 #queue-pending-applications: 0 #queue-active-applications: 2 2017-01-30 10:24:01,786 INFO [AsyncDispatcher event handler] capacity.CapacityScheduler (CapacityScheduler.java:addApplicationAttempt(775)) - Added Application Attempt appattempt_1485752041236_0003_000001 to scheduler from user user in queue default 2017-01-30 10:24:01,786 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: ATTEMPT_ADDED 2017-01-30 10:24:01,786 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(864)) - Processing event for appattempt_1485752041236_0003_000001 of type ATTEMPT_ADDED 2017-01-30 10:24:01,786 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:transition(1065)) - Using blacklist for AM: additions([]) and removals([]) 2017-01-30 10:24:01,786 DEBUG [AsyncDispatcher event handler] capacity.CapacityScheduler (CapacityScheduler.java:allocate(956)) - allocate: pre-update appattempt_1485752041236_0003_000001 ask size =1 2017-01-30 10:24:01,786 DEBUG [AsyncDispatcher event handler] scheduler.ActiveUsersManager (ActiveUsersManager.java:activateApplication(68)) - User user added to activeUsers, currently: 1 2017-01-30 10:24:01,787 DEBUG [AsyncDispatcher event handler] capacity.CapacityScheduler (CapacityScheduler.java:allocate(969)) - allocate: post-update 2017-01-30 10:24:01,787 DEBUG [AsyncDispatcher event handler] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(620)) - showRequests: application=application_1485752041236_0003 headRoom= currentConsumption=0 2017-01-30 10:24:01,787 DEBUG [AsyncDispatcher event handler] placement.LocalitySchedulingPlacementSet (LocalitySchedulingPlacementSet.java:showRequests(369)) - Request={AllocationRequestId: 0, Priority: 0, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: {Execution Type: GUARANTEED, Enforce Execution Type: false}, Node Label Expression: } 2017-01-30 10:24:01,787 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(880)) - appattempt_1485752041236_0003_000001 State change from SUBMITTED to SCHEDULED on event = ATTEMPT_ADDED 2017-01-30 10:24:01,793 INFO [Thread-6] resourcemanager.MockRM (MockRM.java:waitForState(286)) - App State is : ACCEPTED 2017-01-30 10:24:01,793 INFO [Thread-6] resourcemanager.MockRM (MockRM.java:waitForState(360)) - Attempt State is : SCHEDULED 2017-01-30 10:24:01,793 INFO [Thread-6] resourcemanager.MockRM (MockRM.java:waitForState(286)) - App State is : ACCEPTED 2017-01-30 10:24:01,793 INFO [Thread-6] resourcemanager.MockRM (MockRM.java:waitForState(360)) - Attempt State is : SCHEDULED 2017-01-30 10:24:01,793 INFO [Thread-6] resourcemanager.MockRM (MockRM.java:launchAM(1110)) - Launch AM appattempt_1485752041236_0003_000001 2017-01-30 10:24:01,794 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2017-01-30 10:24:01,794 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(657)) - Processing 127.0.0.1:1234 of type STATUS_UPDATE 2017-01-30 10:24:01,794 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2017-01-30 10:24:01,794 DEBUG [AsyncDispatcher event handler] scheduler.AbstractYarnScheduler (AbstractYarnScheduler.java:nodeUpdate(1015)) - nodeUpdate: 127.0.0.1:1234 cluster capacity: 2017-01-30 10:24:01,794 DEBUG [AsyncDispatcher event handler] scheduler.AbstractYarnScheduler (AbstractYarnScheduler.java:nodeUpdate(1048)) - Node being looked for scheduling 127.0.0.1:1234 availableResource: 2017-01-30 10:24:01,794 DEBUG [AsyncDispatcher event handler] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainerOnSingleNode(1283)) - Trying to schedule on node: 127.0.0.1, available: 2017-01-30 10:24:01,794 DEBUG [AsyncDispatcher event handler] capacity.ParentQueue (ParentQueue.java:assignContainers(521)) - Trying to assign containers to child-queue of root 2017-01-30 10:24:01,794 DEBUG [AsyncDispatcher event handler] capacity.AbstractCSQueue (AbstractCSQueue.java:canAssignToThisQueue(665)) - Check assign to queue: root nodePartition: , usedResources: , clusterResources: , currentUsedCapacity: 0.06772487, max-capacity: 1.0 2017-01-30 10:24:01,794 DEBUG [AsyncDispatcher event handler] capacity.ParentQueue (ParentQueue.java:printChildQueues(770)) - printChildQueues - queue: root child-queues: root.defaultusedCapacity=(0.06772487), label=(*) 2017-01-30 10:24:01,795 DEBUG [AsyncDispatcher event handler] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(712)) - Trying to assign to queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.06772487, absoluteUsedCapacity=0.06772487, numApps=2, numContainers=1 2017-01-30 10:24:01,795 DEBUG [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:assignContainers(1003)) - assignContainers: partition= #applications=2 2017-01-30 10:24:01,795 DEBUG [AsyncDispatcher event handler] capacity.AbstractCSQueue (AbstractCSQueue.java:canAssignToThisQueue(665)) - Check assign to queue: default nodePartition: , usedResources: , clusterResources: , currentUsedCapacity: 0.06772487, max-capacity: 1.0 2017-01-30 10:24:01,795 DEBUG [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1498)) - User limit computation for user in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: consumedRatio: 0.0 currentCapacity: activeUsers: 1 clusterCapacity: resourceByLabel: usageratio: 0.06772487 Partition: 2017-01-30 10:24:01,795 DEBUG [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:computeUserLimitAndSetHeadroom(1370)) - Headroom calculation for user user: userLimit= queueMaxAvailRes= consumed= headroom= 2017-01-30 10:24:01,795 DEBUG [AsyncDispatcher event handler] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:assignContainers(955)) - pre-assignContainers for application application_1485752041236_0001 2017-01-30 10:24:01,795 DEBUG [AsyncDispatcher event handler] allocator.IncreaseContainerAllocator (IncreaseContainerAllocator.java:assignContainers(163)) - Skip allocating increase request since we don't have any increase request on this node=127.0.0.1:1234 2017-01-30 10:24:01,795 DEBUG [AsyncDispatcher event handler] allocator.RegularContainerAllocator (RegularContainerAllocator.java:assignContainers(827)) - Skip app_attempt=appattempt_1485752041236_0001_000001, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-label= 2017-01-30 10:24:01,795 DEBUG [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:assignContainers(1081)) - post-assignContainers for application application_1485752041236_0001 2017-01-30 10:24:01,795 DEBUG [AsyncDispatcher event handler] capacity.AbstractCSQueue (AbstractCSQueue.java:canAssignToThisQueue(665)) - Check assign to queue: default nodePartition: , usedResources: , clusterResources: , currentUsedCapacity: 0.06772487, max-capacity: 1.0 2017-01-30 10:24:01,795 DEBUG [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1498)) - User limit computation for user in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: consumedRatio: 0.0 currentCapacity: activeUsers: 1 clusterCapacity: resourceByLabel: usageratio: 0.06772487 Partition: 2017-01-30 10:24:01,796 DEBUG [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:computeUserLimitAndSetHeadroom(1370)) - Headroom calculation for user user: userLimit= queueMaxAvailRes= consumed= headroom= 2017-01-30 10:24:01,796 DEBUG [AsyncDispatcher event handler] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:assignContainers(955)) - pre-assignContainers for application application_1485752041236_0003 2017-01-30 10:24:01,796 DEBUG [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1498)) - User limit computation for user in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: consumedRatio: 0.0 currentCapacity: activeUsers: 1 clusterCapacity: resourceByLabel: usageratio: 0.06772487 Partition: 2017-01-30 10:24:01,796 DEBUG [AsyncDispatcher event handler] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(620)) - showRequests: application=application_1485752041236_0003 headRoom= currentConsumption=0 2017-01-30 10:24:01,796 DEBUG [AsyncDispatcher event handler] placement.LocalitySchedulingPlacementSet (LocalitySchedulingPlacementSet.java:showRequests(369)) - Request={AllocationRequestId: 0, Priority: 0, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: {Execution Type: GUARANTEED, Enforce Execution Type: false}, Node Label Expression: } 2017-01-30 10:24:01,796 DEBUG [AsyncDispatcher event handler] allocator.IncreaseContainerAllocator (IncreaseContainerAllocator.java:assignContainers(163)) - Skip allocating increase request since we don't have any increase request on this node=127.0.0.1:1234 2017-01-30 10:24:01,796 DEBUG [AsyncDispatcher event handler] allocator.RegularContainerAllocator (RegularContainerAllocator.java:assignContainer(478)) - assignContainers: node=127.0.0.1 application=application_1485752041236_0003 priority=0 pendingAsk=,repeat=1> type=OFF_SWITCH 2017-01-30 10:24:01,796 INFO [AsyncDispatcher event handler] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(127)) - assignedContainer application attempt=appattempt_1485752041236_0003_000001 container=null queue=default clusterResource= type=OFF_SWITCH 2017-01-30 10:24:01,796 DEBUG [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:assignContainers(1081)) - post-assignContainers for application application_1485752041236_0003 2017-01-30 10:24:01,796 DEBUG [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1498)) - User limit computation for user in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: consumedRatio: 0.0 currentCapacity: activeUsers: 1 clusterCapacity: resourceByLabel: usageratio: 0.06772487 Partition: 2017-01-30 10:24:01,797 DEBUG [AsyncDispatcher event handler] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(620)) - showRequests: application=application_1485752041236_0003 headRoom= currentConsumption=0 2017-01-30 10:24:01,797 DEBUG [AsyncDispatcher event handler] placement.LocalitySchedulingPlacementSet (LocalitySchedulingPlacementSet.java:showRequests(369)) - Request={AllocationRequestId: 0, Priority: 0, Capability: , # Containers: 1, Location: *, Relax Locality: true, Execution Type Request: {Execution Type: GUARANTEED, Enforce Execution Type: false}, Node Label Expression: } 2017-01-30 10:24:01,797 DEBUG [AsyncDispatcher event handler] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(724)) - Assigned to queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.06772487, absoluteUsedCapacity=0.06772487, numApps=2, numContainers=1 --> , OFF_SWITCH 2017-01-30 10:24:01,797 INFO [AsyncDispatcher event handler] capacity.ParentQueue (ParentQueue.java:assignContainers(603)) - assignedContainer queue=root usedCapacity=0.06772487 absoluteUsedCapacity=0.06772487 used= cluster= 2017-01-30 10:24:01,797 DEBUG [AsyncDispatcher event handler] capacity.ParentQueue (ParentQueue.java:assignContainers(609)) - ParentQ=root assignedSoFarInThisIteration= usedCapacity=0.06772487 absoluteUsedCapacity=0.06772487 2017-01-30 10:24:01,797 DEBUG [AsyncDispatcher event handler] capacity.CapacityScheduler (CapacityScheduler.java:tryCommit(2473)) - Try to commit allocation proposal=New org.apache.hadoop.yarn.server.resourcemanager.scheduler.common.ResourceCommitRequest: ALLOCATED=[(Application=appattempt_1485752041236_0003_000001; Node=127.0.0.1:1234; Resource=)] 2017-01-30 10:24:01,797 DEBUG [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1498)) - User limit computation for user in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: consumedRatio: 0.0 currentCapacity: activeUsers: 1 clusterCapacity: resourceByLabel: usageratio: 0.06772487 Partition: 2017-01-30 10:24:01,797 DEBUG [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:computeUserLimitAndSetHeadroom(1370)) - Headroom calculation for user user: userLimit= queueMaxAvailRes= consumed= headroom= 2017-01-30 10:24:01,797 DEBUG [AsyncDispatcher event handler] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:updateMetricsForAllocatedContainer(880)) - allocate: applicationId=application_1485752041236_0003 container=container_1485752041236_0003_01_000001 host=127.0.0.1:1234 user=user resource= type=OFF_SWITCH 2017-01-30 10:24:01,797 DEBUG [AsyncDispatcher event handler] scheduler.ActiveUsersManager (ActiveUsersManager.java:deactivateApplication(94)) - User user removed from activeUsers, currently: 0 2017-01-30 10:24:01,798 DEBUG [AsyncDispatcher event handler] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(476)) - Processing container_1485752041236_0003_01_000001 of type START 2017-01-30 10:24:01,798 INFO [AsyncDispatcher event handler] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(490)) - container_1485752041236_0003_01_000001 Container Transitioned from NEW to ALLOCATED 2017-01-30 10:24:01,798 DEBUG [AsyncDispatcher event handler] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(174)) - Assigned container container_1485752041236_0003_01_000001 of capacity on host 127.0.0.1:1234, which has 2 containers, used and available after allocation 2017-01-30 10:24:01,798 DEBUG [AsyncDispatcher event handler] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:apply(595)) - allocate: applicationAttemptId=appattempt_1485752041236_0003_000001 container=container_1485752041236_0003_01_000001 host=127.0.0.1 type=OFF_SWITCH 2017-01-30 10:24:01,798 INFO [AsyncDispatcher event handler] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(165)) - USER=user OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1485752041236_0003 CONTAINERID=container_1485752041236_0003_01_000001 RESOURCE= 2017-01-30 10:24:01,798 DEBUG [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1498)) - User limit computation for user in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: consumedRatio: 0.0 currentCapacity: activeUsers: 0 clusterCapacity: resourceByLabel: usageratio: 0.13544974 Partition: 2017-01-30 10:24:01,798 DEBUG [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1498)) - User limit computation for user in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: consumedRatio: 0.0 currentCapacity: activeUsers: 0 clusterCapacity: resourceByLabel: usageratio: 0.13544974 Partition: 2017-01-30 10:24:01,798 DEBUG [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1498)) - User limit computation for user in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: consumedRatio: 0.0 currentCapacity: activeUsers: 0 clusterCapacity: resourceByLabel: usageratio: 0.13544974 Partition: 2017-01-30 10:24:01,799 DEBUG [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:allocateResource(1785)) - default user=user used= numContainers=2 headroom = user-resources= 2017-01-30 10:24:01,799 INFO [AsyncDispatcher event handler] capacity.ParentQueue (ParentQueue.java:apply(1060)) - assignedContainer queue=root usedCapacity=0.13544974 absoluteUsedCapacity=0.13544974 used= cluster= 2017-01-30 10:24:01,799 INFO [AsyncDispatcher event handler] capacity.CapacityScheduler (CapacityScheduler.java:tryCommit(2481)) - Allocation proposal accepted 2017-01-30 10:24:01,799 DEBUG [AsyncDispatcher event handler] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1185)) - Assigned maximum number of off-switch containers: 1, assignments so far: resource:; type:OFF_SWITCH; excessReservation:null; applicationid:null; skipped:NONE; fulfilled reservation:false; allocations(count/resource):1/; reservations(count/resource):0/ 2017-01-30 10:24:01,799 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: CONTAINER_ALLOCATED 2017-01-30 10:24:01,799 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(864)) - Processing event for appattempt_1485752041236_0003_000001 of type CONTAINER_ALLOCATED 2017-01-30 10:24:01,799 DEBUG [AsyncDispatcher event handler] security.BaseContainerTokenSecretManager (BaseContainerTokenSecretManager.java:createPassword(90)) - Creating password for container_1485752041236_0003_01_000001 for user container_1485752041236_0003_01_000001 (auth:SIMPLE) to be run on NM 127.0.0.1:1234 2017-01-30 10:24:01,799 DEBUG [AsyncDispatcher event handler] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(248)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 3 cluster_timestamp: 1485752041236 } attemptId: 1 } id: 1 } nmHostAddr: "127.0.0.1:1234" appSubmitter: "user" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1485752641799 masterKeyId: 313252934 rmIdentifier: 1485752041236 priority { priority: 0 } creationTime: 1485752041796 nodeLabelExpression: "" containerType: APPLICATION_MASTER executionType: GUARANTEED version: 0 2017-01-30 10:24:01,800 DEBUG [AsyncDispatcher event handler] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(248)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 3 cluster_timestamp: 1485752041236 } attemptId: 1 } id: 1 } nmHostAddr: "127.0.0.1:1234" appSubmitter: "user" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1485752641799 masterKeyId: 313252934 rmIdentifier: 1485752041236 priority { priority: 0 } creationTime: 1485752041796 nodeLabelExpression: "" containerType: APPLICATION_MASTER executionType: GUARANTEED version: 0 2017-01-30 10:24:01,800 INFO [AsyncDispatcher event handler] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:createAndGetNMToken(200)) - Sending NMToken for nodeId : 127.0.0.1:1234 for container : container_1485752041236_0003_01_000001 2017-01-30 10:24:01,800 DEBUG [AsyncDispatcher event handler] security.BaseNMTokenSecretManager (BaseNMTokenSecretManager.java:createPassword(75)) - creating password for appattempt_1485752041236_0003_000001 for user user to run on NM 127.0.0.1:1234 2017-01-30 10:24:01,800 DEBUG [AsyncDispatcher event handler] security.NMTokenIdentifier (NMTokenIdentifier.java:write(96)) - Writing NMTokenIdentifier to RPC layer: appAttemptId { application_id { id: 3 cluster_timestamp: 1485752041236 } attemptId: 1 } nodeId { host: "127.0.0.1" port: 1234 } appSubmitter: "user" keyId: -671765034 2017-01-30 10:24:01,800 DEBUG [AsyncDispatcher event handler] security.NMTokenIdentifier (NMTokenIdentifier.java:write(96)) - Writing NMTokenIdentifier to RPC layer: appAttemptId { application_id { id: 3 cluster_timestamp: 1485752041236 } attemptId: 1 } nodeId { host: "127.0.0.1" port: 1234 } appSubmitter: "user" keyId: -671765034 2017-01-30 10:24:01,801 DEBUG [AsyncDispatcher event handler] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(476)) - Processing container_1485752041236_0003_01_000001 of type ACQUIRED 2017-01-30 10:24:01,801 INFO [AsyncDispatcher event handler] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(490)) - container_1485752041236_0003_01_000001 Container Transitioned from ALLOCATED to ACQUIRED 2017-01-30 10:24:01,801 DEBUG [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1498)) - User limit computation for user in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: consumedRatio: 0.0 currentCapacity: activeUsers: 0 clusterCapacity: resourceByLabel: usageratio: 0.13544974 Partition: 2017-01-30 10:24:01,801 INFO [AsyncDispatcher event handler] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:clearNodeSetForAttempt(146)) - Clear node set for appattempt_1485752041236_0003_000001 2017-01-30 10:24:01,801 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:storeAttempt(2134)) - Storing attempt: AppId: application_1485752041236_0003 AttemptId: appattempt_1485752041236_0003_000001 MasterContainer: Container: [ContainerId: container_1485752041236_0003_01_000001, AllocationRequestId: 0, Version: 0, NodeId: 127.0.0.1:1234, NodeHttpAddress: 127.0.0.1:2, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 127.0.0.1:1234 }, ExecutionType: GUARANTEED, ] 2017-01-30 10:24:01,801 DEBUG [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:handleStoreEvent(1094)) - Processing event of type STORE_APP_ATTEMPT 2017-01-30 10:24:01,801 DEBUG [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:transition(315)) - Storing info for attempt: appattempt_1485752041236_0003_000001 2017-01-30 10:24:01,801 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(880)) - appattempt_1485752041236_0003_000001 State change from SCHEDULED to ALLOCATED_SAVING on event = CONTAINER_ALLOCATED 2017-01-30 10:24:01,801 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppRunningOnNodeEvent.EventType: APP_RUNNING_ON_NODE 2017-01-30 10:24:01,802 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(901)) - Processing event for application_1485752041236_0003 of type APP_RUNNING_ON_NODE 2017-01-30 10:24:01,802 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: ATTEMPT_NEW_SAVED 2017-01-30 10:24:01,802 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(864)) - Processing event for appattempt_1485752041236_0003_000001 of type ATTEMPT_NEW_SAVED 2017-01-30 10:24:01,802 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(880)) - appattempt_1485752041236_0003_000001 State change from ALLOCATED_SAVING to ALLOCATED on event = ATTEMPT_NEW_SAVED 2017-01-30 10:24:01,802 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncherEvent.EventType: LAUNCH 2017-01-30 10:24:01,802 INFO [Thread-6] resourcemanager.MockRM (MockRM.java:waitForState(360)) - Attempt State is : ALLOCATED 2017-01-30 10:24:01,802 INFO [Thread-6] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:createAndGetAMRMToken(195)) - Create AMRMToken for ApplicationAttempt: appattempt_1485752041236_0003_000001 2017-01-30 10:24:01,802 INFO [Thread-6] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:createPassword(307)) - Creating password for appattempt_1485752041236_0003_000001 2017-01-30 10:24:01,802 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: LAUNCHED 2017-01-30 10:24:01,802 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(864)) - Processing event for appattempt_1485752041236_0003_000001 of type LAUNCHED 2017-01-30 10:24:01,803 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(880)) - appattempt_1485752041236_0003_000001 State change from ALLOCATED to LAUNCHED on event = LAUNCHED 2017-01-30 10:24:01,803 INFO [Thread-6] resourcemanager.MockRM (MockRM.java:waitForState(360)) - Attempt State is : LAUNCHED 2017-01-30 10:24:01,803 INFO [Thread-6] resourcemanager.MockRM (MockRM.java:waitForState(360)) - Attempt State is : LAUNCHED 2017-01-30 10:24:01,803 DEBUG [Thread-6] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1919)) - PrivilegedAction as:appattempt_1485752041236_0003_000001 (auth:SIMPLE) from:org.apache.hadoop.yarn.server.resourcemanager.MockAM.registerAppAttempt(MockAM.java:117) 2017-01-30 10:24:01,803 INFO [Thread-6] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:registerApplicationMaster(245)) - AM registration appattempt_1485752041236_0003_000001 2017-01-30 10:24:01,803 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptRegistrationEvent.EventType: REGISTERED 2017-01-30 10:24:01,803 INFO [Thread-6] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(186)) - USER=user OPERATION=Register App Master TARGET=ApplicationMasterService RESULT=SUCCESS APPID=application_1485752041236_0003 APPATTEMPTID=appattempt_1485752041236_0003_000001 2017-01-30 10:24:01,803 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(864)) - Processing event for appattempt_1485752041236_0003_000001 of type REGISTERED 2017-01-30 10:24:01,804 INFO [Thread-6] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:registerApplicationMaster(264)) - Setting client token master key 2017-01-30 10:24:01,804 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(880)) - appattempt_1485752041236_0003_000001 State change from LAUNCHED to RUNNING on event = REGISTERED 2017-01-30 10:24:01,804 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEvent.EventType: ATTEMPT_REGISTERED 2017-01-30 10:24:01,804 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(901)) - Processing event for application_1485752041236_0003 of type ATTEMPT_REGISTERED 2017-01-30 10:24:01,804 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(917)) - application_1485752041236_0003 State change from ACCEPTED to RUNNING on event = ATTEMPT_REGISTERED 2017-01-30 10:24:01,804 INFO [Thread-6] resourcemanager.MockRM (MockRM.java:waitForState(286)) - App State is : RUNNING 2017-01-30 10:24:01,804 INFO [Thread-6] resourcemanager.MockRM (MockRM.java:waitForState(286)) - App State is : RUNNING 2017-01-30 10:24:01,804 INFO [Thread-6] resourcemanager.MockRM (MockRM.java:waitForState(360)) - Attempt State is : RUNNING 2017-01-30 10:24:01,804 DEBUG [Thread-6] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1919)) - PrivilegedAction as:appattempt_1485752041236_0001_000001 (auth:SIMPLE) from:org.apache.hadoop.yarn.server.resourcemanager.MockAM.unregisterAppAttempt(MockAM.java:319) 2017-01-30 10:24:01,804 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptUnregistrationEvent.EventType: UNREGISTERED 2017-01-30 10:24:01,804 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(864)) - Processing event for appattempt_1485752041236_0001_000001 of type UNREGISTERED 2017-01-30 10:24:01,805 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:rememberTargetTransitionsAndStoreState(1328)) - Updating application attempt appattempt_1485752041236_0001_000001 with final state: FINISHING, and exit status: -1000 2017-01-30 10:24:01,805 DEBUG [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:handleStoreEvent(1094)) - Processing event of type UPDATE_APP_ATTEMPT 2017-01-30 10:24:01,805 DEBUG [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:transition(346)) - Updating info for attempt: appattempt_1485752041236_0001_000001 2017-01-30 10:24:01,805 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(880)) - appattempt_1485752041236_0001_000001 State change from RUNNING to FINAL_SAVING on event = UNREGISTERED 2017-01-30 10:24:01,805 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: ATTEMPT_UPDATE_SAVED 2017-01-30 10:24:01,805 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(864)) - Processing event for appattempt_1485752041236_0001_000001 of type ATTEMPT_UPDATE_SAVED 2017-01-30 10:24:01,805 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(880)) - appattempt_1485752041236_0001_000001 State change from FINAL_SAVING to FINISHING on event = ATTEMPT_UPDATE_SAVED 2017-01-30 10:24:01,805 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEvent.EventType: ATTEMPT_UNREGISTERED 2017-01-30 10:24:01,805 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(901)) - Processing event for application_1485752041236_0001 of type ATTEMPT_UNREGISTERED 2017-01-30 10:24:01,806 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:rememberTargetTransitionsAndStoreState(1303)) - Updating application application_1485752041236_0001 with final state: FINISHING 2017-01-30 10:24:01,806 DEBUG [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:handleStoreEvent(1094)) - Processing event of type UPDATE_APP 2017-01-30 10:24:01,806 INFO [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:transition(248)) - Updating info for app: application_1485752041236_0001 2017-01-30 10:24:01,806 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(917)) - application_1485752041236_0001 State change from RUNNING to FINAL_SAVING on event = ATTEMPT_UNREGISTERED 2017-01-30 10:24:01,806 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEvent.EventType: APP_UPDATE_SAVED 2017-01-30 10:24:01,806 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(901)) - Processing event for application_1485752041236_0001 of type APP_UPDATE_SAVED 2017-01-30 10:24:01,806 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(917)) - application_1485752041236_0001 State change from FINAL_SAVING to FINISHING on event = APP_UPDATE_SAVED 2017-01-30 10:24:01,806 INFO [Thread-6] resourcemanager.MockRM (MockRM.java:waitForState(360)) - Attempt State is : FINISHING 2017-01-30 10:24:01,806 INFO [Thread-6] util.log (MockNM.java:nodeHeartbeat(178)) - ContainerStatus: ContainerStatus: [ContainerId: container_1485752041236_0001_01_000001, ExecutionType: GUARANTEED, State: COMPLETE, Capability: , Diagnostics: Success, ExitStatus: 0, IP: null, Host: null] 2017-01-30 10:24:01,807 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2017-01-30 10:24:01,807 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(657)) - Processing 127.0.0.1:1234 of type STATUS_UPDATE 2017-01-30 10:24:01,807 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handleContainerStatus(1389)) - Container container_1485752041236_0001_01_000001 is the first container get launched for application application_1485752041236_0001 2017-01-30 10:24:01,807 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppRunningOnNodeEvent.EventType: APP_RUNNING_ON_NODE 2017-01-30 10:24:01,807 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(901)) - Processing event for application_1485752041236_0001 of type APP_RUNNING_ON_NODE 2017-01-30 10:24:01,807 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2017-01-30 10:24:01,807 DEBUG [AsyncDispatcher event handler] scheduler.AbstractYarnScheduler (AbstractYarnScheduler.java:nodeUpdate(1015)) - nodeUpdate: 127.0.0.1:1234 cluster capacity: 2017-01-30 10:24:01,807 DEBUG [AsyncDispatcher event handler] scheduler.AbstractYarnScheduler (AbstractYarnScheduler.java:updateCompletedContainers(949)) - Container FINISHED: container_1485752041236_0001_01_000001 2017-01-30 10:24:01,808 DEBUG [AsyncDispatcher event handler] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(476)) - Processing container_1485752041236_0001_01_000001 of type FINISHED 2017-01-30 10:24:01,808 INFO [AsyncDispatcher event handler] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(490)) - container_1485752041236_0001_01_000001 Container Transitioned from ACQUIRED to COMPLETED 2017-01-30 10:24:01,808 INFO [AsyncDispatcher event handler] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(165)) - USER=user OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1485752041236_0001 CONTAINERID=container_1485752041236_0001_01_000001 RESOURCE= 2017-01-30 10:24:01,808 DEBUG [AsyncDispatcher event handler] scheduler.SchedulerNode (SchedulerNode.java:releaseContainer(294)) - Released container container_1485752041236_0001_01_000001 of capacity on host 127.0.0.1:1234, which currently has 1 containers, used and available, release resources=true 2017-01-30 10:24:01,808 DEBUG [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1498)) - User limit computation for user in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: consumedRatio: 0.0 currentCapacity: activeUsers: 0 clusterCapacity: resourceByLabel: usageratio: 0.06772487 Partition: 2017-01-30 10:24:01,808 DEBUG [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:releaseResource(1831)) - default used= numContainers=1 user=user user-resources= 2017-01-30 10:24:01,808 DEBUG [AsyncDispatcher event handler] capacity.ParentQueue (ParentQueue.java:internalReleaseResource(784)) - completedContainer root: numChildQueue= 1, capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.06772487, numApps=2, numContainers=1, cluster= 2017-01-30 10:24:01,809 DEBUG [AsyncDispatcher event handler] scheduler.AbstractYarnScheduler (AbstractYarnScheduler.java:nodeUpdate(1048)) - Node being looked for scheduling 127.0.0.1:1234 availableResource: 2017-01-30 10:24:01,809 DEBUG [AsyncDispatcher event handler] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainerOnSingleNode(1283)) - Trying to schedule on node: 127.0.0.1, available: 2017-01-30 10:24:01,809 DEBUG [AsyncDispatcher event handler] capacity.ParentQueue (ParentQueue.java:assignContainers(499)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2017-01-30 10:24:01,809 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptContainerFinishedEvent.EventType: CONTAINER_FINISHED 2017-01-30 10:24:01,809 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(864)) - Processing event for appattempt_1485752041236_0001_000001 of type CONTAINER_FINISHED 2017-01-30 10:24:01,809 INFO [AsyncDispatcher event handler] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:unregisterAttempt(782)) - Unregistering app attempt : appattempt_1485752041236_0001_000001 2017-01-30 10:24:01,809 INFO [AsyncDispatcher event handler] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:applicationMasterFinished(124)) - Application finished, removing password for appattempt_1485752041236_0001_000001 2017-01-30 10:24:01,809 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(880)) - appattempt_1485752041236_0001_000001 State change from FINISHING to FINISHED on event = CONTAINER_FINISHED 2017-01-30 10:24:01,809 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEvent.EventType: ATTEMPT_FINISHED 2017-01-30 10:24:01,809 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(901)) - Processing event for application_1485752041236_0001 of type ATTEMPT_FINISHED 2017-01-30 10:24:01,810 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(917)) - application_1485752041236_0001 State change from FINISHING to FINISHED on event = ATTEMPT_FINISHED 2017-01-30 10:24:01,810 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.AppAttemptRemovedSchedulerEvent.EventType: APP_ATTEMPT_REMOVED 2017-01-30 10:24:01,810 INFO [AsyncDispatcher event handler] capacity.CapacityScheduler (CapacityScheduler.java:doneApplicationAttempt(824)) - Application Attempt appattempt_1485752041236_0001_000001 is done. finalState=FINISHED 2017-01-30 10:24:01,810 INFO [AsyncDispatcher event handler] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:clearRequests(158)) - Application application_1485752041236_0001 requests cleared 2017-01-30 10:24:01,810 INFO [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:removeApplicationAttempt(945)) - Application removed - appId: application_1485752041236_0001 user: user queue: default #user-pending-applications: 0 #user-active-applications: 1 #queue-pending-applications: 0 #queue-active-applications: 1 2017-01-30 10:24:01,810 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncherEvent.EventType: CLEANUP 2017-01-30 10:24:01,810 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2017-01-30 10:24:01,810 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(657)) - Processing 127.0.0.1:1234 of type FINISHED_CONTAINERS_PULLED_BY_AM 2017-01-30 10:24:01,810 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeCleanAppEvent.EventType: CLEANUP_APP 2017-01-30 10:24:01,810 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(657)) - Processing 127.0.0.1:1234 of type CLEANUP_APP 2017-01-30 10:24:01,810 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.AppRemovedSchedulerEvent.EventType: APP_REMOVED 2017-01-30 10:24:01,810 INFO [AsyncDispatcher event handler] capacity.ParentQueue (ParentQueue.java:removeApplication(455)) - Application removed - appId: application_1485752041236_0001 user: user leaf-queue of parent: root #applications: 1 2017-01-30 10:24:01,810 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.RMAppManagerEvent.EventType: APP_COMPLETED 2017-01-30 10:24:01,810 DEBUG [AsyncDispatcher event handler] resourcemanager.RMAppManager (RMAppManager.java:handle(539)) - RMAppManager processing event for application_1485752041236_0001 of type APP_COMPLETED 2017-01-30 10:24:01,811 INFO [AsyncDispatcher event handler] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(242)) - USER=user OPERATION=Application Finished - Succeeded TARGET=RMAppManager RESULT=SUCCESS APPID=application_1485752041236_0001 2017-01-30 10:24:01,811 DEBUG [AsyncDispatcher event handler] scheduler.AbstractYarnScheduler (AbstractYarnScheduler.java:getAppResourceUsageReport(343)) - Request for appInfo of unknown attempt appattempt_1485752041236_0001_000001 2017-01-30 10:24:01,811 INFO [AsyncDispatcher event handler] resourcemanager.RMAppManager$ApplicationSummary (RMAppManager.java:logAppSummary(199)) - appId=application_1485752041236_0001,name=name,user=user,queue=default,state=FINISHED,trackingUrl=http://bibinpc:8088/proxy/application_1485752041236_0001/,appMasterHost=,startTime=1485752041470,finishTime=1485752041806,finalStatus=SUCCEEDED,memorySeconds=200,vcoreSeconds=0,preemptedMemorySeconds=0,preemptedVcoreSeconds=0,preemptedAMContainers=0,preemptedNonAMContainers=0,preemptedResources=,applicationType=YARN 2017-01-30 10:24:01,811 INFO [Thread-6] resourcemanager.MockRM (MockRM.java:waitForState(360)) - Attempt State is : FINISHED 2017-01-30 10:24:01,811 INFO [Thread-6] resourcemanager.MockRM (MockRM.java:waitForState(286)) - App State is : FINISHED 2017-01-30 10:24:01,811 DEBUG [DelegationTokenRenewer #4] security.DelegationTokenRenewer (DelegationTokenRenewer.java:removeApplicationFromRenewal(804)) - Removing delegation token for appId=application_1485752041236_0001; token=service1 2017-01-30 10:24:01,822 INFO [Thread-6] resourcemanager.MockRM (MockRM.java:waitForState(360)) - Attempt State is : RUNNING 2017-01-30 10:24:01,822 DEBUG [Thread-6] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1919)) - PrivilegedAction as:appattempt_1485752041236_0003_000001 (auth:SIMPLE) from:org.apache.hadoop.yarn.server.resourcemanager.MockAM.unregisterAppAttempt(MockAM.java:319) 2017-01-30 10:24:01,822 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptUnregistrationEvent.EventType: UNREGISTERED 2017-01-30 10:24:01,822 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(864)) - Processing event for appattempt_1485752041236_0003_000001 of type UNREGISTERED 2017-01-30 10:24:01,823 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:rememberTargetTransitionsAndStoreState(1328)) - Updating application attempt appattempt_1485752041236_0003_000001 with final state: FINISHING, and exit status: -1000 2017-01-30 10:24:01,827 DEBUG [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:handleStoreEvent(1094)) - Processing event of type UPDATE_APP_ATTEMPT 2017-01-30 10:24:01,827 DEBUG [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:transition(346)) - Updating info for attempt: appattempt_1485752041236_0003_000001 2017-01-30 10:24:01,827 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(880)) - appattempt_1485752041236_0003_000001 State change from RUNNING to FINAL_SAVING on event = UNREGISTERED 2017-01-30 10:24:01,827 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: ATTEMPT_UPDATE_SAVED 2017-01-30 10:24:01,827 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(864)) - Processing event for appattempt_1485752041236_0003_000001 of type ATTEMPT_UPDATE_SAVED 2017-01-30 10:24:01,827 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(880)) - appattempt_1485752041236_0003_000001 State change from FINAL_SAVING to FINISHING on event = ATTEMPT_UPDATE_SAVED 2017-01-30 10:24:01,828 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEvent.EventType: ATTEMPT_UNREGISTERED 2017-01-30 10:24:01,828 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(901)) - Processing event for application_1485752041236_0003 of type ATTEMPT_UNREGISTERED 2017-01-30 10:24:01,828 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:rememberTargetTransitionsAndStoreState(1303)) - Updating application application_1485752041236_0003 with final state: FINISHING 2017-01-30 10:24:01,828 DEBUG [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:handleStoreEvent(1094)) - Processing event of type UPDATE_APP 2017-01-30 10:24:01,828 INFO [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:transition(248)) - Updating info for app: application_1485752041236_0003 2017-01-30 10:24:01,828 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(917)) - application_1485752041236_0003 State change from RUNNING to FINAL_SAVING on event = ATTEMPT_UNREGISTERED 2017-01-30 10:24:01,828 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEvent.EventType: APP_UPDATE_SAVED 2017-01-30 10:24:01,828 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(901)) - Processing event for application_1485752041236_0003 of type APP_UPDATE_SAVED 2017-01-30 10:24:01,829 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(917)) - application_1485752041236_0003 State change from FINAL_SAVING to FINISHING on event = APP_UPDATE_SAVED 2017-01-30 10:24:01,829 INFO [Thread-6] resourcemanager.MockRM (MockRM.java:waitForState(360)) - Attempt State is : FINISHING 2017-01-30 10:24:01,829 INFO [Thread-6] util.log (MockNM.java:nodeHeartbeat(178)) - ContainerStatus: ContainerStatus: [ContainerId: container_1485752041236_0003_01_000001, ExecutionType: GUARANTEED, State: COMPLETE, Capability: , Diagnostics: Success, ExitStatus: 0, IP: null, Host: null] 2017-01-30 10:24:01,829 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2017-01-30 10:24:01,829 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(657)) - Processing 127.0.0.1:1234 of type STATUS_UPDATE 2017-01-30 10:24:01,830 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handleContainerStatus(1389)) - Container container_1485752041236_0003_01_000001 is the first container get launched for application application_1485752041236_0003 2017-01-30 10:24:01,830 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppRunningOnNodeEvent.EventType: APP_RUNNING_ON_NODE 2017-01-30 10:24:01,830 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(901)) - Processing event for application_1485752041236_0003 of type APP_RUNNING_ON_NODE 2017-01-30 10:24:01,830 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2017-01-30 10:24:01,830 DEBUG [AsyncDispatcher event handler] scheduler.AbstractYarnScheduler (AbstractYarnScheduler.java:nodeUpdate(1015)) - nodeUpdate: 127.0.0.1:1234 cluster capacity: 2017-01-30 10:24:01,830 DEBUG [AsyncDispatcher event handler] scheduler.AbstractYarnScheduler (AbstractYarnScheduler.java:updateCompletedContainers(949)) - Container FINISHED: container_1485752041236_0003_01_000001 2017-01-30 10:24:01,830 DEBUG [AsyncDispatcher event handler] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(476)) - Processing container_1485752041236_0003_01_000001 of type FINISHED 2017-01-30 10:24:01,830 INFO [AsyncDispatcher event handler] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(490)) - container_1485752041236_0003_01_000001 Container Transitioned from ACQUIRED to COMPLETED 2017-01-30 10:24:01,831 INFO [AsyncDispatcher event handler] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(165)) - USER=user OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1485752041236_0003 CONTAINERID=container_1485752041236_0003_01_000001 RESOURCE= 2017-01-30 10:24:01,831 DEBUG [AsyncDispatcher event handler] scheduler.SchedulerNode (SchedulerNode.java:releaseContainer(294)) - Released container container_1485752041236_0003_01_000001 of capacity on host 127.0.0.1:1234, which currently has 0 containers, used and available, release resources=true 2017-01-30 10:24:01,831 DEBUG [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1498)) - User limit computation for user in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: consumedRatio: 0.0 currentCapacity: activeUsers: 0 clusterCapacity: resourceByLabel: usageratio: 0.0 Partition: 2017-01-30 10:24:01,831 DEBUG [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:releaseResource(1831)) - default used= numContainers=0 user=user user-resources= 2017-01-30 10:24:01,831 DEBUG [AsyncDispatcher event handler] capacity.ParentQueue (ParentQueue.java:internalReleaseResource(784)) - completedContainer root: numChildQueue= 1, capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.0, numApps=1, numContainers=0, cluster= 2017-01-30 10:24:01,831 DEBUG [AsyncDispatcher event handler] scheduler.AbstractYarnScheduler (AbstractYarnScheduler.java:nodeUpdate(1048)) - Node being looked for scheduling 127.0.0.1:1234 availableResource: 2017-01-30 10:24:01,831 DEBUG [AsyncDispatcher event handler] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainerOnSingleNode(1283)) - Trying to schedule on node: 127.0.0.1, available: 2017-01-30 10:24:01,831 DEBUG [AsyncDispatcher event handler] capacity.ParentQueue (ParentQueue.java:assignContainers(499)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2017-01-30 10:24:01,832 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptContainerFinishedEvent.EventType: CONTAINER_FINISHED 2017-01-30 10:24:01,832 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(864)) - Processing event for appattempt_1485752041236_0003_000001 of type CONTAINER_FINISHED 2017-01-30 10:24:01,832 INFO [AsyncDispatcher event handler] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:unregisterAttempt(782)) - Unregistering app attempt : appattempt_1485752041236_0003_000001 2017-01-30 10:24:01,832 INFO [AsyncDispatcher event handler] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:applicationMasterFinished(124)) - Application finished, removing password for appattempt_1485752041236_0003_000001 2017-01-30 10:24:01,832 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(880)) - appattempt_1485752041236_0003_000001 State change from FINISHING to FINISHED on event = CONTAINER_FINISHED 2017-01-30 10:24:01,832 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEvent.EventType: ATTEMPT_FINISHED 2017-01-30 10:24:01,832 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(901)) - Processing event for application_1485752041236_0003 of type ATTEMPT_FINISHED 2017-01-30 10:24:01,832 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(917)) - application_1485752041236_0003 State change from FINISHING to FINISHED on event = ATTEMPT_FINISHED 2017-01-30 10:24:01,833 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.AppAttemptRemovedSchedulerEvent.EventType: APP_ATTEMPT_REMOVED 2017-01-30 10:24:01,833 INFO [AsyncDispatcher event handler] capacity.CapacityScheduler (CapacityScheduler.java:doneApplicationAttempt(824)) - Application Attempt appattempt_1485752041236_0003_000001 is done. finalState=FINISHED 2017-01-30 10:24:01,833 INFO [AsyncDispatcher event handler] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:clearRequests(158)) - Application application_1485752041236_0003 requests cleared 2017-01-30 10:24:01,833 INFO [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:removeApplicationAttempt(945)) - Application removed - appId: application_1485752041236_0003 user: user queue: default #user-pending-applications: 0 #user-active-applications: 0 #queue-pending-applications: 0 #queue-active-applications: 0 2017-01-30 10:24:01,833 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncherEvent.EventType: CLEANUP 2017-01-30 10:24:01,833 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2017-01-30 10:24:01,833 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(657)) - Processing 127.0.0.1:1234 of type FINISHED_CONTAINERS_PULLED_BY_AM 2017-01-30 10:24:01,833 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeCleanAppEvent.EventType: CLEANUP_APP 2017-01-30 10:24:01,833 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(657)) - Processing 127.0.0.1:1234 of type CLEANUP_APP 2017-01-30 10:24:01,833 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.AppRemovedSchedulerEvent.EventType: APP_REMOVED 2017-01-30 10:24:01,833 INFO [AsyncDispatcher event handler] capacity.ParentQueue (ParentQueue.java:removeApplication(455)) - Application removed - appId: application_1485752041236_0003 user: user leaf-queue of parent: root #applications: 0 2017-01-30 10:24:01,833 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(192)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.RMAppManagerEvent.EventType: APP_COMPLETED 2017-01-30 10:24:01,834 DEBUG [AsyncDispatcher event handler] resourcemanager.RMAppManager (RMAppManager.java:handle(539)) - RMAppManager processing event for application_1485752041236_0003 of type APP_COMPLETED 2017-01-30 10:24:01,834 INFO [AsyncDispatcher event handler] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(242)) - USER=user OPERATION=Application Finished - Succeeded TARGET=RMAppManager RESULT=SUCCESS APPID=application_1485752041236_0003 2017-01-30 10:24:01,834 DEBUG [AsyncDispatcher event handler] scheduler.AbstractYarnScheduler (AbstractYarnScheduler.java:getAppResourceUsageReport(343)) - Request for appInfo of unknown attempt appattempt_1485752041236_0003_000001 2017-01-30 10:24:01,834 DEBUG [DelegationTokenRenewer #5] security.DelegationTokenRenewer (DelegationTokenRenewer.java:removeApplicationFromRenewal(804)) - Removing delegation token for appId=application_1485752041236_0003; token=service1 2017-01-30 10:24:01,834 INFO [AsyncDispatcher event handler] resourcemanager.RMAppManager$ApplicationSummary (RMAppManager.java:logAppSummary(199)) - appId=application_1485752041236_0003,name=name,user=user,queue=default,state=FINISHED,trackingUrl=http://bibinpc:8088/proxy/application_1485752041236_0003/,appMasterHost=,startTime=1485752041782,finishTime=1485752041828,finalStatus=SUCCEEDED,memorySeconds=34,vcoreSeconds=0,preemptedMemorySeconds=0,preemptedVcoreSeconds=0,preemptedAMContainers=0,preemptedNonAMContainers=0,preemptedResources=,applicationType=YARN 2017-01-30 10:24:01,835 INFO [Thread-6] resourcemanager.MockRM (MockRM.java:waitForState(360)) - Attempt State is : FINISHED 2017-01-30 10:24:01,835 DEBUG [Delegation Token Canceler] security.DelegationTokenRenewer (DelegationTokenRenewer.java:run(338)) - Cancelling token service1 2017-01-30 10:24:01,835 INFO [Thread-6] resourcemanager.MockRM (MockRM.java:waitForState(286)) - App State is : FINISHED 2017-01-30 10:24:01,835 DEBUG [Delegation Token Canceler] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1919)) - PrivilegedAction as:root (auth:KERBEROS) from:org.apache.hadoop.yarn.server.resourcemanager.security.DelegationTokenRenewer$DelegationTokenCancelThread.run(DelegationTokenRenewer.java:342) 2017-01-30 10:24:01,835 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.security.DelegationTokenRenewer entered state STOPPED