2016-05-05 17:27:13,302 INFO [Thread-0] event.AsyncDispatcher (AsyncDispatcher.java:register(209)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStoreEventType for class org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$ForwardingEventHandler 2016-05-05 17:27:13,335 INFO [Thread-0] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2434)) - found resource core-site.xml at file:/home/moonfang/opensource/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/target/test-classes/core-site.xml 2016-05-05 17:27:13,383 WARN [Thread-0] util.NativeCodeLoader (NativeCodeLoader.java:(60)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2016-05-05 17:27:13,403 INFO [Thread-0] security.Groups (Groups.java:refresh(271)) - clearing userToGroupsMap cache 2016-05-05 17:27:13,454 INFO [Thread-0] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2434)) - found resource yarn-site.xml at file:/home/moonfang/opensource/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/target/test-classes/yarn-site.xml 2016-05-05 17:27:13,581 INFO [Thread-0] event.AsyncDispatcher (AsyncDispatcher.java:register(209)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.RMFatalEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMFatalEventDispatcher 2016-05-05 17:27:13,697 INFO [Thread-0] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:(75)) - NMTokenKeyRollingInterval: 86400000ms and NMTokenKeyActivationDelay: 900000ms 2016-05-05 17:27:13,699 INFO [Thread-0] security.RMContainerTokenSecretManager (RMContainerTokenSecretManager.java:(78)) - ContainerTokenKeyRollingInterval: 86400000ms and ContainerTokenKeyActivationDelay: 900000ms 2016-05-05 17:27:13,702 INFO [Thread-0] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:(94)) - AMRMTokenKeyRollingInterval: 86400000ms and AMRMTokenKeyActivationDelay: 900000 ms 2016-05-05 17:27:13,712 INFO [Thread-0] security.YarnAuthorizationProvider (YarnAuthorizationProvider.java:getInstance(57)) - org.apache.hadoop.yarn.security.ConfiguredYarnAuthorizer is instiantiated. 2016-05-05 17:27:13,712 INFO [Thread-0] recovery.RMStateStoreFactory (RMStateStoreFactory.java:getStore(33)) - Using RMStateStore implementation - class org.apache.hadoop.yarn.server.resourcemanager.recovery.MemoryRMStateStore 2016-05-05 17:27:13,712 INFO [Thread-0] event.AsyncDispatcher (AsyncDispatcher.java:register(209)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStoreEventType for class org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$ForwardingEventHandler 2016-05-05 17:27:13,713 INFO [Thread-0] event.AsyncDispatcher (AsyncDispatcher.java:register(209)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.NodesListManagerEventType for class org.apache.hadoop.yarn.server.resourcemanager.NodesListManager 2016-05-05 17:27:13,713 INFO [Thread-0] resourcemanager.ResourceManager (ResourceManager.java:createScheduler(382)) - Using Scheduler: org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler 2016-05-05 17:27:13,722 INFO [Thread-0] event.AsyncDispatcher (AsyncDispatcher.java:register(209)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.SchedulerEventType for class org.apache.hadoop.yarn.event.EventDispatcher 2016-05-05 17:27:13,722 INFO [Thread-0] event.AsyncDispatcher (AsyncDispatcher.java:register(209)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationEventDispatcher 2016-05-05 17:27:13,724 INFO [Thread-0] event.AsyncDispatcher (AsyncDispatcher.java:register(209)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationAttemptEventDispatcher 2016-05-05 17:27:13,724 INFO [Thread-0] event.AsyncDispatcher (AsyncDispatcher.java:register(209)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$NodeEventDispatcher 2016-05-05 17:27:13,724 INFO [Thread-0] security.RMContainerTokenSecretManager (RMContainerTokenSecretManager.java:rollMasterKey(106)) - Rolling master-key for container-tokens 2016-05-05 17:27:13,729 INFO [Thread-0] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:rollMasterKey(95)) - Rolling master-key for nm-tokens 2016-05-05 17:27:13,750 WARN [Thread-0] impl.MetricsConfig (MetricsConfig.java:loadFirst(127)) - Cannot locate configuration: tried hadoop-metrics2-resourcemanager.properties,hadoop-metrics2.properties 2016-05-05 17:27:13,785 INFO [Thread-0] impl.MetricsSystemImpl (MetricsSystemImpl.java:startTimer(374)) - Scheduled snapshot period at 10 second(s). 2016-05-05 17:27:13,785 INFO [Thread-0] impl.MetricsSystemImpl (MetricsSystemImpl.java:start(192)) - ResourceManager metrics system started 2016-05-05 17:27:13,793 INFO [Thread-0] event.AsyncDispatcher (AsyncDispatcher.java:register(209)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.RMAppManagerEventType for class org.apache.hadoop.yarn.server.resourcemanager.RMAppManager 2016-05-05 17:27:13,798 INFO [Thread-0] event.AsyncDispatcher (AsyncDispatcher.java:register(209)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncherEventType for class org.apache.hadoop.yarn.server.resourcemanager.MockRM$6 2016-05-05 17:27:13,799 INFO [Thread-0] resourcemanager.RMNMInfo (RMNMInfo.java:(63)) - Registered RMNMInfo MBean 2016-05-05 17:27:13,800 INFO [Thread-0] util.HostsFileReader (HostsFileReader.java:refresh(132)) - Refreshing hosts (include/exclude) list 2016-05-05 17:27:13,803 INFO [Thread-0] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2434)) - found resource capacity-scheduler.xml at file:/home/moonfang/opensource/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/target/test-classes/capacity-scheduler.xml 2016-05-05 17:27:13,835 INFO [Thread-0] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(692)) - max alloc mb per queue for root is undefined 2016-05-05 17:27:13,835 INFO [Thread-0] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(696)) - max alloc vcore per queue for root is undefined 2016-05-05 17:27:13,841 INFO [Thread-0] capacity.ParentQueue (ParentQueue.java:setupQueueConfigs(128)) - root, capacity=1.0, asboluteCapacity=1.0, maxCapacity=1.0, asboluteMaxCapacity=1.0, state=RUNNING, acls=ADMINISTER_QUEUE:*SUBMIT_APP:*, labels=*, , reservationsContinueLooking=true 2016-05-05 17:27:13,841 INFO [Thread-0] capacity.ParentQueue (ParentQueue.java:(107)) - Initialized parent-queue root name=root, fullname=root 2016-05-05 17:27:13,850 INFO [Thread-0] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(692)) - max alloc mb per queue for root.default is undefined 2016-05-05 17:27:13,850 INFO [Thread-0] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(696)) - max alloc vcore per queue for root.default is undefined 2016-05-05 17:27:13,852 INFO [Thread-0] capacity.LeafQueue (LeafQueue.java:setupQueueConfigs(223)) - Initializing default capacity = 1.0 [= (float) configuredCapacity / 100 ] asboluteCapacity = 1.0 [= parentAbsoluteCapacity * capacity ] maxCapacity = 1.0 [= configuredMaxCapacity ] absoluteMaxCapacity = 1.0 [= 1.0 maximumCapacity undefined, (parentAbsoluteMaxCapacity * maximumCapacity) / 100 otherwise ] userLimit = 100 [= configuredUserLimit ] userLimitFactor = 1.0 [= configuredUserLimitFactor ] maxApplications = 10000 [= configuredMaximumSystemApplicationsPerQueue or (int)(configuredMaximumSystemApplications * absoluteCapacity)] maxApplicationsPerUser = 10000 [= (int)(maxApplications * (userLimit / 100.0f) * userLimitFactor) ] usedCapacity = 0.0 [= usedResourcesMemory / (clusterResourceMemory * absoluteCapacity)] absoluteUsedCapacity = 0.0 [= usedResourcesMemory / clusterResourceMemory] maxAMResourcePerQueuePercent = 0.1 [= configuredMaximumAMResourcePercent ] minimumAllocationFactor = 0.875 [= (float)(maximumAllocationMemory - minimumAllocationMemory) / maximumAllocationMemory ] maximumAllocation = [= configuredMaxAllocation ] numContainers = 0 [= currentNumContainers ] state = RUNNING [= configuredState ] acls = ADMINISTER_QUEUE:*SUBMIT_APP:* [= configuredAcls ] nodeLocalityDelay = 40 labels=*, reservationsContinueLooking = true preemptionDisabled = true defaultAppPriorityPerQueue = 0 2016-05-05 17:27:13,852 INFO [Thread-0] capacity.CapacityScheduler (CapacityScheduler.java:parseQueue(677)) - Initialized queue: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=0, numContainers=0 2016-05-05 17:27:13,852 INFO [Thread-0] capacity.CapacityScheduler (CapacityScheduler.java:parseQueue(677)) - Initialized queue: root: numChildQueue= 1, capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.0, numApps=0, numContainers=0 2016-05-05 17:27:13,853 INFO [Thread-0] capacity.CapacityScheduler (CapacityScheduler.java:initializeQueues(512)) - Initialized root queue root: numChildQueue= 1, capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.0, numApps=0, numContainers=0 2016-05-05 17:27:13,853 INFO [Thread-0] capacity.CapacityScheduler (CapacityScheduler.java:getUserGroupMappingPlacementRule(460)) - Initialized queue mappings, override: false 2016-05-05 17:27:13,854 INFO [Thread-0] capacity.CapacityScheduler (CapacityScheduler.java:initScheduler(322)) - Initialized CapacityScheduler with calculator=class org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator, minimumAllocation=<>, maximumAllocation=<>, asynchronousScheduling=false, asyncScheduleInterval=5ms 2016-05-05 17:27:13,856 INFO [Thread-0] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2431)) - dynamic-resources.xml not found 2016-05-05 17:27:13,863 INFO [Thread-0] metrics.SystemMetricsPublisher (SystemMetricsPublisher.java:serviceInit(95)) - YARN system metrics publishing service is not enabled 2016-05-05 17:27:13,863 INFO [Thread-0] resourcemanager.ResourceManager (ResourceManager.java:transitionToActive(1044)) - Transitioning to active state 2016-05-05 17:27:13,865 DEBUG [Thread-0] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1780)) - PrivilegedAction as:moonfang (auth:SIMPLE) from:org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.transitionToActive(ResourceManager.java:1046) 2016-05-05 17:27:13,869 DEBUG [Thread-0] service.AbstractService (AbstractService.java:start(197)) - Service Dispatcher is started 2016-05-05 17:27:13,869 DEBUG [Thread-0] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore is started 2016-05-05 17:27:13,869 INFO [Thread-0] resourcemanager.ResourceManager (ResourceManager.java:serviceStart(670)) - Recovery started 2016-05-05 17:27:13,869 INFO [Thread-0] security.RMDelegationTokenSecretManager (RMDelegationTokenSecretManager.java:recover(178)) - recovering RMDelegationTokenSecretManager. 2016-05-05 17:27:13,869 INFO [Thread-0] resourcemanager.RMAppManager (RMAppManager.java:recover(475)) - Recovering 0 applications 2016-05-05 17:27:13,869 INFO [Thread-0] resourcemanager.ResourceManager (ResourceManager.java:serviceStart(677)) - Recovery ended 2016-05-05 17:27:13,869 DEBUG [Thread-0] service.CompositeService (CompositeService.java:serviceStart(115)) - RMActiveServices: starting services, size=14 2016-05-05 17:27:13,872 DEBUG [Thread-0] recovery.RMStateStore (RMStateStore.java:handleStoreEvent(1070)) - Processing event of type UPDATE_AMRM_TOKEN 2016-05-05 17:27:13,872 INFO [Thread-0] recovery.RMStateStore (RMStateStore.java:transition(487)) - Updating AMRMToken 2016-05-05 17:27:13,872 INFO [Thread-0] security.RMContainerTokenSecretManager (RMContainerTokenSecretManager.java:rollMasterKey(106)) - Rolling master-key for container-tokens 2016-05-05 17:27:13,872 INFO [Thread-0] security.RMContainerTokenSecretManager (RMContainerTokenSecretManager.java:rollMasterKey(111)) - Going to activate master-key with key-id -1802453346 in 900000ms 2016-05-05 17:27:13,872 INFO [Thread-0] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:rollMasterKey(95)) - Rolling master-key for nm-tokens 2016-05-05 17:27:13,872 INFO [Thread-0] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:rollMasterKey(100)) - Going to activate master-key with key-id -790324191 in 900000ms 2016-05-05 17:27:13,873 INFO [Thread-0] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2016-05-05 17:27:13,873 INFO [Thread-0] security.RMDelegationTokenSecretManager (RMDelegationTokenSecretManager.java:storeNewMasterKey(87)) - storing master key with keyID 1 2016-05-05 17:27:13,873 DEBUG [Thread-0] recovery.RMStateStore (RMStateStore.java:handleStoreEvent(1070)) - Processing event of type STORE_MASTERKEY 2016-05-05 17:27:13,873 INFO [Thread-0] recovery.RMStateStore (RMStateStore.java:transition(438)) - Storing RMDTMasterKey. 2016-05-05 17:27:13,873 INFO [Thread-0] recovery.RMStateStore (MemoryRMStateStore.java:storeRMDTMasterKeyState(229)) - Store RMDT master key with key id: 1. Currently rmDTMasterKeyState size: 1 2016-05-05 17:27:13,874 DEBUG [Thread-0] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.RMSecretManagerService is started 2016-05-05 17:27:13,874 INFO [Thread[Thread-11,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2016-05-05 17:27:13,874 INFO [Thread[Thread-11,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2016-05-05 17:27:13,874 INFO [Thread[Thread-11,5,main]] security.RMDelegationTokenSecretManager (RMDelegationTokenSecretManager.java:storeNewMasterKey(87)) - storing master key with keyID 2 2016-05-05 17:27:13,875 DEBUG [Thread[Thread-11,5,main]] recovery.RMStateStore (RMStateStore.java:handleStoreEvent(1070)) - Processing event of type STORE_MASTERKEY 2016-05-05 17:27:13,875 INFO [Thread[Thread-11,5,main]] recovery.RMStateStore (RMStateStore.java:transition(438)) - Storing RMDTMasterKey. 2016-05-05 17:27:13,875 INFO [Thread[Thread-11,5,main]] recovery.RMStateStore (MemoryRMStateStore.java:storeRMDTMasterKeyState(229)) - Store RMDT master key with key id: 2. Currently rmDTMasterKeyState size: 2 2016-05-05 17:27:13,875 DEBUG [Thread-0] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.ContainerAllocationExpirer is started 2016-05-05 17:27:13,875 DEBUG [Thread-0] service.AbstractService (AbstractService.java:start(197)) - Service AMLivelinessMonitor is started 2016-05-05 17:27:13,875 DEBUG [Thread-0] service.AbstractService (AbstractService.java:start(197)) - Service AMLivelinessMonitor is started 2016-05-05 17:27:13,875 DEBUG [Thread-0] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.nodelabels.CommonNodeLabelsManager is started 2016-05-05 17:27:13,876 DEBUG [Thread-0] service.CompositeService (CompositeService.java:serviceStart(115)) - org.apache.hadoop.yarn.server.resourcemanager.NodesListManager: starting services, size=0 2016-05-05 17:27:13,876 DEBUG [Thread-0] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.NodesListManager is started 2016-05-05 17:27:13,876 DEBUG [Thread-0] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler is started 2016-05-05 17:27:13,876 DEBUG [Thread-0] service.AbstractService (AbstractService.java:start(197)) - Service SchedulerEventDispatcher is started 2016-05-05 17:27:13,876 DEBUG [Thread-0] service.AbstractService (AbstractService.java:start(197)) - Service NMLivelinessMonitor is started 2016-05-05 17:27:13,876 DEBUG [Thread-0] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.ResourceTrackerService is started 2016-05-05 17:27:13,877 DEBUG [Thread-0] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.util.JvmPauseMonitor is started 2016-05-05 17:27:13,877 DEBUG [Thread-0] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService is started 2016-05-05 17:27:13,877 DEBUG [Thread-0] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.ClientRMService is started 2016-05-05 17:27:13,877 DEBUG [Thread-0] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher is started 2016-05-05 17:27:13,877 DEBUG [Thread-0] service.AbstractService (AbstractService.java:start(197)) - Service RMActiveServices is started 2016-05-05 17:27:13,877 INFO [Thread-0] resourcemanager.ResourceManager (ResourceManager.java:transitionToActive(1060)) - Transitioned to active state 2016-05-05 17:27:13,877 DEBUG [Thread-0] service.CompositeService (CompositeService.java:serviceStart(115)) - ResourceManager: starting services, size=4 2016-05-05 17:27:13,877 DEBUG [Thread-0] service.AbstractService (AbstractService.java:start(197)) - Service Dispatcher is started 2016-05-05 17:27:13,877 INFO [org.apache.hadoop.util.JvmPauseMonitor$Monitor@5e29e202] util.JvmPauseMonitor (JvmPauseMonitor.java:run(188)) - Starting JVM pause monitor 2016-05-05 17:27:13,877 DEBUG [Thread-0] service.CompositeService (CompositeService.java:serviceStart(115)) - org.apache.hadoop.yarn.server.resourcemanager.AdminService: starting services, size=0 2016-05-05 17:27:13,878 DEBUG [Thread-0] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.AdminService is started 2016-05-05 17:27:13,878 DEBUG [Thread-0] service.CompositeService (CompositeService.java:serviceStart(115)) - org.apache.hadoop.yarn.server.resourcemanager.ahs.RMApplicationHistoryWriter: starting services, size=0 2016-05-05 17:27:13,878 DEBUG [Thread-0] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.ahs.RMApplicationHistoryWriter is started 2016-05-05 17:27:13,878 DEBUG [Thread-0] service.CompositeService (CompositeService.java:serviceStart(115)) - org.apache.hadoop.yarn.server.resourcemanager.metrics.SystemMetricsPublisher: starting services, size=0 2016-05-05 17:27:13,878 DEBUG [Thread-0] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.metrics.SystemMetricsPublisher is started 2016-05-05 17:27:13,878 DEBUG [Thread-0] service.AbstractService (AbstractService.java:start(197)) - Service ResourceManager is started 2016-05-05 17:27:13,911 DEBUG [Thread-0] util.RackResolver (RackResolver.java:coreResolve(112)) - Resolved 127.0.0.1 to /default-rack 2016-05-05 17:27:13,920 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStartedEvent.EventType: STARTED 2016-05-05 17:27:13,920 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(618)) - Processing 127.0.0.1:1234 of type STARTED 2016-05-05 17:27:13,920 INFO [Thread-0] resourcemanager.ResourceTrackerService (ResourceTrackerService.java:registerNodeManager(428)) - NodeManager from node 127.0.0.1(cmPort: 1234 httpPort: 2) registered with capability: , assigned nodeId 127.0.0.1:1234 2016-05-05 17:27:13,921 DEBUG [AsyncDispatcher event handler] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.resourcemanager.ClusterMetrics.numActiveNMs with annotation @org.apache.hadoop.metrics2.annotation.Metric(value=[# of active NMs], about=, valueName=Time, type=DEFAULT, always=false, sampleName=Ops) 2016-05-05 17:27:13,921 DEBUG [AsyncDispatcher event handler] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.resourcemanager.ClusterMetrics.numDecommissioningNMs with annotation @org.apache.hadoop.metrics2.annotation.Metric(value=[# of decommissioning NMs], about=, valueName=Time, type=DEFAULT, always=false, sampleName=Ops) 2016-05-05 17:27:13,921 DEBUG [AsyncDispatcher event handler] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.resourcemanager.ClusterMetrics.numDecommissionedNMs with annotation @org.apache.hadoop.metrics2.annotation.Metric(value=[# of decommissioned NMs], about=, valueName=Time, type=DEFAULT, always=false, sampleName=Ops) 2016-05-05 17:27:13,922 DEBUG [AsyncDispatcher event handler] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.resourcemanager.ClusterMetrics.numLostNMs with annotation @org.apache.hadoop.metrics2.annotation.Metric(value=[# of lost NMs], about=, valueName=Time, type=DEFAULT, always=false, sampleName=Ops) 2016-05-05 17:27:13,922 DEBUG [AsyncDispatcher event handler] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.resourcemanager.ClusterMetrics.numUnhealthyNMs with annotation @org.apache.hadoop.metrics2.annotation.Metric(value=[# of unhealthy NMs], about=, valueName=Time, type=DEFAULT, always=false, sampleName=Ops) 2016-05-05 17:27:13,922 DEBUG [AsyncDispatcher event handler] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.resourcemanager.ClusterMetrics.numRebootedNMs with annotation @org.apache.hadoop.metrics2.annotation.Metric(value=[# of Rebooted NMs], about=, valueName=Time, type=DEFAULT, always=false, sampleName=Ops) 2016-05-05 17:27:13,922 DEBUG [AsyncDispatcher event handler] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.resourcemanager.ClusterMetrics.numShutdownNMs with annotation @org.apache.hadoop.metrics2.annotation.Metric(value=[# of Shutdown NMs], about=, valueName=Time, type=DEFAULT, always=false, sampleName=Ops) 2016-05-05 17:27:13,922 DEBUG [AsyncDispatcher event handler] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.yarn.server.resourcemanager.ClusterMetrics.aMLaunchDelay with annotation @org.apache.hadoop.metrics2.annotation.Metric(value=[AM container launch delay], about=, valueName=Time, type=DEFAULT, always=false, sampleName=Ops) 2016-05-05 17:27:13,923 DEBUG [AsyncDispatcher event handler] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.yarn.server.resourcemanager.ClusterMetrics.aMRegisterDelay with annotation @org.apache.hadoop.metrics2.annotation.Metric(value=[AM register delay], about=, valueName=Time, type=DEFAULT, always=false, sampleName=Ops) 2016-05-05 17:27:13,923 DEBUG [AsyncDispatcher event handler] impl.MetricsSystemImpl (MetricsSystemImpl.java:register(232)) - ClusterMetrics, Metrics for the Yarn Cluster 2016-05-05 17:27:13,923 DEBUG [AsyncDispatcher event handler] impl.MetricsConfig (MetricsConfig.java:getProperty(182)) - poking parent 'PropertiesConfiguration' for key: source.source.start_mbeans 2016-05-05 17:27:13,923 DEBUG [AsyncDispatcher event handler] impl.MetricsConfig (MetricsConfig.java:getProperty(182)) - poking parent 'MetricsConfig' for key: source.start_mbeans 2016-05-05 17:27:13,923 DEBUG [AsyncDispatcher event handler] impl.MetricsConfig (MetricsConfig.java:getProperty(182)) - poking parent 'PropertiesConfiguration' for key: *.source.start_mbeans 2016-05-05 17:27:13,924 DEBUG [AsyncDispatcher event handler] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateAttrCache(256)) - Updating attr cache... 2016-05-05 17:27:13,924 DEBUG [AsyncDispatcher event handler] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateAttrCache(270)) - Done. # tags & metrics=14 2016-05-05 17:27:13,924 DEBUG [AsyncDispatcher event handler] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateInfoCache(250)) - Updating info cache... 2016-05-05 17:27:13,924 DEBUG [AsyncDispatcher event handler] impl.MetricsSystemImpl (MBeanInfoBuilder.java:get(109)) - [javax.management.MBeanAttributeInfo[description=Metrics for the Yarn Cluster, name=tag.ClusterMetrics, type=java.lang.String, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Metrics context, name=tag.Context, type=java.lang.String, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Local hostname, name=tag.Hostname, type=java.lang.String, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=# of active NMs, name=NumActiveNMs, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=# of decommissioning NMs, name=NumDecommissioningNMs, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=# of decommissioned NMs, name=NumDecommissionedNMs, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=# of lost NMs, name=NumLostNMs, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=# of unhealthy NMs, name=NumUnhealthyNMs, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=# of Rebooted NMs, name=NumRebootedNMs, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=# of Shutdown NMs, name=NumShutdownNMs, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of ops for aM container launch delay, name=AMLaunchDelayNumOps, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Average time for aM container launch delay, name=AMLaunchDelayAvgTime, type=java.lang.Double, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of ops for aM register delay, name=AMRegisterDelayNumOps, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Average time for aM register delay, name=AMRegisterDelayAvgTime, type=java.lang.Double, read-only, descriptor={}]] 2016-05-05 17:27:13,924 DEBUG [AsyncDispatcher event handler] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateInfoCache(252)) - Done 2016-05-05 17:27:13,924 DEBUG [AsyncDispatcher event handler] util.MBeans (MBeans.java:register(68)) - Registered Hadoop:service=ResourceManager,name=ClusterMetrics 2016-05-05 17:27:13,924 DEBUG [AsyncDispatcher event handler] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:startMBeans(229)) - MBean for source ClusterMetrics registered. 2016-05-05 17:27:13,924 DEBUG [AsyncDispatcher event handler] impl.MetricsSystemImpl (MetricsSystemImpl.java:registerSource(269)) - Registered source ClusterMetrics 2016-05-05 17:27:13,925 INFO [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(630)) - 127.0.0.1:1234 Node Transitioned from NEW to RUNNING 2016-05-05 17:27:13,925 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeAddedSchedulerEvent.EventType: NODE_ADDED 2016-05-05 17:27:13,925 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.NodesListManagerEvent.EventType: NODE_USABLE 2016-05-05 17:27:13,926 DEBUG [AsyncDispatcher event handler] resourcemanager.NodesListManager (NodesListManager.java:handle(315)) - 127.0.0.1:1234 reported usable 2016-05-05 17:27:13,927 INFO [Thread-0] resourcemanager.ClientRMService (ClientRMService.java:getNewApplicationId(317)) - Allocated new applicationId: 1 2016-05-05 17:27:13,930 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeLabelsUpdateSchedulerEvent.EventType: NODE_LABELS_UPDATE 2016-05-05 17:27:13,930 INFO [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:addNode(1490)) - Added node 127.0.0.1:1234 clusterResource: 2016-05-05 17:27:13,971 DEBUG [Thread-0] util.PerformanceAdvisory (JniBasedUnixGroupsMappingWithFallback.java:(41)) - Falling back to shell based 2016-05-05 17:27:13,971 DEBUG [Thread-0] security.JniBasedUnixGroupsMappingWithFallback (JniBasedUnixGroupsMappingWithFallback.java:(45)) - Group mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping 2016-05-05 17:27:13,972 DEBUG [Thread-0] security.Groups (Groups.java:(114)) - Group mapping impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback; cacheTimeout=300000; warningDeltaMs=5000 2016-05-05 17:27:13,972 DEBUG [Thread-0] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1780)) - PrivilegedAction as:moonfang (auth:SIMPLE) from:org.apache.hadoop.yarn.server.resourcemanager.MockRM.submitApp(MockRM.java:645) 2016-05-05 17:27:13,993 INFO [Thread-0] capacity.CapacityScheduler (CapacityScheduler.java:checkAndGetApplicationPriority(2051)) - Priority '0' is acceptable in queue : default for application: application_1462440433865_0001 for the user: moonfang 2016-05-05 17:27:14,000 INFO [Thread-0] rmapp.RMAppImpl (RMAppImpl.java:(439)) - The attemptFailuresValidityInterval for the application: application_1462440433865_0001 is 60000. 2016-05-05 17:27:14,005 INFO [Thread-0] resourcemanager.ClientRMService (ClientRMService.java:submitApplication(621)) - Application with id 1 submitted by user moonfang 2016-05-05 17:27:14,005 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEvent.EventType: START 2016-05-05 17:27:14,005 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(811)) - Processing event for application_1462440433865_0001 of type START 2016-05-05 17:27:14,005 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:transition(1127)) - Storing application with id application_1462440433865_0001 2016-05-05 17:27:14,007 INFO [Thread-0] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(183)) - USER=moonfang OPERATION=Submit Application Request TARGET=ClientRMService RESULT=SUCCESS APPID=application_1462440433865_0001 2016-05-05 17:27:14,010 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStoreAppEvent.EventType: STORE_APP 2016-05-05 17:27:14,010 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(823)) - application_1462440433865_0001 State change from NEW to NEW_SAVING 2016-05-05 17:27:14,010 DEBUG [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:handleStoreEvent(1070)) - Processing event of type STORE_APP 2016-05-05 17:27:14,011 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(186)) - App : application_1462440433865_0001 State is : NEW_SAVING Waiting for state : ACCEPTED 2016-05-05 17:27:14,011 INFO [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:transition(213)) - Storing info for app: application_1462440433865_0001 2016-05-05 17:27:14,011 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEvent.EventType: APP_NEW_SAVED 2016-05-05 17:27:14,011 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(811)) - Processing event for application_1462440433865_0001 of type APP_NEW_SAVED 2016-05-05 17:27:14,011 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(823)) - application_1462440433865_0001 State change from NEW_SAVING to SUBMITTED 2016-05-05 17:27:14,011 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.AppAddedSchedulerEvent.EventType: APP_ADDED 2016-05-05 17:27:14,012 INFO [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:addApplication(352)) - Application added - appId: application_1462440433865_0001 user: moonfang leaf-queue of parent: root #applications: 1 2016-05-05 17:27:14,012 INFO [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:addApplication(785)) - Accepted application application_1462440433865_0001 from user: moonfang, in queue: default 2016-05-05 17:27:14,013 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEvent.EventType: APP_ACCEPTED 2016-05-05 17:27:14,013 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(811)) - Processing event for application_1462440433865_0001 of type APP_ACCEPTED 2016-05-05 17:27:14,018 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(823)) - application_1462440433865_0001 State change from SUBMITTED to ACCEPTED 2016-05-05 17:27:14,019 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppStartAttemptEvent.EventType: START 2016-05-05 17:27:14,019 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(859)) - Processing event for appattempt_1462440433865_0001_000001 of type START 2016-05-05 17:27:14,021 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(192)) - App State is : ACCEPTED 2016-05-05 17:27:14,033 INFO [AsyncDispatcher event handler] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:registerAppAttempt(676)) - Registering app attempt : appattempt_1462440433865_0001_000001 2016-05-05 17:27:14,033 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(871)) - appattempt_1462440433865_0001_000001 State change from NEW to SUBMITTED 2016-05-05 17:27:14,034 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.AppAttemptAddedSchedulerEvent.EventType: APP_ATTEMPT_ADDED 2016-05-05 17:27:14,034 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(259)) - AppAttempt : appattempt_1462440433865_0001_000001 State is : SUBMITTED Waiting for state : SCHEDULED 2016-05-05 17:27:14,044 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(259)) - AppAttempt : appattempt_1462440433865_0001_000001 State is : SUBMITTED Waiting for state : SCHEDULED 2016-05-05 17:27:14,049 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:activateApplications(652)) - application application_1462440433865_0001 AMResource maxAMResourcePerQueuePercent 0.1 amLimit lastClusterResource amIfStarted AM node-partition name 2016-05-05 17:27:14,050 INFO [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:activateApplications(726)) - Application application_1462440433865_0001 from user: moonfang activated in queue: default 2016-05-05 17:27:14,050 INFO [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:addApplicationAttempt(741)) - Application added - appId: application_1462440433865_0001 user: moonfang, leaf-queue: default #user-pending-applications: 0 #user-active-applications: 1 #queue-pending-applications: 0 #queue-active-applications: 1 2016-05-05 17:27:14,051 INFO [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:addApplicationAttempt(825)) - Added Application Attempt appattempt_1462440433865_0001_000001 to scheduler from user moonfang in queue default 2016-05-05 17:27:14,051 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: ATTEMPT_ADDED 2016-05-05 17:27:14,051 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(859)) - Processing event for appattempt_1462440433865_0001_000001 of type ATTEMPT_ADDED 2016-05-05 17:27:14,051 DEBUG [AsyncDispatcher event handler] blacklist.SimpleBlacklistManager (SimpleBlacklistManager.java:getBlacklistUpdates(69)) - blacklist size 0 is less than failure threshold ratio 0.800000011920929 out of total usable nodes 1 2016-05-05 17:27:14,051 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:transition(1041)) - Using blacklist for AM: additions([]) and removals([]) 2016-05-05 17:27:14,051 DEBUG [AsyncDispatcher event handler] capacity.CapacityScheduler (CapacityScheduler.java:allocate(984)) - allocate: pre-update appattempt_1462440433865_0001_000001 ask size =1 2016-05-05 17:27:14,052 DEBUG [AsyncDispatcher event handler] scheduler.ActiveUsersManager (ActiveUsersManager.java:activateApplication(68)) - User moonfang added to activeUsers, currently: 1 2016-05-05 17:27:14,052 DEBUG [AsyncDispatcher event handler] capacity.CapacityScheduler (CapacityScheduler.java:allocate(996)) - allocate: post-update 2016-05-05 17:27:14,052 DEBUG [AsyncDispatcher event handler] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(480)) - showRequests: application=application_1462440433865_0001 headRoom= currentConsumption=0 2016-05-05 17:27:14,052 DEBUG [AsyncDispatcher event handler] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(484)) - showRequests: application=application_1462440433865_0001 request={Priority: 0, Capability: , # Containers: 1, Location: *, Relax Locality: true, Node Label Expression: } 2016-05-05 17:27:14,053 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(871)) - appattempt_1462440433865_0001_000001 State change from SUBMITTED to SCHEDULED 2016-05-05 17:27:14,054 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(265)) - Attempt State is : SCHEDULED 2016-05-05 17:27:14,054 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(192)) - App State is : ACCEPTED 2016-05-05 17:27:14,054 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(265)) - Attempt State is : SCHEDULED Launch AM appattempt_1462440433865_0001_000001 2016-05-05 17:27:14,068 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2016-05-05 17:27:14,068 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(618)) - Processing 127.0.0.1:1234 of type STATUS_UPDATE 2016-05-05 17:27:14,068 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(259)) - AppAttempt : appattempt_1462440433865_0001_000001 State is : SCHEDULED Waiting for state : ALLOCATED 2016-05-05 17:27:14,069 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2016-05-05 17:27:14,069 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1052)) - nodeUpdate: 127.0.0.1:1234 clusterResources: 2016-05-05 17:27:14,069 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1125)) - Node being looked for scheduling 127.0.0.1:1234 availableResource: 2016-05-05 17:27:14,070 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1276)) - Trying to schedule on node: 127.0.0.1, available: 2016-05-05 17:27:14,070 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(416)) - Trying to assign containers to child-queue of root 2016-05-05 17:27:14,071 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:printChildQueues(634)) - printChildQueues - queue: root child-queues: root.defaultusedCapacity=(0.0), label=(*) 2016-05-05 17:27:14,071 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(581)) - Trying to assign to queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=1, numContainers=0 2016-05-05 17:27:14,072 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:assignContainers(872)) - assignContainers: node=127.0.0.1 #applications=1 2016-05-05 17:27:14,072 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1190)) - User limit computation for moonfang in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-05-05 17:27:14,072 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimitAndSetHeadroom(1079)) - Headroom calculation for user moonfang: userLimit= queueMaxAvailRes= consumed= headroom= 2016-05-05 17:27:14,072 DEBUG [SchedulerEventDispatcher:Event Processor] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:assignContainers(496)) - pre-assignContainers for application application_1462440433865_0001 2016-05-05 17:27:14,073 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1190)) - User limit computation for moonfang in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-05-05 17:27:14,073 DEBUG [SchedulerEventDispatcher:Event Processor] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(480)) - showRequests: application=application_1462440433865_0001 headRoom= currentConsumption=0 2016-05-05 17:27:14,073 DEBUG [SchedulerEventDispatcher:Event Processor] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(484)) - showRequests: application=application_1462440433865_0001 request={Priority: 0, Capability: , # Containers: 1, Location: *, Relax Locality: true, Node Label Expression: } 2016-05-05 17:27:14,073 DEBUG [SchedulerEventDispatcher:Event Processor] allocator.IncreaseContainerAllocator (IncreaseContainerAllocator.java:assignContainers(184)) - Skip allocating increase request since we don't have any increase request on this node=127.0.0.1:1234 2016-05-05 17:27:14,074 DEBUG [SchedulerEventDispatcher:Event Processor] allocator.RegularContainerAllocator (RegularContainerAllocator.java:assignContainer(412)) - assignContainers: node=127.0.0.1 application=application_1462440433865_0001 priority=0 request={Priority: 0, Capability: , # Containers: 1, Location: *, Relax Locality: true, Node Label Expression: } type=OFF_SWITCH 2016-05-05 17:27:14,079 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(259)) - AppAttempt : appattempt_1462440433865_0001_000001 State is : SCHEDULED Waiting for state : ALLOCATED 2016-05-05 17:27:14,083 DEBUG [SchedulerEventDispatcher:Event Processor] scheduler.ActiveUsersManager (ActiveUsersManager.java:deactivateApplication(94)) - User moonfang removed from activeUsers, currently: 0 2016-05-05 17:27:14,083 DEBUG [SchedulerEventDispatcher:Event Processor] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:allocate(607)) - allocate: applicationId=application_1462440433865_0001 container=container_1462440433865_0001_01_000001 host=127.0.0.1:1234 user=moonfang resource= type=OFF_SWITCH 2016-05-05 17:27:14,084 DEBUG [SchedulerEventDispatcher:Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(439)) - Processing container_1462440433865_0001_01_000001 of type START 2016-05-05 17:27:14,084 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: CONTAINER_ALLOCATED 2016-05-05 17:27:14,084 INFO [SchedulerEventDispatcher:Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(453)) - container_1462440433865_0001_01_000001 Container Transitioned from NEW to ALLOCATED 2016-05-05 17:27:14,084 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(859)) - Processing event for appattempt_1462440433865_0001_000001 of type CONTAINER_ALLOCATED 2016-05-05 17:27:14,084 DEBUG [SchedulerEventDispatcher:Event Processor] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:allocate(225)) - allocate: applicationAttemptId=appattempt_1462440433865_0001_000001 container=container_1462440433865_0001_01_000001 host=127.0.0.1 type=OFF_SWITCH 2016-05-05 17:27:14,084 INFO [SchedulerEventDispatcher:Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(154)) - USER=moonfang OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1462440433865_0001 CONTAINERID=container_1462440433865_0001_01_000001 RESOURCE= 2016-05-05 17:27:14,085 DEBUG [SchedulerEventDispatcher:Event Processor] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(158)) - Assigned container container_1462440433865_0001_01_000001 of capacity on host 127.0.0.1:1234, which has 1 containers, used and available after allocation 2016-05-05 17:27:14,085 INFO [SchedulerEventDispatcher:Event Processor] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(89)) - assignedContainer application attempt=appattempt_1462440433865_0001_000001 container=container_1462440433865_0001_01_000001 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@14ce299f clusterResource= type=OFF_SWITCH 2016-05-05 17:27:14,085 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:assignContainers(941)) - post-assignContainers for application application_1462440433865_0001 2016-05-05 17:27:14,090 DEBUG [AsyncDispatcher event handler] security.BaseContainerTokenSecretManager (BaseContainerTokenSecretManager.java:createPassword(90)) - Creating password for container_1462440433865_0001_01_000001 for user container_1462440433865_0001_01_000001 (auth:SIMPLE) to be run on NM 127.0.0.1:1234 2016-05-05 17:27:14,139 DEBUG [AsyncDispatcher event handler] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 1 cluster_timestamp: 1462440433865 } attemptId: 1 } id: 1 } nmHostAddr: "127.0.0.1:1234" appSubmitter: "moonfang" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1462441034085 masterKeyId: -1802453347 rmIdentifier: 1462440433865 priority { priority: 0 } creationTime: 1462440434083 nodeLabelExpression: "" containerType: APPLICATION_MASTER executionType: GUARANTEED 2016-05-05 17:27:14,142 DEBUG [AsyncDispatcher event handler] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 1 cluster_timestamp: 1462440433865 } attemptId: 1 } id: 1 } nmHostAddr: "127.0.0.1:1234" appSubmitter: "moonfang" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1462441034085 masterKeyId: -1802453347 rmIdentifier: 1462440433865 priority { priority: 0 } creationTime: 1462440434083 nodeLabelExpression: "" containerType: APPLICATION_MASTER executionType: GUARANTEED 2016-05-05 17:27:14,143 INFO [AsyncDispatcher event handler] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:createAndGetNMToken(200)) - Sending NMToken for nodeId : 127.0.0.1:1234 for container : container_1462440433865_0001_01_000001 2016-05-05 17:27:14,145 DEBUG [AsyncDispatcher event handler] security.BaseNMTokenSecretManager (BaseNMTokenSecretManager.java:createPassword(75)) - creating password for appattempt_1462440433865_0001_000001 for user moonfang to run on NM 127.0.0.1:1234 2016-05-05 17:27:14,145 DEBUG [AsyncDispatcher event handler] security.NMTokenIdentifier (NMTokenIdentifier.java:write(96)) - Writing NMTokenIdentifier to RPC layer: appAttemptId { application_id { id: 1 cluster_timestamp: 1462440433865 } attemptId: 1 } nodeId { host: "127.0.0.1" port: 1234 } appSubmitter: "moonfang" keyId: -790324192 2016-05-05 17:27:14,146 DEBUG [AsyncDispatcher event handler] security.NMTokenIdentifier (NMTokenIdentifier.java:write(96)) - Writing NMTokenIdentifier to RPC layer: appAttemptId { application_id { id: 1 cluster_timestamp: 1462440433865 } attemptId: 1 } nodeId { host: "127.0.0.1" port: 1234 } appSubmitter: "moonfang" keyId: -790324192 2016-05-05 17:27:14,147 DEBUG [AsyncDispatcher event handler] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(439)) - Processing container_1462440433865_0001_01_000001 of type ACQUIRED 2016-05-05 17:27:14,148 INFO [AsyncDispatcher event handler] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(453)) - container_1462440433865_0001_01_000001 Container Transitioned from ALLOCATED to ACQUIRED 2016-05-05 17:27:14,148 DEBUG [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1190)) - User limit computation for moonfang in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-05-05 17:27:14,148 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1190)) - User limit computation for moonfang in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-05-05 17:27:14,148 DEBUG [SchedulerEventDispatcher:Event Processor] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(480)) - showRequests: application=application_1462440433865_0001 headRoom= currentConsumption=1024 2016-05-05 17:27:14,148 DEBUG [SchedulerEventDispatcher:Event Processor] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(484)) - showRequests: application=application_1462440433865_0001 request={Priority: 0, Capability: , # Containers: 0, Location: *, Relax Locality: true, Node Label Expression: } 2016-05-05 17:27:14,149 INFO [AsyncDispatcher event handler] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:clearNodeSetForAttempt(146)) - Clear node set for appattempt_1462440433865_0001_000001 2016-05-05 17:27:14,149 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1190)) - User limit computation for moonfang in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-05-05 17:27:14,149 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:storeAttempt(2075)) - Storing attempt: AppId: application_1462440433865_0001 AttemptId: appattempt_1462440433865_0001_000001 MasterContainer: Container: [ContainerId: container_1462440433865_0001_01_000001, NodeId: 127.0.0.1:1234, NodeHttpAddress: 127.0.0.1:2, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 127.0.0.1:1234 }, ] 2016-05-05 17:27:14,149 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1190)) - User limit computation for moonfang in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-05-05 17:27:14,150 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1190)) - User limit computation for moonfang in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-05-05 17:27:14,150 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:allocateResource(1401)) - default user=moonfang used= numContainers=1 headroom = user-resources= 2016-05-05 17:27:14,150 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(592)) - Assigned to queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.128, absoluteUsedCapacity=0.128, numApps=1, numContainers=1 --> , OFF_SWITCH 2016-05-05 17:27:14,150 INFO [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(606)) - Re-sorting assigned queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.128, absoluteUsedCapacity=0.128, numApps=1, numContainers=1 2016-05-05 17:27:14,150 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:printChildQueues(634)) - printChildQueues - queue: root child-queues: root.defaultusedCapacity=(0.128), label=(*) 2016-05-05 17:27:14,151 INFO [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(469)) - assignedContainer queue=root usedCapacity=0.128 absoluteUsedCapacity=0.128 used= cluster= 2016-05-05 17:27:14,151 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(481)) - ParentQ=root assignedSoFarInThisIteration= usedCapacity=0.128 absoluteUsedCapacity=0.128 2016-05-05 17:27:14,151 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(492)) - Not assigning more than one off-switch container, assignments so far: resource:; type:OFF_SWITCH; excessReservation:null; applicationid:null; skipped:false; fulfilled reservation:false; allocations(count/resource):1/; reservations(count/resource):0/ 2016-05-05 17:27:14,156 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(871)) - appattempt_1462440433865_0001_000001 State change from SCHEDULED to ALLOCATED_SAVING 2016-05-05 17:27:14,156 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStoreAppAttemptEvent.EventType: STORE_APP_ATTEMPT 2016-05-05 17:27:14,157 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(259)) - AppAttempt : appattempt_1462440433865_0001_000001 State is : ALLOCATED_SAVING Waiting for state : ALLOCATED 2016-05-05 17:27:14,157 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppRunningOnNodeEvent.EventType: APP_RUNNING_ON_NODE 2016-05-05 17:27:14,157 DEBUG [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:handleStoreEvent(1070)) - Processing event of type STORE_APP_ATTEMPT 2016-05-05 17:27:14,157 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(811)) - Processing event for application_1462440433865_0001 of type APP_RUNNING_ON_NODE 2016-05-05 17:27:14,157 DEBUG [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:transition(300)) - Storing info for attempt: appattempt_1462440433865_0001_000001 2016-05-05 17:27:14,159 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: ATTEMPT_NEW_SAVED 2016-05-05 17:27:14,159 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(859)) - Processing event for appattempt_1462440433865_0001_000001 of type ATTEMPT_NEW_SAVED 2016-05-05 17:27:14,159 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(871)) - appattempt_1462440433865_0001_000001 State change from ALLOCATED_SAVING to ALLOCATED 2016-05-05 17:27:14,159 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncherEvent.EventType: LAUNCH 2016-05-05 17:27:14,167 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(265)) - Attempt State is : ALLOCATED 2016-05-05 17:27:14,167 INFO [Thread-0] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:createAndGetAMRMToken(195)) - Create AMRMToken for ApplicationAttempt: appattempt_1462440433865_0001_000001 2016-05-05 17:27:14,169 INFO [Thread-0] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:createPassword(307)) - Creating password for appattempt_1462440433865_0001_000001 2016-05-05 17:27:14,171 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(259)) - AppAttempt : appattempt_1462440433865_0001_000001 State is : ALLOCATED Waiting for state : LAUNCHED 2016-05-05 17:27:14,171 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: LAUNCHED 2016-05-05 17:27:14,171 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(859)) - Processing event for appattempt_1462440433865_0001_000001 of type LAUNCHED 2016-05-05 17:27:14,171 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(871)) - appattempt_1462440433865_0001_000001 State change from ALLOCATED to LAUNCHED 2016-05-05 17:27:14,181 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(265)) - Attempt State is : LAUNCHED 2016-05-05 17:27:14,188 INFO [Thread-0] mortbay.log (Slf4jLog.java:info(67)) - Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2016-05-05 17:27:14,188 INFO [Thread-0] mortbay.log (Slf4jLog.java:info(67)) - ContainerStatus: ContainerStatus: [ContainerId: container_1462440433865_0001_01_000001, ExecutionType: GUARANTEED, State: COMPLETE, Capability: , Diagnostics: Success, ExitStatus: 0, ] 2016-05-05 17:27:14,188 INFO [Thread-0] mortbay.log (Slf4jLog.java:info(67)) - entry.getValue() [ContainerStatus: [ContainerId: container_1462440433865_0001_01_000001, ExecutionType: GUARANTEED, State: COMPLETE, Capability: , Diagnostics: Success, ExitStatus: 0, ]] 2016-05-05 17:27:14,188 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2016-05-05 17:27:14,188 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(618)) - Processing 127.0.0.1:1234 of type STATUS_UPDATE 2016-05-05 17:27:14,188 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(259)) - AppAttempt : appattempt_1462440433865_0001_000001 State is : LAUNCHED Waiting for state : FAILED 2016-05-05 17:27:14,189 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handleContainerStatus(1326)) - Container container_1462440433865_0001_01_000001 is the first container get launched for application application_1462440433865_0001 2016-05-05 17:27:14,189 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2016-05-05 17:27:14,189 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1052)) - nodeUpdate: 127.0.0.1:1234 clusterResources: 2016-05-05 17:27:14,189 DEBUG [SchedulerEventDispatcher:Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(439)) - Processing container_1462440433865_0001_01_000001 of type FINISHED 2016-05-05 17:27:14,190 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptContainerFinishedEvent.EventType: CONTAINER_FINISHED 2016-05-05 17:27:14,190 INFO [SchedulerEventDispatcher:Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(453)) - container_1462440433865_0001_01_000001 Container Transitioned from ACQUIRED to COMPLETED 2016-05-05 17:27:14,190 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(859)) - Processing event for appattempt_1462440433865_0001_000001 of type CONTAINER_FINISHED 2016-05-05 17:27:14,190 INFO [SchedulerEventDispatcher:Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(154)) - USER=moonfang OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1462440433865_0001 CONTAINERID=container_1462440433865_0001_01_000001 RESOURCE= 2016-05-05 17:27:14,190 DEBUG [SchedulerEventDispatcher:Event Processor] scheduler.SchedulerNode (SchedulerNode.java:releaseContainer(269)) - Released container container_1462440433865_0001_01_000001 of capacity on host 127.0.0.1:1234, which currently has 0 containers, used and available, release resources=true 2016-05-05 17:27:14,190 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1190)) - User limit computation for moonfang in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-05-05 17:27:14,190 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:releaseResource(1437)) - default used= numContainers=0 user=moonfang user-resources= 2016-05-05 17:27:14,191 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:internalReleaseResource(647)) - completedContainer root: numChildQueue= 1, capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.0, numApps=1, numContainers=0, cluster= 2016-05-05 17:27:14,191 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:internalReleaseResource(660)) - Re-sorting completed queue: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=1, numContainers=0 2016-05-05 17:27:14,191 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1125)) - Node being looked for scheduling 127.0.0.1:1234 availableResource: 2016-05-05 17:27:14,191 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:rememberTargetTransitionsAndStoreState(1301)) - Updating application attempt appattempt_1462440433865_0001_000001 with final state: FAILED, and exit status: 0 2016-05-05 17:27:14,191 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1276)) - Trying to schedule on node: 127.0.0.1, available: 2016-05-05 17:27:14,192 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(404)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-05-05 17:27:14,192 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(871)) - appattempt_1462440433865_0001_000001 State change from LAUNCHED to FINAL_SAVING 2016-05-05 17:27:14,192 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2016-05-05 17:27:14,192 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(618)) - Processing 127.0.0.1:1234 of type FINISHED_CONTAINERS_PULLED_BY_AM 2016-05-05 17:27:14,192 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateUpdateAppAttemptEvent.EventType: UPDATE_APP_ATTEMPT 2016-05-05 17:27:14,192 DEBUG [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:handleStoreEvent(1070)) - Processing event of type UPDATE_APP_ATTEMPT 2016-05-05 17:27:14,192 DEBUG [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:transition(331)) - Updating info for attempt: appattempt_1462440433865_0001_000001 2016-05-05 17:27:14,192 INFO [AsyncDispatcher event handler] recovery.RMStateStore (MemoryRMStateStore.java:updateApplicationAttemptStateInternal(139)) - Updating final state FAILED for attempt: appattempt_1462440433865_0001_000001 2016-05-05 17:27:14,193 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: ATTEMPT_UPDATE_SAVED 2016-05-05 17:27:14,193 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(859)) - Processing event for appattempt_1462440433865_0001_000001 of type ATTEMPT_UPDATE_SAVED 2016-05-05 17:27:14,193 INFO [AsyncDispatcher event handler] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:unregisterAttempt(682)) - Unregistering app attempt : appattempt_1462440433865_0001_000001 2016-05-05 17:27:14,193 INFO [AsyncDispatcher event handler] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:applicationMasterFinished(124)) - Application finished, removing password for appattempt_1462440433865_0001_000001 2016-05-05 17:27:14,193 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(871)) - appattempt_1462440433865_0001_000001 State change from FINAL_SAVING to FAILED 2016-05-05 17:27:14,193 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppFailedAttemptEvent.EventType: ATTEMPT_FAILED 2016-05-05 17:27:14,193 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(811)) - Processing event for application_1462440433865_0001 of type ATTEMPT_FAILED 2016-05-05 17:27:14,194 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:transition(1354)) - The number of failed attempts in previous 60000 milliseconds is 1. The max attempts is 2 2016-05-05 17:27:14,194 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.AppAttemptRemovedSchedulerEvent.EventType: APP_ATTEMPT_REMOVED 2016-05-05 17:27:14,194 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppStartAttemptEvent.EventType: START 2016-05-05 17:27:14,194 INFO [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:doneApplicationAttempt(864)) - Application Attempt appattempt_1462440433865_0001_000001 is done. finalState=FAILED 2016-05-05 17:27:14,194 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(859)) - Processing event for appattempt_1462440433865_0001_000002 of type START 2016-05-05 17:27:14,194 INFO [SchedulerEventDispatcher:Event Processor] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:clearRequests(135)) - Application application_1462440433865_0001 requests cleared 2016-05-05 17:27:14,194 INFO [AsyncDispatcher event handler] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:registerAppAttempt(676)) - Registering app attempt : appattempt_1462440433865_0001_000002 2016-05-05 17:27:14,195 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(871)) - appattempt_1462440433865_0001_000002 State change from NEW to SUBMITTED 2016-05-05 17:27:14,195 INFO [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:removeApplicationAttempt(793)) - Application removed - appId: application_1462440433865_0001 user: moonfang queue: default #user-pending-applications: 0 #user-active-applications: 0 #queue-pending-applications: 0 #queue-active-applications: 0 2016-05-05 17:27:14,195 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.AppAttemptAddedSchedulerEvent.EventType: APP_ATTEMPT_ADDED 2016-05-05 17:27:14,195 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:activateApplications(652)) - application application_1462440433865_0001 AMResource maxAMResourcePerQueuePercent 0.1 amLimit lastClusterResource amIfStarted AM node-partition name 2016-05-05 17:27:14,195 INFO [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:activateApplications(726)) - Application application_1462440433865_0001 from user: moonfang activated in queue: default 2016-05-05 17:27:14,195 INFO [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:addApplicationAttempt(741)) - Application added - appId: application_1462440433865_0001 user: moonfang, leaf-queue: default #user-pending-applications: 0 #user-active-applications: 1 #queue-pending-applications: 0 #queue-active-applications: 1 2016-05-05 17:27:14,195 INFO [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:addApplicationAttempt(825)) - Added Application Attempt appattempt_1462440433865_0001_000002 to scheduler from user moonfang in queue default 2016-05-05 17:27:14,195 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: ATTEMPT_ADDED 2016-05-05 17:27:14,196 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(859)) - Processing event for appattempt_1462440433865_0001_000002 of type ATTEMPT_ADDED 2016-05-05 17:27:14,196 DEBUG [AsyncDispatcher event handler] blacklist.SimpleBlacklistManager (SimpleBlacklistManager.java:getBlacklistUpdates(69)) - blacklist size 0 is less than failure threshold ratio 0.800000011920929 out of total usable nodes 1 2016-05-05 17:27:14,196 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:transition(1041)) - Using blacklist for AM: additions([]) and removals([]) 2016-05-05 17:27:14,196 DEBUG [AsyncDispatcher event handler] capacity.CapacityScheduler (CapacityScheduler.java:allocate(984)) - allocate: pre-update appattempt_1462440433865_0001_000002 ask size =1 2016-05-05 17:27:14,196 DEBUG [AsyncDispatcher event handler] scheduler.ActiveUsersManager (ActiveUsersManager.java:activateApplication(68)) - User moonfang added to activeUsers, currently: 1 2016-05-05 17:27:14,196 DEBUG [AsyncDispatcher event handler] capacity.CapacityScheduler (CapacityScheduler.java:allocate(996)) - allocate: post-update 2016-05-05 17:27:14,196 DEBUG [AsyncDispatcher event handler] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(480)) - showRequests: application=application_1462440433865_0001 headRoom= currentConsumption=0 2016-05-05 17:27:14,196 DEBUG [AsyncDispatcher event handler] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(484)) - showRequests: application=application_1462440433865_0001 request={Priority: 0, Capability: , # Containers: 1, Location: *, Relax Locality: true, Node Label Expression: } 2016-05-05 17:27:14,196 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(871)) - appattempt_1462440433865_0001_000002 State change from SUBMITTED to SCHEDULED 2016-05-05 17:27:14,199 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(265)) - Attempt State is : FAILED 2016-05-05 17:27:14,199 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(192)) - App State is : ACCEPTED 2016-05-05 17:27:14,199 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(192)) - App State is : ACCEPTED 2016-05-05 17:27:14,199 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(265)) - Attempt State is : SCHEDULED Launch AM appattempt_1462440433865_0001_000002 2016-05-05 17:27:14,199 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2016-05-05 17:27:14,199 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(259)) - AppAttempt : appattempt_1462440433865_0001_000002 State is : SCHEDULED Waiting for state : ALLOCATED 2016-05-05 17:27:14,199 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(618)) - Processing 127.0.0.1:1234 of type STATUS_UPDATE 2016-05-05 17:27:14,200 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2016-05-05 17:27:14,200 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1052)) - nodeUpdate: 127.0.0.1:1234 clusterResources: 2016-05-05 17:27:14,200 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1125)) - Node being looked for scheduling 127.0.0.1:1234 availableResource: 2016-05-05 17:27:14,200 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1276)) - Trying to schedule on node: 127.0.0.1, available: 2016-05-05 17:27:14,200 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(416)) - Trying to assign containers to child-queue of root 2016-05-05 17:27:14,200 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:printChildQueues(634)) - printChildQueues - queue: root child-queues: root.defaultusedCapacity=(0.0), label=(*) 2016-05-05 17:27:14,200 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(581)) - Trying to assign to queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=1, numContainers=0 2016-05-05 17:27:14,200 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:assignContainers(872)) - assignContainers: node=127.0.0.1 #applications=1 2016-05-05 17:27:14,200 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1190)) - User limit computation for moonfang in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-05-05 17:27:14,200 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimitAndSetHeadroom(1079)) - Headroom calculation for user moonfang: userLimit= queueMaxAvailRes= consumed= headroom= 2016-05-05 17:27:14,201 DEBUG [SchedulerEventDispatcher:Event Processor] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:assignContainers(496)) - pre-assignContainers for application application_1462440433865_0001 2016-05-05 17:27:14,201 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1190)) - User limit computation for moonfang in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-05-05 17:27:14,201 DEBUG [SchedulerEventDispatcher:Event Processor] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(480)) - showRequests: application=application_1462440433865_0001 headRoom= currentConsumption=0 2016-05-05 17:27:14,201 DEBUG [SchedulerEventDispatcher:Event Processor] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(484)) - showRequests: application=application_1462440433865_0001 request={Priority: 0, Capability: , # Containers: 1, Location: *, Relax Locality: true, Node Label Expression: } 2016-05-05 17:27:14,201 DEBUG [SchedulerEventDispatcher:Event Processor] allocator.IncreaseContainerAllocator (IncreaseContainerAllocator.java:assignContainers(184)) - Skip allocating increase request since we don't have any increase request on this node=127.0.0.1:1234 2016-05-05 17:27:14,201 DEBUG [SchedulerEventDispatcher:Event Processor] allocator.RegularContainerAllocator (RegularContainerAllocator.java:assignContainer(412)) - assignContainers: node=127.0.0.1 application=application_1462440433865_0001 priority=0 request={Priority: 0, Capability: , # Containers: 1, Location: *, Relax Locality: true, Node Label Expression: } type=OFF_SWITCH 2016-05-05 17:27:14,201 DEBUG [SchedulerEventDispatcher:Event Processor] scheduler.ActiveUsersManager (ActiveUsersManager.java:deactivateApplication(94)) - User moonfang removed from activeUsers, currently: 0 2016-05-05 17:27:14,201 DEBUG [SchedulerEventDispatcher:Event Processor] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:allocate(607)) - allocate: applicationId=application_1462440433865_0001 container=container_1462440433865_0001_02_000001 host=127.0.0.1:1234 user=moonfang resource= type=OFF_SWITCH 2016-05-05 17:27:14,201 DEBUG [SchedulerEventDispatcher:Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(439)) - Processing container_1462440433865_0001_02_000001 of type START 2016-05-05 17:27:14,202 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: CONTAINER_ALLOCATED 2016-05-05 17:27:14,202 INFO [SchedulerEventDispatcher:Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(453)) - container_1462440433865_0001_02_000001 Container Transitioned from NEW to ALLOCATED 2016-05-05 17:27:14,202 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(859)) - Processing event for appattempt_1462440433865_0001_000002 of type CONTAINER_ALLOCATED 2016-05-05 17:27:14,202 DEBUG [SchedulerEventDispatcher:Event Processor] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:allocate(225)) - allocate: applicationAttemptId=appattempt_1462440433865_0001_000002 container=container_1462440433865_0001_02_000001 host=127.0.0.1 type=OFF_SWITCH 2016-05-05 17:27:14,202 INFO [SchedulerEventDispatcher:Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(154)) - USER=moonfang OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1462440433865_0001 CONTAINERID=container_1462440433865_0001_02_000001 RESOURCE= 2016-05-05 17:27:14,202 DEBUG [SchedulerEventDispatcher:Event Processor] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(158)) - Assigned container container_1462440433865_0001_02_000001 of capacity on host 127.0.0.1:1234, which has 1 containers, used and available after allocation 2016-05-05 17:27:14,202 INFO [SchedulerEventDispatcher:Event Processor] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(89)) - assignedContainer application attempt=appattempt_1462440433865_0001_000002 container=container_1462440433865_0001_02_000001 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@178fd0ca clusterResource= type=OFF_SWITCH 2016-05-05 17:27:14,202 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:assignContainers(941)) - post-assignContainers for application application_1462440433865_0001 2016-05-05 17:27:14,202 DEBUG [AsyncDispatcher event handler] security.BaseContainerTokenSecretManager (BaseContainerTokenSecretManager.java:createPassword(90)) - Creating password for container_1462440433865_0001_02_000001 for user container_1462440433865_0001_02_000001 (auth:SIMPLE) to be run on NM 127.0.0.1:1234 2016-05-05 17:27:14,203 DEBUG [AsyncDispatcher event handler] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 1 cluster_timestamp: 1462440433865 } attemptId: 2 } id: 1 } nmHostAddr: "127.0.0.1:1234" appSubmitter: "moonfang" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1462441034202 masterKeyId: -1802453347 rmIdentifier: 1462440433865 priority { priority: 0 } creationTime: 1462440434201 nodeLabelExpression: "" containerType: APPLICATION_MASTER executionType: GUARANTEED 2016-05-05 17:27:14,203 DEBUG [AsyncDispatcher event handler] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 1 cluster_timestamp: 1462440433865 } attemptId: 2 } id: 1 } nmHostAddr: "127.0.0.1:1234" appSubmitter: "moonfang" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1462441034202 masterKeyId: -1802453347 rmIdentifier: 1462440433865 priority { priority: 0 } creationTime: 1462440434201 nodeLabelExpression: "" containerType: APPLICATION_MASTER executionType: GUARANTEED 2016-05-05 17:27:14,203 INFO [AsyncDispatcher event handler] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:createAndGetNMToken(200)) - Sending NMToken for nodeId : 127.0.0.1:1234 for container : container_1462440433865_0001_02_000001 2016-05-05 17:27:14,204 DEBUG [AsyncDispatcher event handler] security.BaseNMTokenSecretManager (BaseNMTokenSecretManager.java:createPassword(75)) - creating password for appattempt_1462440433865_0001_000002 for user moonfang to run on NM 127.0.0.1:1234 2016-05-05 17:27:14,204 DEBUG [AsyncDispatcher event handler] security.NMTokenIdentifier (NMTokenIdentifier.java:write(96)) - Writing NMTokenIdentifier to RPC layer: appAttemptId { application_id { id: 1 cluster_timestamp: 1462440433865 } attemptId: 2 } nodeId { host: "127.0.0.1" port: 1234 } appSubmitter: "moonfang" keyId: -790324192 2016-05-05 17:27:14,204 DEBUG [AsyncDispatcher event handler] security.NMTokenIdentifier (NMTokenIdentifier.java:write(96)) - Writing NMTokenIdentifier to RPC layer: appAttemptId { application_id { id: 1 cluster_timestamp: 1462440433865 } attemptId: 2 } nodeId { host: "127.0.0.1" port: 1234 } appSubmitter: "moonfang" keyId: -790324192 2016-05-05 17:27:14,204 DEBUG [AsyncDispatcher event handler] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(439)) - Processing container_1462440433865_0001_02_000001 of type ACQUIRED 2016-05-05 17:27:14,204 INFO [AsyncDispatcher event handler] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(453)) - container_1462440433865_0001_02_000001 Container Transitioned from ALLOCATED to ACQUIRED 2016-05-05 17:27:14,205 DEBUG [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1190)) - User limit computation for moonfang in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-05-05 17:27:14,205 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1190)) - User limit computation for moonfang in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-05-05 17:27:14,205 DEBUG [SchedulerEventDispatcher:Event Processor] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(480)) - showRequests: application=application_1462440433865_0001 headRoom= currentConsumption=1024 2016-05-05 17:27:14,205 DEBUG [SchedulerEventDispatcher:Event Processor] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(484)) - showRequests: application=application_1462440433865_0001 request={Priority: 0, Capability: , # Containers: 0, Location: *, Relax Locality: true, Node Label Expression: } 2016-05-05 17:27:14,205 INFO [AsyncDispatcher event handler] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:clearNodeSetForAttempt(146)) - Clear node set for appattempt_1462440433865_0001_000002 2016-05-05 17:27:14,205 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1190)) - User limit computation for moonfang in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-05-05 17:27:14,205 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:storeAttempt(2075)) - Storing attempt: AppId: application_1462440433865_0001 AttemptId: appattempt_1462440433865_0001_000002 MasterContainer: Container: [ContainerId: container_1462440433865_0001_02_000001, NodeId: 127.0.0.1:1234, NodeHttpAddress: 127.0.0.1:2, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 127.0.0.1:1234 }, ] 2016-05-05 17:27:14,206 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1190)) - User limit computation for moonfang in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-05-05 17:27:14,206 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1190)) - User limit computation for moonfang in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-05-05 17:27:14,206 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(871)) - appattempt_1462440433865_0001_000002 State change from SCHEDULED to ALLOCATED_SAVING 2016-05-05 17:27:14,206 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStoreAppAttemptEvent.EventType: STORE_APP_ATTEMPT 2016-05-05 17:27:14,206 DEBUG [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:handleStoreEvent(1070)) - Processing event of type STORE_APP_ATTEMPT 2016-05-05 17:27:14,206 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppRunningOnNodeEvent.EventType: APP_RUNNING_ON_NODE 2016-05-05 17:27:14,206 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:allocateResource(1401)) - default user=moonfang used= numContainers=1 headroom = user-resources= 2016-05-05 17:27:14,206 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(811)) - Processing event for application_1462440433865_0001 of type APP_RUNNING_ON_NODE 2016-05-05 17:27:14,206 DEBUG [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:transition(300)) - Storing info for attempt: appattempt_1462440433865_0001_000002 2016-05-05 17:27:14,206 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(592)) - Assigned to queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.128, absoluteUsedCapacity=0.128, numApps=1, numContainers=1 --> , OFF_SWITCH 2016-05-05 17:27:14,207 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: ATTEMPT_NEW_SAVED 2016-05-05 17:27:14,207 INFO [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(606)) - Re-sorting assigned queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.128, absoluteUsedCapacity=0.128, numApps=1, numContainers=1 2016-05-05 17:27:14,207 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(859)) - Processing event for appattempt_1462440433865_0001_000002 of type ATTEMPT_NEW_SAVED 2016-05-05 17:27:14,207 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:printChildQueues(634)) - printChildQueues - queue: root child-queues: root.defaultusedCapacity=(0.128), label=(*) 2016-05-05 17:27:14,207 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(871)) - appattempt_1462440433865_0001_000002 State change from ALLOCATED_SAVING to ALLOCATED 2016-05-05 17:27:14,207 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncherEvent.EventType: LAUNCH 2016-05-05 17:27:14,207 INFO [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(469)) - assignedContainer queue=root usedCapacity=0.128 absoluteUsedCapacity=0.128 used= cluster= 2016-05-05 17:27:14,207 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(481)) - ParentQ=root assignedSoFarInThisIteration= usedCapacity=0.128 absoluteUsedCapacity=0.128 2016-05-05 17:27:14,207 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(492)) - Not assigning more than one off-switch container, assignments so far: resource:; type:OFF_SWITCH; excessReservation:null; applicationid:null; skipped:false; fulfilled reservation:false; allocations(count/resource):1/; reservations(count/resource):0/ 2016-05-05 17:27:14,209 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(265)) - Attempt State is : ALLOCATED 2016-05-05 17:27:14,210 INFO [Thread-0] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:createAndGetAMRMToken(195)) - Create AMRMToken for ApplicationAttempt: appattempt_1462440433865_0001_000002 2016-05-05 17:27:14,210 INFO [Thread-0] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:createPassword(307)) - Creating password for appattempt_1462440433865_0001_000002 2016-05-05 17:27:14,210 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: LAUNCHED 2016-05-05 17:27:14,210 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(259)) - AppAttempt : appattempt_1462440433865_0001_000002 State is : ALLOCATED Waiting for state : LAUNCHED 2016-05-05 17:27:14,210 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(859)) - Processing event for appattempt_1462440433865_0001_000002 of type LAUNCHED 2016-05-05 17:27:14,210 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(871)) - appattempt_1462440433865_0001_000002 State change from ALLOCATED to LAUNCHED 2016-05-05 17:27:14,220 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(265)) - Attempt State is : LAUNCHED 2016-05-05 17:27:14,220 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(265)) - Attempt State is : LAUNCHED 2016-05-05 17:27:14,229 DEBUG [Thread-0] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1780)) - PrivilegedAction as:appattempt_1462440433865_0001_000002 (auth:SIMPLE) from:org.apache.hadoop.yarn.server.resourcemanager.MockAM.registerAppAttempt(MockAM.java:116) 2016-05-05 17:27:14,229 INFO [Thread-0] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:registerApplicationMaster(235)) - AM registration appattempt_1462440433865_0001_000002 2016-05-05 17:27:14,230 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptRegistrationEvent.EventType: REGISTERED 2016-05-05 17:27:14,230 INFO [Thread-0] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(175)) - USER=moonfang OPERATION=Register App Master TARGET=ApplicationMasterService RESULT=SUCCESS APPID=application_1462440433865_0001 APPATTEMPTID=appattempt_1462440433865_0001_000002 2016-05-05 17:27:14,230 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(859)) - Processing event for appattempt_1462440433865_0001_000002 of type REGISTERED 2016-05-05 17:27:14,230 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(871)) - appattempt_1462440433865_0001_000002 State change from LAUNCHED to RUNNING 2016-05-05 17:27:14,230 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEvent.EventType: ATTEMPT_REGISTERED 2016-05-05 17:27:14,230 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(811)) - Processing event for application_1462440433865_0001 of type ATTEMPT_REGISTERED 2016-05-05 17:27:14,230 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(823)) - application_1462440433865_0001 State change from ACCEPTED to RUNNING 2016-05-05 17:27:14,234 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(192)) - App State is : RUNNING 2016-05-05 17:27:14,234 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(265)) - Attempt State is : RUNNING 2016-05-05 17:27:14,234 INFO [Thread-0] mortbay.log (Slf4jLog.java:info(67)) - ContainerStatus: ContainerStatus: [ContainerId: container_1462440433865_0001_02_000001, ExecutionType: GUARANTEED, State: COMPLETE, Capability: , Diagnostics: Success, ExitStatus: 0, ] 2016-05-05 17:27:14,235 INFO [Thread-0] mortbay.log (Slf4jLog.java:info(67)) - entry.getValue() [ContainerStatus: [ContainerId: container_1462440433865_0001_02_000001, ExecutionType: GUARANTEED, State: COMPLETE, Capability: , Diagnostics: Success, ExitStatus: 0, ]] 2016-05-05 17:27:14,235 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(259)) - AppAttempt : appattempt_1462440433865_0001_000002 State is : RUNNING Waiting for state : FAILED 2016-05-05 17:27:14,235 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2016-05-05 17:27:14,235 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(618)) - Processing 127.0.0.1:1234 of type STATUS_UPDATE 2016-05-05 17:27:14,235 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2016-05-05 17:27:14,235 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1052)) - nodeUpdate: 127.0.0.1:1234 clusterResources: 2016-05-05 17:27:14,235 DEBUG [SchedulerEventDispatcher:Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(439)) - Processing container_1462440433865_0001_02_000001 of type FINISHED 2016-05-05 17:27:14,236 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptContainerFinishedEvent.EventType: CONTAINER_FINISHED 2016-05-05 17:27:14,236 INFO [SchedulerEventDispatcher:Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(453)) - container_1462440433865_0001_02_000001 Container Transitioned from ACQUIRED to COMPLETED 2016-05-05 17:27:14,236 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(859)) - Processing event for appattempt_1462440433865_0001_000002 of type CONTAINER_FINISHED 2016-05-05 17:27:14,236 INFO [SchedulerEventDispatcher:Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(154)) - USER=moonfang OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1462440433865_0001 CONTAINERID=container_1462440433865_0001_02_000001 RESOURCE= 2016-05-05 17:27:14,236 DEBUG [SchedulerEventDispatcher:Event Processor] scheduler.SchedulerNode (SchedulerNode.java:releaseContainer(269)) - Released container container_1462440433865_0001_02_000001 of capacity on host 127.0.0.1:1234, which currently has 0 containers, used and available, release resources=true 2016-05-05 17:27:14,236 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1190)) - User limit computation for moonfang in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-05-05 17:27:14,236 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:releaseResource(1437)) - default used= numContainers=0 user=moonfang user-resources= 2016-05-05 17:27:14,236 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:internalReleaseResource(647)) - completedContainer root: numChildQueue= 1, capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.0, numApps=1, numContainers=0, cluster= 2016-05-05 17:27:14,236 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:rememberTargetTransitionsAndStoreState(1301)) - Updating application attempt appattempt_1462440433865_0001_000002 with final state: FAILED, and exit status: 0 2016-05-05 17:27:14,236 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:internalReleaseResource(660)) - Re-sorting completed queue: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=1, numContainers=0 2016-05-05 17:27:14,236 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(871)) - appattempt_1462440433865_0001_000002 State change from RUNNING to FINAL_SAVING 2016-05-05 17:27:14,236 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateUpdateAppAttemptEvent.EventType: UPDATE_APP_ATTEMPT 2016-05-05 17:27:14,237 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2016-05-05 17:27:14,237 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(618)) - Processing 127.0.0.1:1234 of type FINISHED_CONTAINERS_PULLED_BY_AM 2016-05-05 17:27:14,237 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1125)) - Node being looked for scheduling 127.0.0.1:1234 availableResource: 2016-05-05 17:27:14,237 DEBUG [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:handleStoreEvent(1070)) - Processing event of type UPDATE_APP_ATTEMPT 2016-05-05 17:27:14,237 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1276)) - Trying to schedule on node: 127.0.0.1, available: 2016-05-05 17:27:14,237 DEBUG [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:transition(331)) - Updating info for attempt: appattempt_1462440433865_0001_000002 2016-05-05 17:27:14,237 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(404)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-05-05 17:27:14,237 INFO [AsyncDispatcher event handler] recovery.RMStateStore (MemoryRMStateStore.java:updateApplicationAttemptStateInternal(139)) - Updating final state FAILED for attempt: appattempt_1462440433865_0001_000002 2016-05-05 17:27:14,237 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: ATTEMPT_UPDATE_SAVED 2016-05-05 17:27:14,237 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(859)) - Processing event for appattempt_1462440433865_0001_000002 of type ATTEMPT_UPDATE_SAVED 2016-05-05 17:27:14,237 INFO [AsyncDispatcher event handler] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:unregisterAttempt(682)) - Unregistering app attempt : appattempt_1462440433865_0001_000002 2016-05-05 17:27:14,238 INFO [AsyncDispatcher event handler] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:applicationMasterFinished(124)) - Application finished, removing password for appattempt_1462440433865_0001_000002 2016-05-05 17:27:14,238 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(871)) - appattempt_1462440433865_0001_000002 State change from FINAL_SAVING to FAILED 2016-05-05 17:27:14,238 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppFailedAttemptEvent.EventType: ATTEMPT_FAILED 2016-05-05 17:27:14,238 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(811)) - Processing event for application_1462440433865_0001 of type ATTEMPT_FAILED 2016-05-05 17:27:14,238 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:transition(1354)) - The number of failed attempts in previous 60000 milliseconds is 2. The max attempts is 2 2016-05-05 17:27:14,238 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:rememberTargetTransitionsAndStoreState(1149)) - Updating application application_1462440433865_0001 with final state: FAILED 2016-05-05 17:27:14,239 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateUpdateAppEvent.EventType: UPDATE_APP 2016-05-05 17:27:14,239 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(823)) - application_1462440433865_0001 State change from RUNNING to FINAL_SAVING 2016-05-05 17:27:14,239 DEBUG [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:handleStoreEvent(1070)) - Processing event of type UPDATE_APP 2016-05-05 17:27:14,239 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.AppAttemptRemovedSchedulerEvent.EventType: APP_ATTEMPT_REMOVED 2016-05-05 17:27:14,239 INFO [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:transition(242)) - Updating info for app: application_1462440433865_0001 2016-05-05 17:27:14,239 INFO [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:doneApplicationAttempt(864)) - Application Attempt appattempt_1462440433865_0001_000002 is done. finalState=FAILED 2016-05-05 17:27:14,239 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncherEvent.EventType: CLEANUP 2016-05-05 17:27:14,239 INFO [AsyncDispatcher event handler] recovery.RMStateStore (MemoryRMStateStore.java:updateApplicationStateInternal(107)) - Updating final state FAILED for app: application_1462440433865_0001 2016-05-05 17:27:14,239 INFO [SchedulerEventDispatcher:Event Processor] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:clearRequests(135)) - Application application_1462440433865_0001 requests cleared 2016-05-05 17:27:14,239 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEvent.EventType: APP_UPDATE_SAVED 2016-05-05 17:27:14,239 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(811)) - Processing event for application_1462440433865_0001 of type APP_UPDATE_SAVED 2016-05-05 17:27:14,239 INFO [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:removeApplicationAttempt(793)) - Application removed - appId: application_1462440433865_0001 user: moonfang queue: default #user-pending-applications: 0 #user-active-applications: 0 #queue-pending-applications: 0 #queue-active-applications: 0 2016-05-05 17:27:14,240 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:transition(1086)) - Application application_1462440433865_0001 failed 2 times in previous 60000 milliseconds due to AM Container for appattempt_1462440433865_0001_000002 exited with exitCode: 0 Failing this attempt.Diagnostics: SuccessFor more detailed output, check the application tracking page: http://moonfang-Lenovo-Product:8088/cluster/app/application_1462440433865_0001 Then click on links to logs of each attempt. . Failing the application. 2016-05-05 17:27:14,240 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(823)) - application_1462440433865_0001 State change from FINAL_SAVING to FAILED 2016-05-05 17:27:14,240 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeCleanAppEvent.EventType: CLEANUP_APP 2016-05-05 17:27:14,240 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(618)) - Processing 127.0.0.1:1234 of type CLEANUP_APP 2016-05-05 17:27:14,241 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.AppRemovedSchedulerEvent.EventType: APP_REMOVED 2016-05-05 17:27:14,241 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.RMAppManagerEvent.EventType: APP_COMPLETED 2016-05-05 17:27:14,241 INFO [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:removeApplication(377)) - Application removed - appId: application_1462440433865_0001 user: moonfang leaf-queue of parent: root #applications: 0 2016-05-05 17:27:14,241 DEBUG [AsyncDispatcher event handler] resourcemanager.RMAppManager (RMAppManager.java:handle(484)) - RMAppManager processing event for application_1462440433865_0001 of type APP_COMPLETED 2016-05-05 17:27:14,241 WARN [AsyncDispatcher event handler] resourcemanager.RMAuditLogger (RMAuditLogger.java:logFailure(340)) - USER=moonfang OPERATION=Application Finished - Failed TARGET=RMAppManager RESULT=FAILURE DESCRIPTION=App failed with state: FAILED PERMISSIONS=Application application_1462440433865_0001 failed 2 times in previous 60000 milliseconds due to AM Container for appattempt_1462440433865_0001_000002 exited with exitCode: 0 Failing this attempt.Diagnostics: SuccessFor more detailed output, check the application tracking page: http://moonfang-Lenovo-Product:8088/cluster/app/application_1462440433865_0001 Then click on links to logs of each attempt. . Failing the application. APPID=application_1462440433865_0001 2016-05-05 17:27:14,241 DEBUG [AsyncDispatcher event handler] scheduler.AbstractYarnScheduler (AbstractYarnScheduler.java:getAppResourceUsageReport(281)) - Request for appInfo of unknown attempt appattempt_1462440433865_0001_000002 2016-05-05 17:27:14,242 INFO [AsyncDispatcher event handler] resourcemanager.RMAppManager$ApplicationSummary (RMAppManager.java:logAppSummary(195)) - appId=application_1462440433865_0001,name=,user=moonfang,queue=default,state=FAILED,trackingUrl=http://moonfang-Lenovo-Product:8088/cluster/app/application_1462440433865_0001,appMasterHost=N/A,startTime=1462440434000,finishTime=1462440434238,finalStatus=FAILED,memorySeconds=142,vcoreSeconds=0,preemptedAMContainers=0,preemptedNonAMContainers=0,preemptedResources=,applicationType=YARN 2016-05-05 17:27:14,245 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(265)) - Attempt State is : FAILED 2016-05-05 17:27:14,245 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(192)) - App State is : FAILED 2016-05-05 17:27:14,245 INFO [Thread-0] resourcemanager.ClientRMService (ClientRMService.java:getNewApplicationId(317)) - Allocated new applicationId: 2 2016-05-05 17:27:14,246 DEBUG [Thread-0] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1780)) - PrivilegedAction as:moonfang (auth:SIMPLE) from:org.apache.hadoop.yarn.server.resourcemanager.MockRM.submitApp(MockRM.java:645) 2016-05-05 17:27:14,246 INFO [Thread-0] capacity.CapacityScheduler (CapacityScheduler.java:checkAndGetApplicationPriority(2051)) - Priority '0' is acceptable in queue : default for application: application_1462440433865_0002 for the user: moonfang 2016-05-05 17:27:14,246 INFO [Thread-0] rmapp.RMAppImpl (RMAppImpl.java:(439)) - The attemptFailuresValidityInterval for the application: application_1462440433865_0002 is 10000. 2016-05-05 17:27:14,246 INFO [Thread-0] resourcemanager.ClientRMService (ClientRMService.java:submitApplication(621)) - Application with id 2 submitted by user moonfang 2016-05-05 17:27:14,246 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEvent.EventType: START 2016-05-05 17:27:14,246 INFO [Thread-0] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(183)) - USER=moonfang OPERATION=Submit Application Request TARGET=ClientRMService RESULT=SUCCESS APPID=application_1462440433865_0002 2016-05-05 17:27:14,246 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(811)) - Processing event for application_1462440433865_0002 of type START 2016-05-05 17:27:14,247 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:transition(1127)) - Storing application with id application_1462440433865_0002 2016-05-05 17:27:14,247 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(823)) - application_1462440433865_0002 State change from NEW to NEW_SAVING 2016-05-05 17:27:14,247 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStoreAppEvent.EventType: STORE_APP 2016-05-05 17:27:14,247 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(186)) - App : application_1462440433865_0002 State is : NEW_SAVING Waiting for state : ACCEPTED 2016-05-05 17:27:14,247 DEBUG [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:handleStoreEvent(1070)) - Processing event of type STORE_APP 2016-05-05 17:27:14,247 INFO [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:transition(213)) - Storing info for app: application_1462440433865_0002 2016-05-05 17:27:14,247 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEvent.EventType: APP_NEW_SAVED 2016-05-05 17:27:14,247 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(811)) - Processing event for application_1462440433865_0002 of type APP_NEW_SAVED 2016-05-05 17:27:14,247 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(823)) - application_1462440433865_0002 State change from NEW_SAVING to SUBMITTED 2016-05-05 17:27:14,248 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.AppAddedSchedulerEvent.EventType: APP_ADDED 2016-05-05 17:27:14,248 INFO [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:addApplication(352)) - Application added - appId: application_1462440433865_0002 user: moonfang leaf-queue of parent: root #applications: 1 2016-05-05 17:27:14,248 INFO [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:addApplication(785)) - Accepted application application_1462440433865_0002 from user: moonfang, in queue: default 2016-05-05 17:27:14,248 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEvent.EventType: APP_ACCEPTED 2016-05-05 17:27:14,248 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(811)) - Processing event for application_1462440433865_0002 of type APP_ACCEPTED 2016-05-05 17:27:14,248 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(823)) - application_1462440433865_0002 State change from SUBMITTED to ACCEPTED 2016-05-05 17:27:14,248 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppStartAttemptEvent.EventType: START 2016-05-05 17:27:14,249 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(859)) - Processing event for appattempt_1462440433865_0002_000001 of type START 2016-05-05 17:27:14,249 INFO [AsyncDispatcher event handler] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:registerAppAttempt(676)) - Registering app attempt : appattempt_1462440433865_0002_000001 2016-05-05 17:27:14,249 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(871)) - appattempt_1462440433865_0002_000001 State change from NEW to SUBMITTED 2016-05-05 17:27:14,249 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.AppAttemptAddedSchedulerEvent.EventType: APP_ATTEMPT_ADDED 2016-05-05 17:27:14,249 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:activateApplications(652)) - application application_1462440433865_0002 AMResource maxAMResourcePerQueuePercent 0.1 amLimit lastClusterResource amIfStarted AM node-partition name 2016-05-05 17:27:14,249 INFO [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:activateApplications(726)) - Application application_1462440433865_0002 from user: moonfang activated in queue: default 2016-05-05 17:27:14,249 INFO [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:addApplicationAttempt(741)) - Application added - appId: application_1462440433865_0002 user: moonfang, leaf-queue: default #user-pending-applications: 0 #user-active-applications: 1 #queue-pending-applications: 0 #queue-active-applications: 1 2016-05-05 17:27:14,249 INFO [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:addApplicationAttempt(825)) - Added Application Attempt appattempt_1462440433865_0002_000001 to scheduler from user moonfang in queue default 2016-05-05 17:27:14,249 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: ATTEMPT_ADDED 2016-05-05 17:27:14,250 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(859)) - Processing event for appattempt_1462440433865_0002_000001 of type ATTEMPT_ADDED 2016-05-05 17:27:14,250 DEBUG [AsyncDispatcher event handler] blacklist.SimpleBlacklistManager (SimpleBlacklistManager.java:getBlacklistUpdates(69)) - blacklist size 0 is less than failure threshold ratio 0.800000011920929 out of total usable nodes 1 2016-05-05 17:27:14,250 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:transition(1041)) - Using blacklist for AM: additions([]) and removals([]) 2016-05-05 17:27:14,250 DEBUG [AsyncDispatcher event handler] capacity.CapacityScheduler (CapacityScheduler.java:allocate(984)) - allocate: pre-update appattempt_1462440433865_0002_000001 ask size =1 2016-05-05 17:27:14,250 DEBUG [AsyncDispatcher event handler] scheduler.ActiveUsersManager (ActiveUsersManager.java:activateApplication(68)) - User moonfang added to activeUsers, currently: 1 2016-05-05 17:27:14,250 DEBUG [AsyncDispatcher event handler] capacity.CapacityScheduler (CapacityScheduler.java:allocate(996)) - allocate: post-update 2016-05-05 17:27:14,250 DEBUG [AsyncDispatcher event handler] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(480)) - showRequests: application=application_1462440433865_0002 headRoom= currentConsumption=0 2016-05-05 17:27:14,250 DEBUG [AsyncDispatcher event handler] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(484)) - showRequests: application=application_1462440433865_0002 request={Priority: 0, Capability: , # Containers: 1, Location: *, Relax Locality: true, Node Label Expression: } 2016-05-05 17:27:14,250 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(871)) - appattempt_1462440433865_0002_000001 State change from SUBMITTED to SCHEDULED 2016-05-05 17:27:14,257 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(192)) - App State is : ACCEPTED 2016-05-05 17:27:14,257 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(265)) - Attempt State is : SCHEDULED 2016-05-05 17:27:14,257 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(192)) - App State is : ACCEPTED 2016-05-05 17:27:14,257 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(265)) - Attempt State is : SCHEDULED Launch AM appattempt_1462440433865_0002_000001 2016-05-05 17:27:14,258 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2016-05-05 17:27:14,258 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(618)) - Processing 127.0.0.1:1234 of type STATUS_UPDATE 2016-05-05 17:27:14,258 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(259)) - AppAttempt : appattempt_1462440433865_0002_000001 State is : SCHEDULED Waiting for state : ALLOCATED 2016-05-05 17:27:14,258 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2016-05-05 17:27:14,258 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1052)) - nodeUpdate: 127.0.0.1:1234 clusterResources: 2016-05-05 17:27:14,258 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1125)) - Node being looked for scheduling 127.0.0.1:1234 availableResource: 2016-05-05 17:27:14,258 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1276)) - Trying to schedule on node: 127.0.0.1, available: 2016-05-05 17:27:14,258 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(416)) - Trying to assign containers to child-queue of root 2016-05-05 17:27:14,258 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:printChildQueues(634)) - printChildQueues - queue: root child-queues: root.defaultusedCapacity=(0.0), label=(*) 2016-05-05 17:27:14,259 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(581)) - Trying to assign to queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=1, numContainers=0 2016-05-05 17:27:14,259 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:assignContainers(872)) - assignContainers: node=127.0.0.1 #applications=1 2016-05-05 17:27:14,259 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1190)) - User limit computation for moonfang in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-05-05 17:27:14,259 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimitAndSetHeadroom(1079)) - Headroom calculation for user moonfang: userLimit= queueMaxAvailRes= consumed= headroom= 2016-05-05 17:27:14,259 DEBUG [SchedulerEventDispatcher:Event Processor] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:assignContainers(496)) - pre-assignContainers for application application_1462440433865_0002 2016-05-05 17:27:14,259 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1190)) - User limit computation for moonfang in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-05-05 17:27:14,259 DEBUG [SchedulerEventDispatcher:Event Processor] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(480)) - showRequests: application=application_1462440433865_0002 headRoom= currentConsumption=0 2016-05-05 17:27:14,259 DEBUG [SchedulerEventDispatcher:Event Processor] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(484)) - showRequests: application=application_1462440433865_0002 request={Priority: 0, Capability: , # Containers: 1, Location: *, Relax Locality: true, Node Label Expression: } 2016-05-05 17:27:14,259 DEBUG [SchedulerEventDispatcher:Event Processor] allocator.IncreaseContainerAllocator (IncreaseContainerAllocator.java:assignContainers(184)) - Skip allocating increase request since we don't have any increase request on this node=127.0.0.1:1234 2016-05-05 17:27:14,259 DEBUG [SchedulerEventDispatcher:Event Processor] allocator.RegularContainerAllocator (RegularContainerAllocator.java:assignContainer(412)) - assignContainers: node=127.0.0.1 application=application_1462440433865_0002 priority=0 request={Priority: 0, Capability: , # Containers: 1, Location: *, Relax Locality: true, Node Label Expression: } type=OFF_SWITCH 2016-05-05 17:27:14,260 DEBUG [SchedulerEventDispatcher:Event Processor] scheduler.ActiveUsersManager (ActiveUsersManager.java:deactivateApplication(94)) - User moonfang removed from activeUsers, currently: 0 2016-05-05 17:27:14,260 DEBUG [SchedulerEventDispatcher:Event Processor] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:allocate(607)) - allocate: applicationId=application_1462440433865_0002 container=container_1462440433865_0002_01_000001 host=127.0.0.1:1234 user=moonfang resource= type=OFF_SWITCH 2016-05-05 17:27:14,260 DEBUG [SchedulerEventDispatcher:Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(439)) - Processing container_1462440433865_0002_01_000001 of type START 2016-05-05 17:27:14,260 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: CONTAINER_ALLOCATED 2016-05-05 17:27:14,260 INFO [SchedulerEventDispatcher:Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(453)) - container_1462440433865_0002_01_000001 Container Transitioned from NEW to ALLOCATED 2016-05-05 17:27:14,260 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(859)) - Processing event for appattempt_1462440433865_0002_000001 of type CONTAINER_ALLOCATED 2016-05-05 17:27:14,260 DEBUG [SchedulerEventDispatcher:Event Processor] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:allocate(225)) - allocate: applicationAttemptId=appattempt_1462440433865_0002_000001 container=container_1462440433865_0002_01_000001 host=127.0.0.1 type=OFF_SWITCH 2016-05-05 17:27:14,260 INFO [SchedulerEventDispatcher:Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(154)) - USER=moonfang OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1462440433865_0002 CONTAINERID=container_1462440433865_0002_01_000001 RESOURCE= 2016-05-05 17:27:14,260 DEBUG [SchedulerEventDispatcher:Event Processor] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(158)) - Assigned container container_1462440433865_0002_01_000001 of capacity on host 127.0.0.1:1234, which has 1 containers, used and available after allocation 2016-05-05 17:27:14,260 INFO [SchedulerEventDispatcher:Event Processor] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(89)) - assignedContainer application attempt=appattempt_1462440433865_0002_000001 container=container_1462440433865_0002_01_000001 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@3c77665b clusterResource= type=OFF_SWITCH 2016-05-05 17:27:14,261 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:assignContainers(941)) - post-assignContainers for application application_1462440433865_0002 2016-05-05 17:27:14,261 DEBUG [AsyncDispatcher event handler] security.BaseContainerTokenSecretManager (BaseContainerTokenSecretManager.java:createPassword(90)) - Creating password for container_1462440433865_0002_01_000001 for user container_1462440433865_0002_01_000001 (auth:SIMPLE) to be run on NM 127.0.0.1:1234 2016-05-05 17:27:14,261 DEBUG [AsyncDispatcher event handler] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 2 cluster_timestamp: 1462440433865 } attemptId: 1 } id: 1 } nmHostAddr: "127.0.0.1:1234" appSubmitter: "moonfang" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1462441034261 masterKeyId: -1802453347 rmIdentifier: 1462440433865 priority { priority: 0 } creationTime: 1462440434260 nodeLabelExpression: "" containerType: APPLICATION_MASTER executionType: GUARANTEED 2016-05-05 17:27:14,262 DEBUG [AsyncDispatcher event handler] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 2 cluster_timestamp: 1462440433865 } attemptId: 1 } id: 1 } nmHostAddr: "127.0.0.1:1234" appSubmitter: "moonfang" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1462441034261 masterKeyId: -1802453347 rmIdentifier: 1462440433865 priority { priority: 0 } creationTime: 1462440434260 nodeLabelExpression: "" containerType: APPLICATION_MASTER executionType: GUARANTEED 2016-05-05 17:27:14,262 INFO [AsyncDispatcher event handler] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:createAndGetNMToken(200)) - Sending NMToken for nodeId : 127.0.0.1:1234 for container : container_1462440433865_0002_01_000001 2016-05-05 17:27:14,262 DEBUG [AsyncDispatcher event handler] security.BaseNMTokenSecretManager (BaseNMTokenSecretManager.java:createPassword(75)) - creating password for appattempt_1462440433865_0002_000001 for user moonfang to run on NM 127.0.0.1:1234 2016-05-05 17:27:14,262 DEBUG [AsyncDispatcher event handler] security.NMTokenIdentifier (NMTokenIdentifier.java:write(96)) - Writing NMTokenIdentifier to RPC layer: appAttemptId { application_id { id: 2 cluster_timestamp: 1462440433865 } attemptId: 1 } nodeId { host: "127.0.0.1" port: 1234 } appSubmitter: "moonfang" keyId: -790324192 2016-05-05 17:27:14,262 DEBUG [AsyncDispatcher event handler] security.NMTokenIdentifier (NMTokenIdentifier.java:write(96)) - Writing NMTokenIdentifier to RPC layer: appAttemptId { application_id { id: 2 cluster_timestamp: 1462440433865 } attemptId: 1 } nodeId { host: "127.0.0.1" port: 1234 } appSubmitter: "moonfang" keyId: -790324192 2016-05-05 17:27:14,263 DEBUG [AsyncDispatcher event handler] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(439)) - Processing container_1462440433865_0002_01_000001 of type ACQUIRED 2016-05-05 17:27:14,263 INFO [AsyncDispatcher event handler] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(453)) - container_1462440433865_0002_01_000001 Container Transitioned from ALLOCATED to ACQUIRED 2016-05-05 17:27:14,263 DEBUG [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1190)) - User limit computation for moonfang in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-05-05 17:27:14,263 INFO [AsyncDispatcher event handler] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:clearNodeSetForAttempt(146)) - Clear node set for appattempt_1462440433865_0002_000001 2016-05-05 17:27:14,263 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1190)) - User limit computation for moonfang in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-05-05 17:27:14,263 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:storeAttempt(2075)) - Storing attempt: AppId: application_1462440433865_0002 AttemptId: appattempt_1462440433865_0002_000001 MasterContainer: Container: [ContainerId: container_1462440433865_0002_01_000001, NodeId: 127.0.0.1:1234, NodeHttpAddress: 127.0.0.1:2, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 127.0.0.1:1234 }, ] 2016-05-05 17:27:14,263 DEBUG [SchedulerEventDispatcher:Event Processor] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(480)) - showRequests: application=application_1462440433865_0002 headRoom= currentConsumption=1024 2016-05-05 17:27:14,263 DEBUG [SchedulerEventDispatcher:Event Processor] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(484)) - showRequests: application=application_1462440433865_0002 request={Priority: 0, Capability: , # Containers: 0, Location: *, Relax Locality: true, Node Label Expression: } 2016-05-05 17:27:14,264 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(871)) - appattempt_1462440433865_0002_000001 State change from SCHEDULED to ALLOCATED_SAVING 2016-05-05 17:27:14,264 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStoreAppAttemptEvent.EventType: STORE_APP_ATTEMPT 2016-05-05 17:27:14,264 DEBUG [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:handleStoreEvent(1070)) - Processing event of type STORE_APP_ATTEMPT 2016-05-05 17:27:14,264 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppRunningOnNodeEvent.EventType: APP_RUNNING_ON_NODE 2016-05-05 17:27:14,264 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1190)) - User limit computation for moonfang in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-05-05 17:27:14,264 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(811)) - Processing event for application_1462440433865_0002 of type APP_RUNNING_ON_NODE 2016-05-05 17:27:14,264 DEBUG [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:transition(300)) - Storing info for attempt: appattempt_1462440433865_0002_000001 2016-05-05 17:27:14,264 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1190)) - User limit computation for moonfang in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-05-05 17:27:14,264 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: ATTEMPT_NEW_SAVED 2016-05-05 17:27:14,264 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(859)) - Processing event for appattempt_1462440433865_0002_000001 of type ATTEMPT_NEW_SAVED 2016-05-05 17:27:14,264 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1190)) - User limit computation for moonfang in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-05-05 17:27:14,264 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(871)) - appattempt_1462440433865_0002_000001 State change from ALLOCATED_SAVING to ALLOCATED 2016-05-05 17:27:14,264 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:allocateResource(1401)) - default user=moonfang used= numContainers=1 headroom = user-resources= 2016-05-05 17:27:14,265 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncherEvent.EventType: LAUNCH 2016-05-05 17:27:14,265 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(592)) - Assigned to queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.128, absoluteUsedCapacity=0.128, numApps=1, numContainers=1 --> , OFF_SWITCH 2016-05-05 17:27:14,265 INFO [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(606)) - Re-sorting assigned queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.128, absoluteUsedCapacity=0.128, numApps=1, numContainers=1 2016-05-05 17:27:14,265 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:printChildQueues(634)) - printChildQueues - queue: root child-queues: root.defaultusedCapacity=(0.128), label=(*) 2016-05-05 17:27:14,265 INFO [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(469)) - assignedContainer queue=root usedCapacity=0.128 absoluteUsedCapacity=0.128 used= cluster= 2016-05-05 17:27:14,265 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(481)) - ParentQ=root assignedSoFarInThisIteration= usedCapacity=0.128 absoluteUsedCapacity=0.128 2016-05-05 17:27:14,265 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(492)) - Not assigning more than one off-switch container, assignments so far: resource:; type:OFF_SWITCH; excessReservation:null; applicationid:null; skipped:false; fulfilled reservation:false; allocations(count/resource):1/; reservations(count/resource):0/ 2016-05-05 17:27:14,268 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(265)) - Attempt State is : ALLOCATED 2016-05-05 17:27:14,268 INFO [Thread-0] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:createAndGetAMRMToken(195)) - Create AMRMToken for ApplicationAttempt: appattempt_1462440433865_0002_000001 2016-05-05 17:27:14,268 INFO [Thread-0] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:createPassword(307)) - Creating password for appattempt_1462440433865_0002_000001 2016-05-05 17:27:14,268 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(259)) - AppAttempt : appattempt_1462440433865_0002_000001 State is : ALLOCATED Waiting for state : LAUNCHED 2016-05-05 17:27:14,268 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: LAUNCHED 2016-05-05 17:27:14,268 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(859)) - Processing event for appattempt_1462440433865_0002_000001 of type LAUNCHED 2016-05-05 17:27:14,269 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(871)) - appattempt_1462440433865_0002_000001 State change from ALLOCATED to LAUNCHED 2016-05-05 17:27:14,279 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(265)) - Attempt State is : LAUNCHED 2016-05-05 17:27:14,279 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(265)) - Attempt State is : LAUNCHED 2016-05-05 17:27:14,279 DEBUG [Thread-0] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1780)) - PrivilegedAction as:appattempt_1462440433865_0002_000001 (auth:SIMPLE) from:org.apache.hadoop.yarn.server.resourcemanager.MockAM.registerAppAttempt(MockAM.java:116) 2016-05-05 17:27:14,279 INFO [Thread-0] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:registerApplicationMaster(235)) - AM registration appattempt_1462440433865_0002_000001 2016-05-05 17:27:14,279 INFO [Thread-0] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(175)) - USER=moonfang OPERATION=Register App Master TARGET=ApplicationMasterService RESULT=SUCCESS APPID=application_1462440433865_0002 APPATTEMPTID=appattempt_1462440433865_0002_000001 2016-05-05 17:27:14,279 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptRegistrationEvent.EventType: REGISTERED 2016-05-05 17:27:14,279 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(186)) - App : application_1462440433865_0002 State is : ACCEPTED Waiting for state : RUNNING 2016-05-05 17:27:14,279 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(859)) - Processing event for appattempt_1462440433865_0002_000001 of type REGISTERED 2016-05-05 17:27:14,280 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(871)) - appattempt_1462440433865_0002_000001 State change from LAUNCHED to RUNNING 2016-05-05 17:27:14,280 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEvent.EventType: ATTEMPT_REGISTERED 2016-05-05 17:27:14,280 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(811)) - Processing event for application_1462440433865_0002 of type ATTEMPT_REGISTERED 2016-05-05 17:27:14,280 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(823)) - application_1462440433865_0002 State change from ACCEPTED to RUNNING 2016-05-05 17:27:14,290 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(192)) - App State is : RUNNING 2016-05-05 17:27:14,290 INFO [Thread-0] mortbay.log (Slf4jLog.java:info(67)) - ContainerStatus: ContainerStatus: [ContainerId: container_1462440433865_0002_01_000001, ExecutionType: GUARANTEED, State: COMPLETE, Capability: , Diagnostics: Success, ExitStatus: 0, ] 2016-05-05 17:27:14,290 INFO [Thread-0] mortbay.log (Slf4jLog.java:info(67)) - entry.getValue() [ContainerStatus: [ContainerId: container_1462440433865_0002_01_000001, ExecutionType: GUARANTEED, State: COMPLETE, Capability: , Diagnostics: Success, ExitStatus: 0, ]] 2016-05-05 17:27:14,290 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(259)) - AppAttempt : appattempt_1462440433865_0002_000001 State is : RUNNING Waiting for state : FAILED 2016-05-05 17:27:14,290 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2016-05-05 17:27:14,291 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(618)) - Processing 127.0.0.1:1234 of type STATUS_UPDATE 2016-05-05 17:27:14,291 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handleContainerStatus(1326)) - Container container_1462440433865_0002_01_000001 is the first container get launched for application application_1462440433865_0002 2016-05-05 17:27:14,291 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2016-05-05 17:27:14,291 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1052)) - nodeUpdate: 127.0.0.1:1234 clusterResources: 2016-05-05 17:27:14,291 DEBUG [SchedulerEventDispatcher:Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(439)) - Processing container_1462440433865_0002_01_000001 of type FINISHED 2016-05-05 17:27:14,291 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptContainerFinishedEvent.EventType: CONTAINER_FINISHED 2016-05-05 17:27:14,291 INFO [SchedulerEventDispatcher:Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(453)) - container_1462440433865_0002_01_000001 Container Transitioned from ACQUIRED to COMPLETED 2016-05-05 17:27:14,291 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(859)) - Processing event for appattempt_1462440433865_0002_000001 of type CONTAINER_FINISHED 2016-05-05 17:27:14,291 INFO [SchedulerEventDispatcher:Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(154)) - USER=moonfang OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1462440433865_0002 CONTAINERID=container_1462440433865_0002_01_000001 RESOURCE= 2016-05-05 17:27:14,291 DEBUG [SchedulerEventDispatcher:Event Processor] scheduler.SchedulerNode (SchedulerNode.java:releaseContainer(269)) - Released container container_1462440433865_0002_01_000001 of capacity on host 127.0.0.1:1234, which currently has 0 containers, used and available, release resources=true 2016-05-05 17:27:14,292 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1190)) - User limit computation for moonfang in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-05-05 17:27:14,292 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:releaseResource(1437)) - default used= numContainers=0 user=moonfang user-resources= 2016-05-05 17:27:14,292 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:internalReleaseResource(647)) - completedContainer root: numChildQueue= 1, capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.0, numApps=1, numContainers=0, cluster= 2016-05-05 17:27:14,292 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:rememberTargetTransitionsAndStoreState(1301)) - Updating application attempt appattempt_1462440433865_0002_000001 with final state: FAILED, and exit status: 0 2016-05-05 17:27:14,292 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:internalReleaseResource(660)) - Re-sorting completed queue: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=1, numContainers=0 2016-05-05 17:27:14,292 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateUpdateAppAttemptEvent.EventType: UPDATE_APP_ATTEMPT 2016-05-05 17:27:14,292 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(871)) - appattempt_1462440433865_0002_000001 State change from RUNNING to FINAL_SAVING 2016-05-05 17:27:14,292 DEBUG [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:handleStoreEvent(1070)) - Processing event of type UPDATE_APP_ATTEMPT 2016-05-05 17:27:14,292 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1125)) - Node being looked for scheduling 127.0.0.1:1234 availableResource: 2016-05-05 17:27:14,292 DEBUG [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:transition(331)) - Updating info for attempt: appattempt_1462440433865_0002_000001 2016-05-05 17:27:14,292 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2016-05-05 17:27:14,292 INFO [AsyncDispatcher event handler] recovery.RMStateStore (MemoryRMStateStore.java:updateApplicationAttemptStateInternal(139)) - Updating final state FAILED for attempt: appattempt_1462440433865_0002_000001 2016-05-05 17:27:14,292 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1276)) - Trying to schedule on node: 127.0.0.1, available: 2016-05-05 17:27:14,292 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(618)) - Processing 127.0.0.1:1234 of type FINISHED_CONTAINERS_PULLED_BY_AM 2016-05-05 17:27:14,293 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(404)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-05-05 17:27:14,293 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: ATTEMPT_UPDATE_SAVED 2016-05-05 17:27:14,293 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(859)) - Processing event for appattempt_1462440433865_0002_000001 of type ATTEMPT_UPDATE_SAVED 2016-05-05 17:27:14,293 INFO [AsyncDispatcher event handler] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:unregisterAttempt(682)) - Unregistering app attempt : appattempt_1462440433865_0002_000001 2016-05-05 17:27:14,293 INFO [AsyncDispatcher event handler] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:applicationMasterFinished(124)) - Application finished, removing password for appattempt_1462440433865_0002_000001 2016-05-05 17:27:14,293 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(871)) - appattempt_1462440433865_0002_000001 State change from FINAL_SAVING to FAILED 2016-05-05 17:27:14,293 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppFailedAttemptEvent.EventType: ATTEMPT_FAILED 2016-05-05 17:27:14,293 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(811)) - Processing event for application_1462440433865_0002 of type ATTEMPT_FAILED 2016-05-05 17:27:14,293 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:transition(1354)) - The number of failed attempts in previous 10000 milliseconds is 1. The max attempts is 2 2016-05-05 17:27:14,294 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(823)) - application_1462440433865_0002 State change from RUNNING to ACCEPTED 2016-05-05 17:27:14,294 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.AppAttemptRemovedSchedulerEvent.EventType: APP_ATTEMPT_REMOVED 2016-05-05 17:27:14,294 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncherEvent.EventType: CLEANUP 2016-05-05 17:27:14,294 INFO [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:doneApplicationAttempt(864)) - Application Attempt appattempt_1462440433865_0002_000001 is done. finalState=FAILED 2016-05-05 17:27:14,294 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppStartAttemptEvent.EventType: START 2016-05-05 17:27:14,294 INFO [SchedulerEventDispatcher:Event Processor] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:clearRequests(135)) - Application application_1462440433865_0002 requests cleared 2016-05-05 17:27:14,294 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(859)) - Processing event for appattempt_1462440433865_0002_000002 of type START 2016-05-05 17:27:14,294 INFO [AsyncDispatcher event handler] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:registerAppAttempt(676)) - Registering app attempt : appattempt_1462440433865_0002_000002 2016-05-05 17:27:14,294 INFO [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:removeApplicationAttempt(793)) - Application removed - appId: application_1462440433865_0002 user: moonfang queue: default #user-pending-applications: 0 #user-active-applications: 0 #queue-pending-applications: 0 #queue-active-applications: 0 2016-05-05 17:27:14,294 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(871)) - appattempt_1462440433865_0002_000002 State change from NEW to SUBMITTED 2016-05-05 17:27:14,295 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.AppAttemptAddedSchedulerEvent.EventType: APP_ATTEMPT_ADDED 2016-05-05 17:27:14,295 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:activateApplications(652)) - application application_1462440433865_0002 AMResource maxAMResourcePerQueuePercent 0.1 amLimit lastClusterResource amIfStarted AM node-partition name 2016-05-05 17:27:14,295 INFO [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:activateApplications(726)) - Application application_1462440433865_0002 from user: moonfang activated in queue: default 2016-05-05 17:27:14,295 INFO [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:addApplicationAttempt(741)) - Application added - appId: application_1462440433865_0002 user: moonfang, leaf-queue: default #user-pending-applications: 0 #user-active-applications: 1 #queue-pending-applications: 0 #queue-active-applications: 1 2016-05-05 17:27:14,295 INFO [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:addApplicationAttempt(825)) - Added Application Attempt appattempt_1462440433865_0002_000002 to scheduler from user moonfang in queue default 2016-05-05 17:27:14,295 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: ATTEMPT_ADDED 2016-05-05 17:27:14,295 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(859)) - Processing event for appattempt_1462440433865_0002_000002 of type ATTEMPT_ADDED 2016-05-05 17:27:14,295 DEBUG [AsyncDispatcher event handler] blacklist.SimpleBlacklistManager (SimpleBlacklistManager.java:getBlacklistUpdates(69)) - blacklist size 0 is less than failure threshold ratio 0.800000011920929 out of total usable nodes 1 2016-05-05 17:27:14,295 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:transition(1041)) - Using blacklist for AM: additions([]) and removals([]) 2016-05-05 17:27:14,296 DEBUG [AsyncDispatcher event handler] capacity.CapacityScheduler (CapacityScheduler.java:allocate(984)) - allocate: pre-update appattempt_1462440433865_0002_000002 ask size =1 2016-05-05 17:27:14,296 DEBUG [AsyncDispatcher event handler] scheduler.ActiveUsersManager (ActiveUsersManager.java:activateApplication(68)) - User moonfang added to activeUsers, currently: 1 2016-05-05 17:27:14,296 DEBUG [AsyncDispatcher event handler] capacity.CapacityScheduler (CapacityScheduler.java:allocate(996)) - allocate: post-update 2016-05-05 17:27:14,296 DEBUG [AsyncDispatcher event handler] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(480)) - showRequests: application=application_1462440433865_0002 headRoom= currentConsumption=0 2016-05-05 17:27:14,296 DEBUG [AsyncDispatcher event handler] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(484)) - showRequests: application=application_1462440433865_0002 request={Priority: 0, Capability: , # Containers: 1, Location: *, Relax Locality: true, Node Label Expression: } 2016-05-05 17:27:14,296 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(871)) - appattempt_1462440433865_0002_000002 State change from SUBMITTED to SCHEDULED 2016-05-05 17:27:14,301 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(265)) - Attempt State is : FAILED 2016-05-05 17:27:29,301 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(192)) - App State is : ACCEPTED 2016-05-05 17:27:29,301 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(192)) - App State is : ACCEPTED 2016-05-05 17:27:29,301 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(265)) - Attempt State is : SCHEDULED Launch AM appattempt_1462440433865_0002_000002 2016-05-05 17:27:29,301 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(259)) - AppAttempt : appattempt_1462440433865_0002_000002 State is : SCHEDULED Waiting for state : ALLOCATED 2016-05-05 17:27:29,301 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2016-05-05 17:27:29,302 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(618)) - Processing 127.0.0.1:1234 of type STATUS_UPDATE 2016-05-05 17:27:29,302 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2016-05-05 17:27:29,302 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1052)) - nodeUpdate: 127.0.0.1:1234 clusterResources: 2016-05-05 17:27:29,302 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1125)) - Node being looked for scheduling 127.0.0.1:1234 availableResource: 2016-05-05 17:27:29,302 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1276)) - Trying to schedule on node: 127.0.0.1, available: 2016-05-05 17:27:29,302 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(416)) - Trying to assign containers to child-queue of root 2016-05-05 17:27:29,302 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:printChildQueues(634)) - printChildQueues - queue: root child-queues: root.defaultusedCapacity=(0.0), label=(*) 2016-05-05 17:27:29,302 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(581)) - Trying to assign to queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=1, numContainers=0 2016-05-05 17:27:29,302 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:assignContainers(872)) - assignContainers: node=127.0.0.1 #applications=1 2016-05-05 17:27:29,303 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1190)) - User limit computation for moonfang in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-05-05 17:27:29,303 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimitAndSetHeadroom(1079)) - Headroom calculation for user moonfang: userLimit= queueMaxAvailRes= consumed= headroom= 2016-05-05 17:27:29,303 DEBUG [SchedulerEventDispatcher:Event Processor] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:assignContainers(496)) - pre-assignContainers for application application_1462440433865_0002 2016-05-05 17:27:29,303 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1190)) - User limit computation for moonfang in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-05-05 17:27:29,303 DEBUG [SchedulerEventDispatcher:Event Processor] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(480)) - showRequests: application=application_1462440433865_0002 headRoom= currentConsumption=0 2016-05-05 17:27:29,303 DEBUG [SchedulerEventDispatcher:Event Processor] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(484)) - showRequests: application=application_1462440433865_0002 request={Priority: 0, Capability: , # Containers: 1, Location: *, Relax Locality: true, Node Label Expression: } 2016-05-05 17:27:29,303 DEBUG [SchedulerEventDispatcher:Event Processor] allocator.IncreaseContainerAllocator (IncreaseContainerAllocator.java:assignContainers(184)) - Skip allocating increase request since we don't have any increase request on this node=127.0.0.1:1234 2016-05-05 17:27:29,303 DEBUG [SchedulerEventDispatcher:Event Processor] allocator.RegularContainerAllocator (RegularContainerAllocator.java:assignContainer(412)) - assignContainers: node=127.0.0.1 application=application_1462440433865_0002 priority=0 request={Priority: 0, Capability: , # Containers: 1, Location: *, Relax Locality: true, Node Label Expression: } type=OFF_SWITCH 2016-05-05 17:27:29,303 DEBUG [SchedulerEventDispatcher:Event Processor] scheduler.ActiveUsersManager (ActiveUsersManager.java:deactivateApplication(94)) - User moonfang removed from activeUsers, currently: 0 2016-05-05 17:27:29,304 DEBUG [SchedulerEventDispatcher:Event Processor] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:allocate(607)) - allocate: applicationId=application_1462440433865_0002 container=container_1462440433865_0002_02_000001 host=127.0.0.1:1234 user=moonfang resource= type=OFF_SWITCH 2016-05-05 17:27:29,304 DEBUG [SchedulerEventDispatcher:Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(439)) - Processing container_1462440433865_0002_02_000001 of type START 2016-05-05 17:27:29,304 INFO [SchedulerEventDispatcher:Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(453)) - container_1462440433865_0002_02_000001 Container Transitioned from NEW to ALLOCATED 2016-05-05 17:27:29,304 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: CONTAINER_ALLOCATED 2016-05-05 17:27:29,304 DEBUG [SchedulerEventDispatcher:Event Processor] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:allocate(225)) - allocate: applicationAttemptId=appattempt_1462440433865_0002_000002 container=container_1462440433865_0002_02_000001 host=127.0.0.1 type=OFF_SWITCH 2016-05-05 17:27:29,304 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(859)) - Processing event for appattempt_1462440433865_0002_000002 of type CONTAINER_ALLOCATED 2016-05-05 17:27:29,304 INFO [SchedulerEventDispatcher:Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(154)) - USER=moonfang OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1462440433865_0002 CONTAINERID=container_1462440433865_0002_02_000001 RESOURCE= 2016-05-05 17:27:29,304 DEBUG [SchedulerEventDispatcher:Event Processor] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(158)) - Assigned container container_1462440433865_0002_02_000001 of capacity on host 127.0.0.1:1234, which has 1 containers, used and available after allocation 2016-05-05 17:27:29,304 INFO [SchedulerEventDispatcher:Event Processor] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(89)) - assignedContainer application attempt=appattempt_1462440433865_0002_000002 container=container_1462440433865_0002_02_000001 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@1c75afe5 clusterResource= type=OFF_SWITCH 2016-05-05 17:27:29,304 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:assignContainers(941)) - post-assignContainers for application application_1462440433865_0002 2016-05-05 17:27:29,305 DEBUG [AsyncDispatcher event handler] security.BaseContainerTokenSecretManager (BaseContainerTokenSecretManager.java:createPassword(90)) - Creating password for container_1462440433865_0002_02_000001 for user container_1462440433865_0002_02_000001 (auth:SIMPLE) to be run on NM 127.0.0.1:1234 2016-05-05 17:27:29,305 DEBUG [AsyncDispatcher event handler] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 2 cluster_timestamp: 1462440433865 } attemptId: 2 } id: 1 } nmHostAddr: "127.0.0.1:1234" appSubmitter: "moonfang" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1462441049304 masterKeyId: -1802453347 rmIdentifier: 1462440433865 priority { priority: 0 } creationTime: 1462440449303 nodeLabelExpression: "" containerType: APPLICATION_MASTER executionType: GUARANTEED 2016-05-05 17:27:29,305 DEBUG [AsyncDispatcher event handler] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 2 cluster_timestamp: 1462440433865 } attemptId: 2 } id: 1 } nmHostAddr: "127.0.0.1:1234" appSubmitter: "moonfang" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1462441049304 masterKeyId: -1802453347 rmIdentifier: 1462440433865 priority { priority: 0 } creationTime: 1462440449303 nodeLabelExpression: "" containerType: APPLICATION_MASTER executionType: GUARANTEED 2016-05-05 17:27:29,306 INFO [AsyncDispatcher event handler] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:createAndGetNMToken(200)) - Sending NMToken for nodeId : 127.0.0.1:1234 for container : container_1462440433865_0002_02_000001 2016-05-05 17:27:29,306 DEBUG [AsyncDispatcher event handler] security.BaseNMTokenSecretManager (BaseNMTokenSecretManager.java:createPassword(75)) - creating password for appattempt_1462440433865_0002_000002 for user moonfang to run on NM 127.0.0.1:1234 2016-05-05 17:27:29,306 DEBUG [AsyncDispatcher event handler] security.NMTokenIdentifier (NMTokenIdentifier.java:write(96)) - Writing NMTokenIdentifier to RPC layer: appAttemptId { application_id { id: 2 cluster_timestamp: 1462440433865 } attemptId: 2 } nodeId { host: "127.0.0.1" port: 1234 } appSubmitter: "moonfang" keyId: -790324192 2016-05-05 17:27:29,308 DEBUG [AsyncDispatcher event handler] security.NMTokenIdentifier (NMTokenIdentifier.java:write(96)) - Writing NMTokenIdentifier to RPC layer: appAttemptId { application_id { id: 2 cluster_timestamp: 1462440433865 } attemptId: 2 } nodeId { host: "127.0.0.1" port: 1234 } appSubmitter: "moonfang" keyId: -790324192 2016-05-05 17:27:29,308 DEBUG [AsyncDispatcher event handler] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(439)) - Processing container_1462440433865_0002_02_000001 of type ACQUIRED 2016-05-05 17:27:29,309 INFO [AsyncDispatcher event handler] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(453)) - container_1462440433865_0002_02_000001 Container Transitioned from ALLOCATED to ACQUIRED 2016-05-05 17:27:29,309 DEBUG [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1190)) - User limit computation for moonfang in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-05-05 17:27:29,309 INFO [AsyncDispatcher event handler] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:clearNodeSetForAttempt(146)) - Clear node set for appattempt_1462440433865_0002_000002 2016-05-05 17:27:29,309 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1190)) - User limit computation for moonfang in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-05-05 17:27:29,309 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:storeAttempt(2075)) - Storing attempt: AppId: application_1462440433865_0002 AttemptId: appattempt_1462440433865_0002_000002 MasterContainer: Container: [ContainerId: container_1462440433865_0002_02_000001, NodeId: 127.0.0.1:1234, NodeHttpAddress: 127.0.0.1:2, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 127.0.0.1:1234 }, ] 2016-05-05 17:27:29,309 DEBUG [SchedulerEventDispatcher:Event Processor] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(480)) - showRequests: application=application_1462440433865_0002 headRoom= currentConsumption=1024 2016-05-05 17:27:29,309 DEBUG [SchedulerEventDispatcher:Event Processor] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(484)) - showRequests: application=application_1462440433865_0002 request={Priority: 0, Capability: , # Containers: 0, Location: *, Relax Locality: true, Node Label Expression: } 2016-05-05 17:27:29,310 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStoreAppAttemptEvent.EventType: STORE_APP_ATTEMPT 2016-05-05 17:27:29,310 DEBUG [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:handleStoreEvent(1070)) - Processing event of type STORE_APP_ATTEMPT 2016-05-05 17:27:29,310 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(871)) - appattempt_1462440433865_0002_000002 State change from SCHEDULED to ALLOCATED_SAVING 2016-05-05 17:27:29,310 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1190)) - User limit computation for moonfang in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-05-05 17:27:29,310 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppRunningOnNodeEvent.EventType: APP_RUNNING_ON_NODE 2016-05-05 17:27:29,310 DEBUG [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:transition(300)) - Storing info for attempt: appattempt_1462440433865_0002_000002 2016-05-05 17:27:29,310 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1190)) - User limit computation for moonfang in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-05-05 17:27:29,310 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(811)) - Processing event for application_1462440433865_0002 of type APP_RUNNING_ON_NODE 2016-05-05 17:27:29,310 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: ATTEMPT_NEW_SAVED 2016-05-05 17:27:29,310 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1190)) - User limit computation for moonfang in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-05-05 17:27:29,310 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(859)) - Processing event for appattempt_1462440433865_0002_000002 of type ATTEMPT_NEW_SAVED 2016-05-05 17:27:29,310 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:allocateResource(1401)) - default user=moonfang used= numContainers=1 headroom = user-resources= 2016-05-05 17:27:29,310 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(871)) - appattempt_1462440433865_0002_000002 State change from ALLOCATED_SAVING to ALLOCATED 2016-05-05 17:27:29,310 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(592)) - Assigned to queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.128, absoluteUsedCapacity=0.128, numApps=1, numContainers=1 --> , OFF_SWITCH 2016-05-05 17:27:29,310 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncherEvent.EventType: LAUNCH 2016-05-05 17:27:29,310 INFO [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(606)) - Re-sorting assigned queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.128, absoluteUsedCapacity=0.128, numApps=1, numContainers=1 2016-05-05 17:27:29,311 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:printChildQueues(634)) - printChildQueues - queue: root child-queues: root.defaultusedCapacity=(0.128), label=(*) 2016-05-05 17:27:29,311 INFO [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(469)) - assignedContainer queue=root usedCapacity=0.128 absoluteUsedCapacity=0.128 used= cluster= 2016-05-05 17:27:29,311 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(481)) - ParentQ=root assignedSoFarInThisIteration= usedCapacity=0.128 absoluteUsedCapacity=0.128 2016-05-05 17:27:29,311 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(492)) - Not assigning more than one off-switch container, assignments so far: resource:; type:OFF_SWITCH; excessReservation:null; applicationid:null; skipped:false; fulfilled reservation:false; allocations(count/resource):1/; reservations(count/resource):0/ 2016-05-05 17:27:29,312 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(265)) - Attempt State is : ALLOCATED 2016-05-05 17:27:29,312 INFO [Thread-0] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:createAndGetAMRMToken(195)) - Create AMRMToken for ApplicationAttempt: appattempt_1462440433865_0002_000002 2016-05-05 17:27:29,312 INFO [Thread-0] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:createPassword(307)) - Creating password for appattempt_1462440433865_0002_000002 2016-05-05 17:27:29,312 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: LAUNCHED 2016-05-05 17:27:29,312 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(259)) - AppAttempt : appattempt_1462440433865_0002_000002 State is : ALLOCATED Waiting for state : LAUNCHED 2016-05-05 17:27:29,312 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(859)) - Processing event for appattempt_1462440433865_0002_000002 of type LAUNCHED 2016-05-05 17:27:29,312 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(871)) - appattempt_1462440433865_0002_000002 State change from ALLOCATED to LAUNCHED 2016-05-05 17:27:29,322 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(265)) - Attempt State is : LAUNCHED 2016-05-05 17:27:29,322 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(265)) - Attempt State is : LAUNCHED 2016-05-05 17:27:29,322 DEBUG [Thread-0] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1780)) - PrivilegedAction as:appattempt_1462440433865_0002_000002 (auth:SIMPLE) from:org.apache.hadoop.yarn.server.resourcemanager.MockAM.registerAppAttempt(MockAM.java:116) 2016-05-05 17:27:29,323 INFO [Thread-0] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:registerApplicationMaster(235)) - AM registration appattempt_1462440433865_0002_000002 2016-05-05 17:27:29,323 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptRegistrationEvent.EventType: REGISTERED 2016-05-05 17:27:29,323 INFO [Thread-0] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(175)) - USER=moonfang OPERATION=Register App Master TARGET=ApplicationMasterService RESULT=SUCCESS APPID=application_1462440433865_0002 APPATTEMPTID=appattempt_1462440433865_0002_000002 2016-05-05 17:27:29,323 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(859)) - Processing event for appattempt_1462440433865_0002_000002 of type REGISTERED 2016-05-05 17:27:29,323 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(186)) - App : application_1462440433865_0002 State is : ACCEPTED Waiting for state : RUNNING 2016-05-05 17:27:29,323 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(871)) - appattempt_1462440433865_0002_000002 State change from LAUNCHED to RUNNING 2016-05-05 17:27:29,323 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEvent.EventType: ATTEMPT_REGISTERED 2016-05-05 17:27:29,323 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(811)) - Processing event for application_1462440433865_0002 of type ATTEMPT_REGISTERED 2016-05-05 17:27:29,323 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(823)) - application_1462440433865_0002 State change from ACCEPTED to RUNNING 2016-05-05 17:27:29,333 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(192)) - App State is : RUNNING 2016-05-05 17:27:29,333 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(265)) - Attempt State is : RUNNING 2016-05-05 17:27:29,333 INFO [Thread-0] mortbay.log (Slf4jLog.java:info(67)) - ContainerStatus: ContainerStatus: [ContainerId: container_1462440433865_0002_02_000001, ExecutionType: GUARANTEED, State: COMPLETE, Capability: , Diagnostics: Success, ExitStatus: 0, ] 2016-05-05 17:27:29,334 INFO [Thread-0] mortbay.log (Slf4jLog.java:info(67)) - entry.getValue() [ContainerStatus: [ContainerId: container_1462440433865_0002_02_000001, ExecutionType: GUARANTEED, State: COMPLETE, Capability: , Diagnostics: Success, ExitStatus: 0, ]] 2016-05-05 17:27:29,334 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2016-05-05 17:27:29,334 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(259)) - AppAttempt : appattempt_1462440433865_0002_000002 State is : RUNNING Waiting for state : FAILED 2016-05-05 17:27:29,334 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(618)) - Processing 127.0.0.1:1234 of type STATUS_UPDATE 2016-05-05 17:27:29,334 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2016-05-05 17:27:29,334 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1052)) - nodeUpdate: 127.0.0.1:1234 clusterResources: 2016-05-05 17:27:29,334 DEBUG [SchedulerEventDispatcher:Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(439)) - Processing container_1462440433865_0002_02_000001 of type FINISHED 2016-05-05 17:27:29,335 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptContainerFinishedEvent.EventType: CONTAINER_FINISHED 2016-05-05 17:27:29,335 INFO [SchedulerEventDispatcher:Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(453)) - container_1462440433865_0002_02_000001 Container Transitioned from ACQUIRED to COMPLETED 2016-05-05 17:27:29,335 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(859)) - Processing event for appattempt_1462440433865_0002_000002 of type CONTAINER_FINISHED 2016-05-05 17:27:29,335 INFO [SchedulerEventDispatcher:Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(154)) - USER=moonfang OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1462440433865_0002 CONTAINERID=container_1462440433865_0002_02_000001 RESOURCE= 2016-05-05 17:27:29,335 DEBUG [SchedulerEventDispatcher:Event Processor] scheduler.SchedulerNode (SchedulerNode.java:releaseContainer(269)) - Released container container_1462440433865_0002_02_000001 of capacity on host 127.0.0.1:1234, which currently has 0 containers, used and available, release resources=true 2016-05-05 17:27:29,335 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1190)) - User limit computation for moonfang in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-05-05 17:27:29,335 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:releaseResource(1437)) - default used= numContainers=0 user=moonfang user-resources= 2016-05-05 17:27:29,335 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:internalReleaseResource(647)) - completedContainer root: numChildQueue= 1, capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.0, numApps=1, numContainers=0, cluster= 2016-05-05 17:27:29,335 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:rememberTargetTransitionsAndStoreState(1301)) - Updating application attempt appattempt_1462440433865_0002_000002 with final state: FAILED, and exit status: 0 2016-05-05 17:27:29,335 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:internalReleaseResource(660)) - Re-sorting completed queue: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=1, numContainers=0 2016-05-05 17:27:29,336 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(871)) - appattempt_1462440433865_0002_000002 State change from RUNNING to FINAL_SAVING 2016-05-05 17:27:29,336 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateUpdateAppAttemptEvent.EventType: UPDATE_APP_ATTEMPT 2016-05-05 17:27:29,336 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2016-05-05 17:27:29,336 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1125)) - Node being looked for scheduling 127.0.0.1:1234 availableResource: 2016-05-05 17:27:29,336 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(618)) - Processing 127.0.0.1:1234 of type FINISHED_CONTAINERS_PULLED_BY_AM 2016-05-05 17:27:29,336 DEBUG [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:handleStoreEvent(1070)) - Processing event of type UPDATE_APP_ATTEMPT 2016-05-05 17:27:29,336 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1276)) - Trying to schedule on node: 127.0.0.1, available: 2016-05-05 17:27:29,336 DEBUG [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:transition(331)) - Updating info for attempt: appattempt_1462440433865_0002_000002 2016-05-05 17:27:29,336 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(404)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-05-05 17:27:29,336 INFO [AsyncDispatcher event handler] recovery.RMStateStore (MemoryRMStateStore.java:updateApplicationAttemptStateInternal(139)) - Updating final state FAILED for attempt: appattempt_1462440433865_0002_000002 2016-05-05 17:27:29,336 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: ATTEMPT_UPDATE_SAVED 2016-05-05 17:27:29,336 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(859)) - Processing event for appattempt_1462440433865_0002_000002 of type ATTEMPT_UPDATE_SAVED 2016-05-05 17:27:29,336 INFO [AsyncDispatcher event handler] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:unregisterAttempt(682)) - Unregistering app attempt : appattempt_1462440433865_0002_000002 2016-05-05 17:27:29,336 INFO [AsyncDispatcher event handler] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:applicationMasterFinished(124)) - Application finished, removing password for appattempt_1462440433865_0002_000002 2016-05-05 17:27:29,337 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(871)) - appattempt_1462440433865_0002_000002 State change from FINAL_SAVING to FAILED 2016-05-05 17:27:29,337 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppFailedAttemptEvent.EventType: ATTEMPT_FAILED 2016-05-05 17:27:29,337 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(811)) - Processing event for application_1462440433865_0002 of type ATTEMPT_FAILED 2016-05-05 17:27:29,337 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:transition(1354)) - The number of failed attempts in previous 10000 milliseconds is 1. The max attempts is 2 2016-05-05 17:27:29,337 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(823)) - application_1462440433865_0002 State change from RUNNING to ACCEPTED 2016-05-05 17:27:29,337 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.AppAttemptRemovedSchedulerEvent.EventType: APP_ATTEMPT_REMOVED 2016-05-05 17:27:29,337 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncherEvent.EventType: CLEANUP 2016-05-05 17:27:29,337 INFO [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:doneApplicationAttempt(864)) - Application Attempt appattempt_1462440433865_0002_000002 is done. finalState=FAILED 2016-05-05 17:27:29,337 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppStartAttemptEvent.EventType: START 2016-05-05 17:27:29,337 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(859)) - Processing event for appattempt_1462440433865_0002_000003 of type START 2016-05-05 17:27:29,337 INFO [SchedulerEventDispatcher:Event Processor] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:clearRequests(135)) - Application application_1462440433865_0002 requests cleared 2016-05-05 17:27:29,338 INFO [AsyncDispatcher event handler] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:registerAppAttempt(676)) - Registering app attempt : appattempt_1462440433865_0002_000003 2016-05-05 17:27:29,338 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(871)) - appattempt_1462440433865_0002_000003 State change from NEW to SUBMITTED 2016-05-05 17:27:29,338 INFO [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:removeApplicationAttempt(793)) - Application removed - appId: application_1462440433865_0002 user: moonfang queue: default #user-pending-applications: 0 #user-active-applications: 0 #queue-pending-applications: 0 #queue-active-applications: 0 2016-05-05 17:27:29,338 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.AppAttemptAddedSchedulerEvent.EventType: APP_ATTEMPT_ADDED 2016-05-05 17:27:29,338 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:activateApplications(652)) - application application_1462440433865_0002 AMResource maxAMResourcePerQueuePercent 0.1 amLimit lastClusterResource amIfStarted AM node-partition name 2016-05-05 17:27:29,338 INFO [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:activateApplications(726)) - Application application_1462440433865_0002 from user: moonfang activated in queue: default 2016-05-05 17:27:29,338 INFO [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:addApplicationAttempt(741)) - Application added - appId: application_1462440433865_0002 user: moonfang, leaf-queue: default #user-pending-applications: 0 #user-active-applications: 1 #queue-pending-applications: 0 #queue-active-applications: 1 2016-05-05 17:27:29,338 INFO [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:addApplicationAttempt(825)) - Added Application Attempt appattempt_1462440433865_0002_000003 to scheduler from user moonfang in queue default 2016-05-05 17:27:29,338 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: ATTEMPT_ADDED 2016-05-05 17:27:29,339 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(859)) - Processing event for appattempt_1462440433865_0002_000003 of type ATTEMPT_ADDED 2016-05-05 17:27:29,339 DEBUG [AsyncDispatcher event handler] blacklist.SimpleBlacklistManager (SimpleBlacklistManager.java:getBlacklistUpdates(69)) - blacklist size 0 is less than failure threshold ratio 0.800000011920929 out of total usable nodes 1 2016-05-05 17:27:29,339 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:transition(1041)) - Using blacklist for AM: additions([]) and removals([]) 2016-05-05 17:27:29,339 DEBUG [AsyncDispatcher event handler] capacity.CapacityScheduler (CapacityScheduler.java:allocate(984)) - allocate: pre-update appattempt_1462440433865_0002_000003 ask size =1 2016-05-05 17:27:29,339 DEBUG [AsyncDispatcher event handler] scheduler.ActiveUsersManager (ActiveUsersManager.java:activateApplication(68)) - User moonfang added to activeUsers, currently: 1 2016-05-05 17:27:29,339 DEBUG [AsyncDispatcher event handler] capacity.CapacityScheduler (CapacityScheduler.java:allocate(996)) - allocate: post-update 2016-05-05 17:27:29,339 DEBUG [AsyncDispatcher event handler] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(480)) - showRequests: application=application_1462440433865_0002 headRoom= currentConsumption=0 2016-05-05 17:27:29,339 DEBUG [AsyncDispatcher event handler] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(484)) - showRequests: application=application_1462440433865_0002 request={Priority: 0, Capability: , # Containers: 1, Location: *, Relax Locality: true, Node Label Expression: } 2016-05-05 17:27:29,339 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(871)) - appattempt_1462440433865_0002_000003 State change from SUBMITTED to SCHEDULED 2016-05-05 17:27:29,344 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(265)) - Attempt State is : FAILED 2016-05-05 17:27:29,344 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(192)) - App State is : ACCEPTED 2016-05-05 17:27:29,344 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(192)) - App State is : ACCEPTED 2016-05-05 17:27:29,344 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(265)) - Attempt State is : SCHEDULED Launch AM appattempt_1462440433865_0002_000003 2016-05-05 17:27:29,345 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2016-05-05 17:27:29,345 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(259)) - AppAttempt : appattempt_1462440433865_0002_000003 State is : SCHEDULED Waiting for state : ALLOCATED 2016-05-05 17:27:29,345 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(618)) - Processing 127.0.0.1:1234 of type STATUS_UPDATE 2016-05-05 17:27:29,345 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2016-05-05 17:27:29,345 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1052)) - nodeUpdate: 127.0.0.1:1234 clusterResources: 2016-05-05 17:27:29,345 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1125)) - Node being looked for scheduling 127.0.0.1:1234 availableResource: 2016-05-05 17:27:29,345 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1276)) - Trying to schedule on node: 127.0.0.1, available: 2016-05-05 17:27:29,345 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(416)) - Trying to assign containers to child-queue of root 2016-05-05 17:27:29,345 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:printChildQueues(634)) - printChildQueues - queue: root child-queues: root.defaultusedCapacity=(0.0), label=(*) 2016-05-05 17:27:29,346 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(581)) - Trying to assign to queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=1, numContainers=0 2016-05-05 17:27:29,346 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:assignContainers(872)) - assignContainers: node=127.0.0.1 #applications=1 2016-05-05 17:27:29,346 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1190)) - User limit computation for moonfang in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-05-05 17:27:29,346 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimitAndSetHeadroom(1079)) - Headroom calculation for user moonfang: userLimit= queueMaxAvailRes= consumed= headroom= 2016-05-05 17:27:29,346 DEBUG [SchedulerEventDispatcher:Event Processor] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:assignContainers(496)) - pre-assignContainers for application application_1462440433865_0002 2016-05-05 17:27:29,346 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1190)) - User limit computation for moonfang in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-05-05 17:27:29,346 DEBUG [SchedulerEventDispatcher:Event Processor] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(480)) - showRequests: application=application_1462440433865_0002 headRoom= currentConsumption=0 2016-05-05 17:27:29,346 DEBUG [SchedulerEventDispatcher:Event Processor] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(484)) - showRequests: application=application_1462440433865_0002 request={Priority: 0, Capability: , # Containers: 1, Location: *, Relax Locality: true, Node Label Expression: } 2016-05-05 17:27:29,346 DEBUG [SchedulerEventDispatcher:Event Processor] allocator.IncreaseContainerAllocator (IncreaseContainerAllocator.java:assignContainers(184)) - Skip allocating increase request since we don't have any increase request on this node=127.0.0.1:1234 2016-05-05 17:27:29,347 DEBUG [SchedulerEventDispatcher:Event Processor] allocator.RegularContainerAllocator (RegularContainerAllocator.java:assignContainer(412)) - assignContainers: node=127.0.0.1 application=application_1462440433865_0002 priority=0 request={Priority: 0, Capability: , # Containers: 1, Location: *, Relax Locality: true, Node Label Expression: } type=OFF_SWITCH 2016-05-05 17:27:29,347 DEBUG [SchedulerEventDispatcher:Event Processor] scheduler.ActiveUsersManager (ActiveUsersManager.java:deactivateApplication(94)) - User moonfang removed from activeUsers, currently: 0 2016-05-05 17:27:29,347 DEBUG [SchedulerEventDispatcher:Event Processor] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:allocate(607)) - allocate: applicationId=application_1462440433865_0002 container=container_1462440433865_0002_03_000001 host=127.0.0.1:1234 user=moonfang resource= type=OFF_SWITCH 2016-05-05 17:27:29,347 DEBUG [SchedulerEventDispatcher:Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(439)) - Processing container_1462440433865_0002_03_000001 of type START 2016-05-05 17:27:29,347 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: CONTAINER_ALLOCATED 2016-05-05 17:27:29,347 INFO [SchedulerEventDispatcher:Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(453)) - container_1462440433865_0002_03_000001 Container Transitioned from NEW to ALLOCATED 2016-05-05 17:27:29,347 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(859)) - Processing event for appattempt_1462440433865_0002_000003 of type CONTAINER_ALLOCATED 2016-05-05 17:27:29,348 DEBUG [SchedulerEventDispatcher:Event Processor] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:allocate(225)) - allocate: applicationAttemptId=appattempt_1462440433865_0002_000003 container=container_1462440433865_0002_03_000001 host=127.0.0.1 type=OFF_SWITCH 2016-05-05 17:27:29,348 INFO [SchedulerEventDispatcher:Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(154)) - USER=moonfang OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1462440433865_0002 CONTAINERID=container_1462440433865_0002_03_000001 RESOURCE= 2016-05-05 17:27:29,348 DEBUG [SchedulerEventDispatcher:Event Processor] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(158)) - Assigned container container_1462440433865_0002_03_000001 of capacity on host 127.0.0.1:1234, which has 1 containers, used and available after allocation 2016-05-05 17:27:29,348 INFO [SchedulerEventDispatcher:Event Processor] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(89)) - assignedContainer application attempt=appattempt_1462440433865_0002_000003 container=container_1462440433865_0002_03_000001 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@45fbba28 clusterResource= type=OFF_SWITCH 2016-05-05 17:27:29,348 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:assignContainers(941)) - post-assignContainers for application application_1462440433865_0002 2016-05-05 17:27:29,348 DEBUG [AsyncDispatcher event handler] security.BaseContainerTokenSecretManager (BaseContainerTokenSecretManager.java:createPassword(90)) - Creating password for container_1462440433865_0002_03_000001 for user container_1462440433865_0002_03_000001 (auth:SIMPLE) to be run on NM 127.0.0.1:1234 2016-05-05 17:27:29,349 DEBUG [AsyncDispatcher event handler] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 2 cluster_timestamp: 1462440433865 } attemptId: 3 } id: 1 } nmHostAddr: "127.0.0.1:1234" appSubmitter: "moonfang" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1462441049348 masterKeyId: -1802453347 rmIdentifier: 1462440433865 priority { priority: 0 } creationTime: 1462440449347 nodeLabelExpression: "" containerType: APPLICATION_MASTER executionType: GUARANTEED 2016-05-05 17:27:29,349 DEBUG [AsyncDispatcher event handler] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 2 cluster_timestamp: 1462440433865 } attemptId: 3 } id: 1 } nmHostAddr: "127.0.0.1:1234" appSubmitter: "moonfang" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1462441049348 masterKeyId: -1802453347 rmIdentifier: 1462440433865 priority { priority: 0 } creationTime: 1462440449347 nodeLabelExpression: "" containerType: APPLICATION_MASTER executionType: GUARANTEED 2016-05-05 17:27:29,349 INFO [AsyncDispatcher event handler] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:createAndGetNMToken(200)) - Sending NMToken for nodeId : 127.0.0.1:1234 for container : container_1462440433865_0002_03_000001 2016-05-05 17:27:29,349 DEBUG [AsyncDispatcher event handler] security.BaseNMTokenSecretManager (BaseNMTokenSecretManager.java:createPassword(75)) - creating password for appattempt_1462440433865_0002_000003 for user moonfang to run on NM 127.0.0.1:1234 2016-05-05 17:27:29,350 DEBUG [AsyncDispatcher event handler] security.NMTokenIdentifier (NMTokenIdentifier.java:write(96)) - Writing NMTokenIdentifier to RPC layer: appAttemptId { application_id { id: 2 cluster_timestamp: 1462440433865 } attemptId: 3 } nodeId { host: "127.0.0.1" port: 1234 } appSubmitter: "moonfang" keyId: -790324192 2016-05-05 17:27:29,350 DEBUG [AsyncDispatcher event handler] security.NMTokenIdentifier (NMTokenIdentifier.java:write(96)) - Writing NMTokenIdentifier to RPC layer: appAttemptId { application_id { id: 2 cluster_timestamp: 1462440433865 } attemptId: 3 } nodeId { host: "127.0.0.1" port: 1234 } appSubmitter: "moonfang" keyId: -790324192 2016-05-05 17:27:29,350 DEBUG [AsyncDispatcher event handler] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(439)) - Processing container_1462440433865_0002_03_000001 of type ACQUIRED 2016-05-05 17:27:29,350 INFO [AsyncDispatcher event handler] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(453)) - container_1462440433865_0002_03_000001 Container Transitioned from ALLOCATED to ACQUIRED 2016-05-05 17:27:29,350 DEBUG [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1190)) - User limit computation for moonfang in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-05-05 17:27:29,351 INFO [AsyncDispatcher event handler] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:clearNodeSetForAttempt(146)) - Clear node set for appattempt_1462440433865_0002_000003 2016-05-05 17:27:29,351 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1190)) - User limit computation for moonfang in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-05-05 17:27:29,351 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:storeAttempt(2075)) - Storing attempt: AppId: application_1462440433865_0002 AttemptId: appattempt_1462440433865_0002_000003 MasterContainer: Container: [ContainerId: container_1462440433865_0002_03_000001, NodeId: 127.0.0.1:1234, NodeHttpAddress: 127.0.0.1:2, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 127.0.0.1:1234 }, ] 2016-05-05 17:27:29,351 DEBUG [SchedulerEventDispatcher:Event Processor] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(480)) - showRequests: application=application_1462440433865_0002 headRoom= currentConsumption=1024 2016-05-05 17:27:29,351 DEBUG [SchedulerEventDispatcher:Event Processor] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(484)) - showRequests: application=application_1462440433865_0002 request={Priority: 0, Capability: , # Containers: 0, Location: *, Relax Locality: true, Node Label Expression: } 2016-05-05 17:27:29,351 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStoreAppAttemptEvent.EventType: STORE_APP_ATTEMPT 2016-05-05 17:27:29,352 DEBUG [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:handleStoreEvent(1070)) - Processing event of type STORE_APP_ATTEMPT 2016-05-05 17:27:29,352 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1190)) - User limit computation for moonfang in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-05-05 17:27:29,351 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(871)) - appattempt_1462440433865_0002_000003 State change from SCHEDULED to ALLOCATED_SAVING 2016-05-05 17:27:29,352 DEBUG [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:transition(300)) - Storing info for attempt: appattempt_1462440433865_0002_000003 2016-05-05 17:27:29,352 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1190)) - User limit computation for moonfang in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-05-05 17:27:29,352 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppRunningOnNodeEvent.EventType: APP_RUNNING_ON_NODE 2016-05-05 17:27:29,352 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(811)) - Processing event for application_1462440433865_0002 of type APP_RUNNING_ON_NODE 2016-05-05 17:27:29,352 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1190)) - User limit computation for moonfang in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-05-05 17:27:29,352 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: ATTEMPT_NEW_SAVED 2016-05-05 17:27:29,352 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:allocateResource(1401)) - default user=moonfang used= numContainers=1 headroom = user-resources= 2016-05-05 17:27:29,352 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(859)) - Processing event for appattempt_1462440433865_0002_000003 of type ATTEMPT_NEW_SAVED 2016-05-05 17:27:29,352 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(592)) - Assigned to queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.128, absoluteUsedCapacity=0.128, numApps=1, numContainers=1 --> , OFF_SWITCH 2016-05-05 17:27:29,352 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(871)) - appattempt_1462440433865_0002_000003 State change from ALLOCATED_SAVING to ALLOCATED 2016-05-05 17:27:29,352 INFO [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(606)) - Re-sorting assigned queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.128, absoluteUsedCapacity=0.128, numApps=1, numContainers=1 2016-05-05 17:27:29,353 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncherEvent.EventType: LAUNCH 2016-05-05 17:27:29,353 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:printChildQueues(634)) - printChildQueues - queue: root child-queues: root.defaultusedCapacity=(0.128), label=(*) 2016-05-05 17:27:29,353 INFO [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(469)) - assignedContainer queue=root usedCapacity=0.128 absoluteUsedCapacity=0.128 used= cluster= 2016-05-05 17:27:29,353 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(481)) - ParentQ=root assignedSoFarInThisIteration= usedCapacity=0.128 absoluteUsedCapacity=0.128 2016-05-05 17:27:29,353 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(492)) - Not assigning more than one off-switch container, assignments so far: resource:; type:OFF_SWITCH; excessReservation:null; applicationid:null; skipped:false; fulfilled reservation:false; allocations(count/resource):1/; reservations(count/resource):0/ 2016-05-05 17:27:29,355 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(265)) - Attempt State is : ALLOCATED 2016-05-05 17:27:29,355 INFO [Thread-0] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:createAndGetAMRMToken(195)) - Create AMRMToken for ApplicationAttempt: appattempt_1462440433865_0002_000003 2016-05-05 17:27:29,355 INFO [Thread-0] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:createPassword(307)) - Creating password for appattempt_1462440433865_0002_000003 2016-05-05 17:27:29,355 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: LAUNCHED 2016-05-05 17:27:29,355 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(259)) - AppAttempt : appattempt_1462440433865_0002_000003 State is : ALLOCATED Waiting for state : LAUNCHED 2016-05-05 17:27:29,355 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(859)) - Processing event for appattempt_1462440433865_0002_000003 of type LAUNCHED 2016-05-05 17:27:29,355 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(871)) - appattempt_1462440433865_0002_000003 State change from ALLOCATED to LAUNCHED 2016-05-05 17:27:29,365 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(265)) - Attempt State is : LAUNCHED 2016-05-05 17:27:29,366 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(265)) - Attempt State is : LAUNCHED 2016-05-05 17:27:29,366 DEBUG [Thread-0] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1780)) - PrivilegedAction as:appattempt_1462440433865_0002_000003 (auth:SIMPLE) from:org.apache.hadoop.yarn.server.resourcemanager.MockAM.registerAppAttempt(MockAM.java:116) 2016-05-05 17:27:29,366 INFO [Thread-0] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:registerApplicationMaster(235)) - AM registration appattempt_1462440433865_0002_000003 2016-05-05 17:27:29,366 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptRegistrationEvent.EventType: REGISTERED 2016-05-05 17:27:29,366 INFO [Thread-0] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(175)) - USER=moonfang OPERATION=Register App Master TARGET=ApplicationMasterService RESULT=SUCCESS APPID=application_1462440433865_0002 APPATTEMPTID=appattempt_1462440433865_0002_000003 2016-05-05 17:27:29,366 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(859)) - Processing event for appattempt_1462440433865_0002_000003 of type REGISTERED 2016-05-05 17:27:29,366 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(186)) - App : application_1462440433865_0002 State is : ACCEPTED Waiting for state : RUNNING 2016-05-05 17:27:29,366 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(871)) - appattempt_1462440433865_0002_000003 State change from LAUNCHED to RUNNING 2016-05-05 17:27:29,366 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEvent.EventType: ATTEMPT_REGISTERED 2016-05-05 17:27:29,367 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(811)) - Processing event for application_1462440433865_0002 of type ATTEMPT_REGISTERED 2016-05-05 17:27:29,367 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(823)) - application_1462440433865_0002 State change from ACCEPTED to RUNNING 2016-05-05 17:27:29,377 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(192)) - App State is : RUNNING 2016-05-05 17:27:29,377 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(265)) - Attempt State is : RUNNING 2016-05-05 17:27:29,377 DEBUG [Thread-0] service.AbstractService (AbstractService.java:enterState(452)) - Service: ResourceManager entered state INITED 2016-05-05 17:27:29,377 INFO [Thread-0] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2434)) - found resource core-site.xml at file:/home/moonfang/opensource/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/target/test-classes/core-site.xml 2016-05-05 17:27:29,386 DEBUG [Thread-0] util.PerformanceAdvisory (JniBasedUnixGroupsMappingWithFallback.java:(41)) - Falling back to shell based 2016-05-05 17:27:29,386 DEBUG [Thread-0] security.JniBasedUnixGroupsMappingWithFallback (JniBasedUnixGroupsMappingWithFallback.java:(45)) - Group mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping 2016-05-05 17:27:29,386 DEBUG [Thread-0] security.Groups (Groups.java:(114)) - Group mapping impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback; cacheTimeout=300000; warningDeltaMs=5000 2016-05-05 17:27:29,386 INFO [Thread-0] security.Groups (Groups.java:refresh(271)) - clearing userToGroupsMap cache 2016-05-05 17:27:29,388 INFO [Thread-0] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2434)) - found resource yarn-site.xml at file:/home/moonfang/opensource/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/target/test-classes/yarn-site.xml 2016-05-05 17:27:29,397 INFO [Thread-0] event.AsyncDispatcher (AsyncDispatcher.java:register(209)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.RMFatalEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMFatalEventDispatcher 2016-05-05 17:27:29,398 DEBUG [Thread-0] service.CompositeService (CompositeService.java:addService(73)) - Adding service Dispatcher 2016-05-05 17:27:29,398 DEBUG [Thread-0] service.CompositeService (CompositeService.java:addService(73)) - Adding service org.apache.hadoop.yarn.server.resourcemanager.AdminService 2016-05-05 17:27:29,398 DEBUG [Thread-0] service.AbstractService (AbstractService.java:enterState(452)) - Service: RMActiveServices entered state INITED 2016-05-05 17:27:29,398 INFO [Thread-0] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:(75)) - NMTokenKeyRollingInterval: 86400000ms and NMTokenKeyActivationDelay: 900000ms 2016-05-05 17:27:29,398 INFO [Thread-0] security.RMContainerTokenSecretManager (RMContainerTokenSecretManager.java:(78)) - ContainerTokenKeyRollingInterval: 86400000ms and ContainerTokenKeyActivationDelay: 900000ms 2016-05-05 17:27:29,399 INFO [Thread-0] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:(94)) - AMRMTokenKeyRollingInterval: 86400000ms and AMRMTokenKeyActivationDelay: 900000 ms 2016-05-05 17:27:29,399 DEBUG [Thread-0] service.CompositeService (CompositeService.java:addService(73)) - Adding service org.apache.hadoop.yarn.server.resourcemanager.RMSecretManagerService 2016-05-05 17:27:29,399 DEBUG [Thread-0] service.CompositeService (CompositeService.java:addService(73)) - Adding service org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.ContainerAllocationExpirer 2016-05-05 17:27:29,399 DEBUG [Thread-0] service.CompositeService (CompositeService.java:addService(73)) - Adding service AMLivelinessMonitor 2016-05-05 17:27:29,399 DEBUG [Thread-0] service.CompositeService (CompositeService.java:addService(73)) - Adding service AMLivelinessMonitor 2016-05-05 17:27:29,399 DEBUG [Thread-0] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.nodelabels.CommonNodeLabelsManager entered state INITED 2016-05-05 17:27:29,399 DEBUG [Thread-0] service.CompositeService (CompositeService.java:addService(73)) - Adding service org.apache.hadoop.yarn.nodelabels.CommonNodeLabelsManager 2016-05-05 17:27:29,399 INFO [Thread-0] recovery.RMStateStoreFactory (RMStateStoreFactory.java:getStore(33)) - Using RMStateStore implementation - class org.apache.hadoop.yarn.server.resourcemanager.recovery.MemoryRMStateStore 2016-05-05 17:27:29,399 DEBUG [Thread-0] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore entered state INITED 2016-05-05 17:27:29,399 DEBUG [Thread-0] service.AbstractService (AbstractService.java:enterState(452)) - Service: Dispatcher entered state INITED 2016-05-05 17:27:29,400 INFO [Thread-0] event.AsyncDispatcher (AsyncDispatcher.java:register(209)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStoreEventType for class org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$ForwardingEventHandler 2016-05-05 17:27:29,400 INFO [Thread-0] event.AsyncDispatcher (AsyncDispatcher.java:register(209)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.NodesListManagerEventType for class org.apache.hadoop.yarn.server.resourcemanager.NodesListManager 2016-05-05 17:27:29,400 DEBUG [Thread-0] service.CompositeService (CompositeService.java:addService(73)) - Adding service org.apache.hadoop.yarn.server.resourcemanager.NodesListManager 2016-05-05 17:27:29,400 INFO [Thread-0] resourcemanager.ResourceManager (ResourceManager.java:createScheduler(382)) - Using Scheduler: org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler 2016-05-05 17:27:29,400 DEBUG [Thread-0] service.CompositeService (CompositeService.java:addService(73)) - Adding service org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler 2016-05-05 17:27:29,400 DEBUG [Thread-0] service.CompositeService (CompositeService.java:addService(73)) - Adding service SchedulerEventDispatcher 2016-05-05 17:27:29,400 INFO [Thread-0] event.AsyncDispatcher (AsyncDispatcher.java:register(209)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.SchedulerEventType for class org.apache.hadoop.yarn.event.EventDispatcher 2016-05-05 17:27:29,400 INFO [Thread-0] event.AsyncDispatcher (AsyncDispatcher.java:register(209)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationEventDispatcher 2016-05-05 17:27:29,400 INFO [Thread-0] event.AsyncDispatcher (AsyncDispatcher.java:register(209)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationAttemptEventDispatcher 2016-05-05 17:27:29,400 INFO [Thread-0] event.AsyncDispatcher (AsyncDispatcher.java:register(209)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$NodeEventDispatcher 2016-05-05 17:27:29,400 DEBUG [Thread-0] service.CompositeService (CompositeService.java:addService(73)) - Adding service NMLivelinessMonitor 2016-05-05 17:27:29,400 INFO [Thread-0] security.RMContainerTokenSecretManager (RMContainerTokenSecretManager.java:rollMasterKey(106)) - Rolling master-key for container-tokens 2016-05-05 17:27:29,400 INFO [Thread-0] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:rollMasterKey(95)) - Rolling master-key for nm-tokens 2016-05-05 17:27:29,400 DEBUG [Thread-0] service.CompositeService (CompositeService.java:addService(73)) - Adding service org.apache.hadoop.yarn.server.resourcemanager.ResourceTrackerService 2016-05-05 17:27:29,400 WARN [Thread-0] impl.MetricsSystemImpl (MetricsSystemImpl.java:init(152)) - ResourceManager metrics system already initialized! 2016-05-05 17:27:29,400 DEBUG [Thread-0] service.CompositeService (CompositeService.java:addService(73)) - Adding service org.apache.hadoop.util.JvmPauseMonitor 2016-05-05 17:27:29,401 DEBUG [Thread-0] service.CompositeService (CompositeService.java:addService(73)) - Adding service org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService 2016-05-05 17:27:29,401 INFO [Thread-0] event.AsyncDispatcher (AsyncDispatcher.java:register(209)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.RMAppManagerEventType for class org.apache.hadoop.yarn.server.resourcemanager.RMAppManager 2016-05-05 17:27:29,401 DEBUG [Thread-0] service.CompositeService (CompositeService.java:addService(73)) - Adding service org.apache.hadoop.yarn.server.resourcemanager.ClientRMService 2016-05-05 17:27:29,401 INFO [Thread-0] event.AsyncDispatcher (AsyncDispatcher.java:register(209)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncherEventType for class org.apache.hadoop.yarn.server.resourcemanager.MockRM$6 2016-05-05 17:27:29,401 DEBUG [Thread-0] service.CompositeService (CompositeService.java:addService(73)) - Adding service org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher 2016-05-05 17:27:29,402 WARN [Thread-0] util.MBeans (MBeans.java:getMBeanName(125)) - Error creating MBean object name: Hadoop:service=ResourceManager,name=RMNMInfo org.apache.hadoop.metrics2.MetricsException: org.apache.hadoop.metrics2.MetricsException: Hadoop:service=ResourceManager,name=RMNMInfo already exists! at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.newObjectName(DefaultMetricsSystem.java:130) at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.newMBeanName(DefaultMetricsSystem.java:110) at org.apache.hadoop.metrics2.util.MBeans.getMBeanName(MBeans.java:123) at org.apache.hadoop.metrics2.util.MBeans.register(MBeans.java:64) at org.apache.hadoop.yarn.server.resourcemanager.RMNMInfo.(RMNMInfo.java:59) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceInit(ResourceManager.java:656) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAndInitActiveServices(ResourceManager.java:1000) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:290) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.yarn.server.resourcemanager.MockRM.(MockRM.java:133) at org.apache.hadoop.yarn.server.resourcemanager.MockRM.(MockRM.java:126) at org.apache.hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart.testRMAppAttemptFailuresValidityInterval(TestAMRestart.java:879) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74) Caused by: org.apache.hadoop.metrics2.MetricsException: Hadoop:service=ResourceManager,name=RMNMInfo already exists! at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.newObjectName(DefaultMetricsSystem.java:126) ... 21 more 2016-05-05 17:27:29,403 INFO [Thread-0] resourcemanager.RMNMInfo (RMNMInfo.java:(63)) - Registered RMNMInfo MBean 2016-05-05 17:27:29,403 DEBUG [Thread-0] service.CompositeService (CompositeService.java:serviceInit(104)) - RMActiveServices: initing services, size=14 2016-05-05 17:27:29,403 DEBUG [Thread-0] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.RMSecretManagerService entered state INITED 2016-05-05 17:27:29,403 DEBUG [Thread-0] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.ContainerAllocationExpirer entered state INITED 2016-05-05 17:27:29,403 DEBUG [Thread-0] service.AbstractService (AbstractService.java:enterState(452)) - Service: AMLivelinessMonitor entered state INITED 2016-05-05 17:27:29,403 DEBUG [Thread-0] service.AbstractService (AbstractService.java:enterState(452)) - Service: AMLivelinessMonitor entered state INITED 2016-05-05 17:27:29,403 DEBUG [Thread-0] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.NodesListManager entered state INITED 2016-05-05 17:27:29,403 INFO [Thread-0] util.HostsFileReader (HostsFileReader.java:refresh(132)) - Refreshing hosts (include/exclude) list 2016-05-05 17:27:29,404 DEBUG [Thread-0] resourcemanager.NodesListManager (NodesListManager.java:printConfiguredHosts(116)) - hostsReader: in= out= 2016-05-05 17:27:29,404 DEBUG [Thread-0] service.CompositeService (CompositeService.java:serviceInit(104)) - org.apache.hadoop.yarn.server.resourcemanager.NodesListManager: initing services, size=0 2016-05-05 17:27:29,404 DEBUG [Thread-0] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler entered state INITED 2016-05-05 17:27:29,404 INFO [Thread-0] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2434)) - found resource capacity-scheduler.xml at file:/home/moonfang/opensource/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/target/test-classes/capacity-scheduler.xml 2016-05-05 17:27:29,415 DEBUG [Thread-0] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getQueues(629)) - CSConf - getQueues called for: queuePrefix=yarn.scheduler.capacity.root. 2016-05-05 17:27:29,415 DEBUG [Thread-0] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getQueues(639)) - CSConf - getQueues: queuePrefix=yarn.scheduler.capacity.root., queues=default 2016-05-05 17:27:29,415 DEBUG [Thread-0] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getNonLabeledQueueCapacity(341)) - CSConf - getCapacity: queuePrefix=yarn.scheduler.capacity.root., capacity=100.0 2016-05-05 17:27:29,416 DEBUG [Thread-0] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getNonLabeledQueueCapacity(341)) - CSConf - getCapacity: queuePrefix=yarn.scheduler.capacity.root., capacity=100.0 2016-05-05 17:27:29,416 DEBUG [Thread-0] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(685)) - max alloc mb per queue for root is -1 2016-05-05 17:27:29,416 DEBUG [Thread-0] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(687)) - max alloc vcores per queue for root is -1 2016-05-05 17:27:29,416 INFO [Thread-0] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(692)) - max alloc mb per queue for root is undefined 2016-05-05 17:27:29,416 INFO [Thread-0] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(696)) - max alloc vcore per queue for root is undefined 2016-05-05 17:27:29,416 INFO [Thread-0] capacity.ParentQueue (ParentQueue.java:setupQueueConfigs(128)) - root, capacity=1.0, asboluteCapacity=1.0, maxCapacity=1.0, asboluteMaxCapacity=1.0, state=RUNNING, acls=ADMINISTER_QUEUE:*SUBMIT_APP:*, labels=*, , reservationsContinueLooking=true 2016-05-05 17:27:29,416 INFO [Thread-0] capacity.ParentQueue (ParentQueue.java:(107)) - Initialized parent-queue root name=root, fullname=root 2016-05-05 17:27:29,416 DEBUG [Thread-0] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getQueues(629)) - CSConf - getQueues called for: queuePrefix=yarn.scheduler.capacity.root.default. 2016-05-05 17:27:29,416 DEBUG [Thread-0] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getQueues(639)) - CSConf - getQueues: queuePrefix=yarn.scheduler.capacity.root.default., queues= 2016-05-05 17:27:29,417 DEBUG [Thread-0] capacity.LeafQueue (LeafQueue.java:(143)) - LeafQueue: name=default, fullname=root.default 2016-05-05 17:27:29,417 DEBUG [Thread-0] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getNonLabeledQueueCapacity(341)) - CSConf - getCapacity: queuePrefix=yarn.scheduler.capacity.root.default., capacity=100.0 2016-05-05 17:27:29,417 DEBUG [Thread-0] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(685)) - max alloc mb per queue for root.default is -1 2016-05-05 17:27:29,418 DEBUG [Thread-0] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(687)) - max alloc vcores per queue for root.default is -1 2016-05-05 17:27:29,418 INFO [Thread-0] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(692)) - max alloc mb per queue for root.default is undefined 2016-05-05 17:27:29,418 INFO [Thread-0] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(696)) - max alloc vcore per queue for root.default is undefined 2016-05-05 17:27:29,418 INFO [Thread-0] capacity.LeafQueue (LeafQueue.java:setupQueueConfigs(223)) - Initializing default capacity = 1.0 [= (float) configuredCapacity / 100 ] asboluteCapacity = 1.0 [= parentAbsoluteCapacity * capacity ] maxCapacity = 1.0 [= configuredMaxCapacity ] absoluteMaxCapacity = 1.0 [= 1.0 maximumCapacity undefined, (parentAbsoluteMaxCapacity * maximumCapacity) / 100 otherwise ] userLimit = 100 [= configuredUserLimit ] userLimitFactor = 1.0 [= configuredUserLimitFactor ] maxApplications = 10000 [= configuredMaximumSystemApplicationsPerQueue or (int)(configuredMaximumSystemApplications * absoluteCapacity)] maxApplicationsPerUser = 10000 [= (int)(maxApplications * (userLimit / 100.0f) * userLimitFactor) ] usedCapacity = 0.0 [= usedResourcesMemory / (clusterResourceMemory * absoluteCapacity)] absoluteUsedCapacity = 0.0 [= usedResourcesMemory / clusterResourceMemory] maxAMResourcePerQueuePercent = 0.1 [= configuredMaximumAMResourcePercent ] minimumAllocationFactor = 0.875 [= (float)(maximumAllocationMemory - minimumAllocationMemory) / maximumAllocationMemory ] maximumAllocation = [= configuredMaxAllocation ] numContainers = 0 [= currentNumContainers ] state = RUNNING [= configuredState ] acls = ADMINISTER_QUEUE:*SUBMIT_APP:* [= configuredAcls ] nodeLocalityDelay = 40 labels=*, reservationsContinueLooking = true preemptionDisabled = true defaultAppPriorityPerQueue = 0 2016-05-05 17:27:29,419 INFO [Thread-0] capacity.CapacityScheduler (CapacityScheduler.java:parseQueue(677)) - Initialized queue: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=0, numContainers=0 2016-05-05 17:27:29,419 DEBUG [Thread-0] capacity.ParentQueue (ParentQueue.java:setChildQueues(173)) - setChildQueues: root.defaultusedCapacity=(0.0), label=(*) 2016-05-05 17:27:29,419 INFO [Thread-0] capacity.CapacityScheduler (CapacityScheduler.java:parseQueue(677)) - Initialized queue: root: numChildQueue= 1, capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.0, numApps=0, numContainers=0 2016-05-05 17:27:29,419 INFO [Thread-0] capacity.CapacityScheduler (CapacityScheduler.java:initializeQueues(512)) - Initialized root queue root: numChildQueue= 1, capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.0, numApps=0, numContainers=0 2016-05-05 17:27:29,419 INFO [Thread-0] capacity.CapacityScheduler (CapacityScheduler.java:getUserGroupMappingPlacementRule(460)) - Initialized queue mappings, override: false 2016-05-05 17:27:29,419 INFO [Thread-0] capacity.CapacityScheduler (CapacityScheduler.java:initScheduler(322)) - Initialized CapacityScheduler with calculator=class org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator, minimumAllocation=<>, maximumAllocation=<>, asynchronousScheduling=false, asyncScheduleInterval=5ms 2016-05-05 17:27:29,419 DEBUG [Thread-0] service.AbstractService (AbstractService.java:enterState(452)) - Service: SchedulerEventDispatcher entered state INITED 2016-05-05 17:27:29,419 DEBUG [Thread-0] service.AbstractService (AbstractService.java:enterState(452)) - Service: NMLivelinessMonitor entered state INITED 2016-05-05 17:27:29,419 DEBUG [Thread-0] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.ResourceTrackerService entered state INITED 2016-05-05 17:27:29,420 INFO [Thread-0] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2431)) - dynamic-resources.xml not found 2016-05-05 17:27:29,420 DEBUG [Thread-0] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.util.JvmPauseMonitor entered state INITED 2016-05-05 17:27:29,420 DEBUG [Thread-0] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService entered state INITED 2016-05-05 17:27:29,420 DEBUG [Thread-0] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.ClientRMService entered state INITED 2016-05-05 17:27:29,421 DEBUG [Thread-0] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher entered state INITED 2016-05-05 17:27:29,421 DEBUG [Thread-0] service.CompositeService (CompositeService.java:addService(73)) - Adding service org.apache.hadoop.yarn.server.resourcemanager.ahs.RMApplicationHistoryWriter 2016-05-05 17:27:29,421 DEBUG [Thread-0] service.CompositeService (CompositeService.java:addService(73)) - Adding service org.apache.hadoop.yarn.server.resourcemanager.metrics.SystemMetricsPublisher 2016-05-05 17:27:29,421 DEBUG [Thread-0] service.CompositeService (CompositeService.java:serviceInit(104)) - ResourceManager: initing services, size=4 2016-05-05 17:27:29,421 DEBUG [Thread-0] service.AbstractService (AbstractService.java:enterState(452)) - Service: Dispatcher entered state INITED 2016-05-05 17:27:29,422 DEBUG [Thread-0] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.AdminService entered state INITED 2016-05-05 17:27:29,422 DEBUG [Thread-0] service.CompositeService (CompositeService.java:serviceInit(104)) - org.apache.hadoop.yarn.server.resourcemanager.AdminService: initing services, size=0 2016-05-05 17:27:29,422 DEBUG [Thread-0] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.ahs.RMApplicationHistoryWriter entered state INITED 2016-05-05 17:27:29,422 DEBUG [Thread-0] service.CompositeService (CompositeService.java:serviceInit(104)) - org.apache.hadoop.yarn.server.resourcemanager.ahs.RMApplicationHistoryWriter: initing services, size=0 2016-05-05 17:27:29,422 DEBUG [Thread-0] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.metrics.SystemMetricsPublisher entered state INITED 2016-05-05 17:27:29,422 INFO [Thread-0] metrics.SystemMetricsPublisher (SystemMetricsPublisher.java:serviceInit(95)) - YARN system metrics publishing service is not enabled 2016-05-05 17:27:29,422 DEBUG [Thread-0] service.CompositeService (CompositeService.java:serviceInit(104)) - org.apache.hadoop.yarn.server.resourcemanager.metrics.SystemMetricsPublisher: initing services, size=0 2016-05-05 17:27:29,422 INFO [Thread-0] resourcemanager.ResourceManager (ResourceManager.java:transitionToActive(1044)) - Transitioning to active state 2016-05-05 17:27:29,422 DEBUG [Thread-0] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1780)) - PrivilegedAction as:moonfang (auth:SIMPLE) from:org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.transitionToActive(ResourceManager.java:1046) 2016-05-05 17:27:29,422 INFO [Thread-0] resourcemanager.ResourceManager (ResourceManager.java:serviceStart(670)) - Recovery started 2016-05-05 17:27:29,422 INFO [Thread-0] security.RMDelegationTokenSecretManager (RMDelegationTokenSecretManager.java:recover(178)) - recovering RMDelegationTokenSecretManager. 2016-05-05 17:27:29,423 INFO [Thread-0] resourcemanager.RMAppManager (RMAppManager.java:recover(475)) - Recovering 2 applications 2016-05-05 17:27:29,423 INFO [Thread-0] capacity.CapacityScheduler (CapacityScheduler.java:checkAndGetApplicationPriority(2051)) - Priority '0' is acceptable in queue : default for application: application_1462440433865_0001 for the user: moonfang 2016-05-05 17:27:29,423 INFO [Thread-0] rmapp.RMAppImpl (RMAppImpl.java:(439)) - The attemptFailuresValidityInterval for the application: application_1462440433865_0001 is 60000. 2016-05-05 17:27:29,423 DEBUG [Thread-0] rmapp.RMAppImpl (RMAppImpl.java:handle(811)) - Processing event for application_1462440433865_0001 of type RECOVER 2016-05-05 17:27:29,423 INFO [Thread-0] rmapp.RMAppImpl (RMAppImpl.java:recover(836)) - Recovering app: application_1462440433865_0001 with 2 attempts and final state = FAILED 2016-05-05 17:27:29,424 INFO [Thread-0] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:recover(905)) - Recovering attempt: appattempt_1462440433865_0001_000001 with final state: FAILED 2016-05-05 17:27:29,425 INFO [Thread-0] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:recover(905)) - Recovering attempt: appattempt_1462440433865_0001_000002 with final state: FAILED 2016-05-05 17:27:29,425 DEBUG [Thread-0] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(859)) - Processing event for appattempt_1462440433865_0001_000001 of type RECOVER 2016-05-05 17:27:29,425 INFO [Thread-0] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(871)) - appattempt_1462440433865_0001_000001 State change from NEW to FAILED 2016-05-05 17:27:29,425 DEBUG [Thread-0] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(859)) - Processing event for appattempt_1462440433865_0001_000002 of type RECOVER 2016-05-05 17:27:29,425 INFO [Thread-0] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(871)) - appattempt_1462440433865_0001_000002 State change from NEW to FAILED 2016-05-05 17:27:29,425 INFO [Thread-0] rmapp.RMAppImpl (RMAppImpl.java:handle(823)) - application_1462440433865_0001 State change from NEW to FAILED 2016-05-05 17:27:29,426 INFO [Thread-0] capacity.CapacityScheduler (CapacityScheduler.java:checkAndGetApplicationPriority(2051)) - Priority '0' is acceptable in queue : default for application: application_1462440433865_0002 for the user: moonfang 2016-05-05 17:27:29,426 INFO [Thread-0] rmapp.RMAppImpl (RMAppImpl.java:(439)) - The attemptFailuresValidityInterval for the application: application_1462440433865_0002 is 10000. 2016-05-05 17:27:29,426 DEBUG [Thread-0] rmapp.RMAppImpl (RMAppImpl.java:handle(811)) - Processing event for application_1462440433865_0002 of type RECOVER 2016-05-05 17:27:29,426 INFO [Thread-0] rmapp.RMAppImpl (RMAppImpl.java:recover(836)) - Recovering app: application_1462440433865_0002 with 3 attempts and final state = null 2016-05-05 17:27:29,426 INFO [Thread-0] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:recover(905)) - Recovering attempt: appattempt_1462440433865_0002_000001 with final state: FAILED 2016-05-05 17:27:29,427 INFO [Thread-0] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:recover(905)) - Recovering attempt: appattempt_1462440433865_0002_000002 with final state: FAILED 2016-05-05 17:27:29,427 INFO [Thread-0] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:recover(905)) - Recovering attempt: appattempt_1462440433865_0002_000003 with final state: null 2016-05-05 17:27:29,427 INFO [Thread-0] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:createAndGetAMRMToken(195)) - Create AMRMToken for ApplicationAttempt: appattempt_1462440433865_0002_000003 2016-05-05 17:27:29,428 INFO [Thread-0] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:createPassword(307)) - Creating password for appattempt_1462440433865_0002_000003 2016-05-05 17:27:29,428 INFO [Thread-0] capacity.ParentQueue (ParentQueue.java:addApplication(352)) - Application added - appId: application_1462440433865_0002 user: moonfang leaf-queue of parent: root #applications: 1 2016-05-05 17:27:29,428 INFO [Thread-0] capacity.CapacityScheduler (CapacityScheduler.java:addApplicationOnRecovery(742)) - Accepted application application_1462440433865_0002 from user: moonfang, in queue: default 2016-05-05 17:27:29,428 DEBUG [Thread-0] capacity.CapacityScheduler (CapacityScheduler.java:addApplicationOnRecovery(745)) - application_1462440433865_0002 is recovering. Skip notifying APP_ACCEPTED 2016-05-05 17:27:29,428 DEBUG [Thread-0] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(859)) - Processing event for appattempt_1462440433865_0002_000001 of type RECOVER 2016-05-05 17:27:29,428 INFO [Thread-0] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(871)) - appattempt_1462440433865_0002_000001 State change from NEW to FAILED 2016-05-05 17:27:29,428 DEBUG [Thread-0] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(859)) - Processing event for appattempt_1462440433865_0002_000002 of type RECOVER 2016-05-05 17:27:29,428 INFO [Thread-0] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(871)) - appattempt_1462440433865_0002_000002 State change from NEW to FAILED 2016-05-05 17:27:29,428 DEBUG [Thread-0] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(859)) - Processing event for appattempt_1462440433865_0002_000003 of type RECOVER 2016-05-05 17:27:29,429 INFO [Thread-0] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(871)) - appattempt_1462440433865_0002_000003 State change from NEW to LAUNCHED 2016-05-05 17:27:29,429 INFO [Thread-0] rmapp.RMAppImpl (RMAppImpl.java:handle(823)) - application_1462440433865_0002 State change from NEW to ACCEPTED 2016-05-05 17:27:29,429 INFO [Thread-0] resourcemanager.ResourceManager (ResourceManager.java:serviceStart(677)) - Recovery ended 2016-05-05 17:27:29,429 DEBUG [Thread-0] service.CompositeService (CompositeService.java:serviceStart(115)) - RMActiveServices: starting services, size=14 2016-05-05 17:27:29,429 INFO [Thread-0] security.RMContainerTokenSecretManager (RMContainerTokenSecretManager.java:rollMasterKey(106)) - Rolling master-key for container-tokens 2016-05-05 17:27:29,429 INFO [Thread-0] security.RMContainerTokenSecretManager (RMContainerTokenSecretManager.java:rollMasterKey(111)) - Going to activate master-key with key-id 1543136109 in 900000ms 2016-05-05 17:27:29,429 INFO [Thread-0] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:rollMasterKey(95)) - Rolling master-key for nm-tokens 2016-05-05 17:27:29,429 INFO [Thread-0] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:rollMasterKey(100)) - Going to activate master-key with key-id -1949980207 in 900000ms 2016-05-05 17:27:29,429 INFO [Thread-0] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2016-05-05 17:27:29,430 INFO [Thread-0] security.RMDelegationTokenSecretManager (RMDelegationTokenSecretManager.java:storeNewMasterKey(87)) - storing master key with keyID 3 2016-05-05 17:27:29,430 DEBUG [Thread-0] recovery.RMStateStore (RMStateStore.java:handleStoreEvent(1070)) - Processing event of type STORE_MASTERKEY 2016-05-05 17:27:29,430 INFO [Thread-0] recovery.RMStateStore (RMStateStore.java:transition(438)) - Storing RMDTMasterKey. 2016-05-05 17:27:29,430 INFO [Thread-0] recovery.RMStateStore (MemoryRMStateStore.java:storeRMDTMasterKeyState(229)) - Store RMDT master key with key id: 3. Currently rmDTMasterKeyState size: 3 2016-05-05 17:27:29,430 DEBUG [Thread-0] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.RMSecretManagerService is started 2016-05-05 17:27:29,430 INFO [Thread[Thread-24,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2016-05-05 17:27:29,430 INFO [Thread[Thread-24,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2016-05-05 17:27:29,430 DEBUG [Thread-0] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.ContainerAllocationExpirer is started 2016-05-05 17:27:29,431 INFO [Thread[Thread-24,5,main]] security.RMDelegationTokenSecretManager (RMDelegationTokenSecretManager.java:storeNewMasterKey(87)) - storing master key with keyID 4 2016-05-05 17:27:29,431 DEBUG [Thread[Thread-24,5,main]] recovery.RMStateStore (RMStateStore.java:handleStoreEvent(1070)) - Processing event of type STORE_MASTERKEY 2016-05-05 17:27:29,431 INFO [Thread[Thread-24,5,main]] recovery.RMStateStore (RMStateStore.java:transition(438)) - Storing RMDTMasterKey. 2016-05-05 17:27:29,431 DEBUG [Thread-0] service.AbstractService (AbstractService.java:start(197)) - Service AMLivelinessMonitor is started 2016-05-05 17:27:29,431 INFO [Thread[Thread-24,5,main]] recovery.RMStateStore (MemoryRMStateStore.java:storeRMDTMasterKeyState(229)) - Store RMDT master key with key id: 4. Currently rmDTMasterKeyState size: 4 2016-05-05 17:27:29,431 DEBUG [Thread-0] service.AbstractService (AbstractService.java:start(197)) - Service AMLivelinessMonitor is started 2016-05-05 17:27:29,431 DEBUG [Thread-0] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.nodelabels.CommonNodeLabelsManager is started 2016-05-05 17:27:29,431 DEBUG [Thread-0] service.CompositeService (CompositeService.java:serviceStart(115)) - org.apache.hadoop.yarn.server.resourcemanager.NodesListManager: starting services, size=0 2016-05-05 17:27:29,431 DEBUG [Thread-0] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.NodesListManager is started 2016-05-05 17:27:29,432 DEBUG [Thread-0] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler is started 2016-05-05 17:27:29,432 DEBUG [Thread-0] service.AbstractService (AbstractService.java:start(197)) - Service SchedulerEventDispatcher is started 2016-05-05 17:27:29,432 DEBUG [Thread-0] service.AbstractService (AbstractService.java:start(197)) - Service NMLivelinessMonitor is started 2016-05-05 17:27:29,432 DEBUG [Thread-0] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.ResourceTrackerService is started 2016-05-05 17:27:29,432 DEBUG [Thread-0] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.util.JvmPauseMonitor is started 2016-05-05 17:27:29,432 INFO [org.apache.hadoop.util.JvmPauseMonitor$Monitor@41704837] util.JvmPauseMonitor (JvmPauseMonitor.java:run(188)) - Starting JVM pause monitor 2016-05-05 17:27:29,432 DEBUG [Thread-0] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService is started 2016-05-05 17:27:29,432 DEBUG [Thread-0] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.ClientRMService is started 2016-05-05 17:27:29,432 DEBUG [Thread-0] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher is started 2016-05-05 17:27:29,432 DEBUG [Thread-0] service.AbstractService (AbstractService.java:start(197)) - Service RMActiveServices is started 2016-05-05 17:27:29,433 INFO [Thread-0] resourcemanager.ResourceManager (ResourceManager.java:transitionToActive(1060)) - Transitioned to active state 2016-05-05 17:27:29,433 DEBUG [Thread-0] service.CompositeService (CompositeService.java:serviceStart(115)) - ResourceManager: starting services, size=4 2016-05-05 17:27:29,433 DEBUG [Thread-0] service.AbstractService (AbstractService.java:start(197)) - Service Dispatcher is started 2016-05-05 17:27:29,433 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.RMAppManagerEvent.EventType: APP_COMPLETED 2016-05-05 17:27:29,433 DEBUG [Thread-0] service.CompositeService (CompositeService.java:serviceStart(115)) - org.apache.hadoop.yarn.server.resourcemanager.AdminService: starting services, size=0 2016-05-05 17:27:29,433 DEBUG [AsyncDispatcher event handler] resourcemanager.RMAppManager (RMAppManager.java:handle(484)) - RMAppManager processing event for application_1462440433865_0001 of type APP_COMPLETED 2016-05-05 17:27:29,433 DEBUG [Thread-0] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.AdminService is started 2016-05-05 17:27:29,433 WARN [AsyncDispatcher event handler] resourcemanager.RMAuditLogger (RMAuditLogger.java:logFailure(340)) - USER=moonfang OPERATION=Application Finished - Failed TARGET=RMAppManager RESULT=FAILURE DESCRIPTION=App failed with state: FAILED PERMISSIONS=Application application_1462440433865_0001 failed 2 times in previous 60000 milliseconds due to AM Container for appattempt_1462440433865_0001_000002 exited with exitCode: 0 Failing this attempt.Diagnostics: SuccessFor more detailed output, check the application tracking page: http://moonfang-Lenovo-Product:8088/cluster/app/application_1462440433865_0001 Then click on links to logs of each attempt. . Failing the application. APPID=application_1462440433865_0001 2016-05-05 17:27:29,433 DEBUG [Thread-0] service.CompositeService (CompositeService.java:serviceStart(115)) - org.apache.hadoop.yarn.server.resourcemanager.ahs.RMApplicationHistoryWriter: starting services, size=0 2016-05-05 17:27:29,433 DEBUG [Thread-0] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.ahs.RMApplicationHistoryWriter is started 2016-05-05 17:27:29,433 DEBUG [AsyncDispatcher event handler] scheduler.AbstractYarnScheduler (AbstractYarnScheduler.java:getAppResourceUsageReport(281)) - Request for appInfo of unknown attempt appattempt_1462440433865_0001_000002 2016-05-05 17:27:29,433 DEBUG [Thread-0] service.CompositeService (CompositeService.java:serviceStart(115)) - org.apache.hadoop.yarn.server.resourcemanager.metrics.SystemMetricsPublisher: starting services, size=0 2016-05-05 17:27:29,433 DEBUG [Thread-0] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.metrics.SystemMetricsPublisher is started 2016-05-05 17:27:29,433 DEBUG [Thread-0] service.AbstractService (AbstractService.java:start(197)) - Service ResourceManager is started 2016-05-05 17:27:29,433 INFO [AsyncDispatcher event handler] resourcemanager.RMAppManager$ApplicationSummary (RMAppManager.java:logAppSummary(195)) - appId=application_1462440433865_0001,name=,user=moonfang,queue=default,state=FAILED,trackingUrl=http://moonfang-Lenovo-Product:8088/proxy/application_1462440433865_0001/,appMasterHost=N/A,startTime=1462440434000,finishTime=1462440434238,finalStatus=FAILED,memorySeconds=142,vcoreSeconds=0,preemptedAMContainers=0,preemptedNonAMContainers=0,preemptedResources=,applicationType=YARN 2016-05-05 17:27:29,444 DEBUG [Thread-0] util.RackResolver (RackResolver.java:coreResolve(112)) - Resolved 127.0.0.1 to /default-rack 2016-05-05 17:27:29,445 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStartedEvent.EventType: STARTED 2016-05-05 17:27:29,445 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(618)) - Processing 127.0.0.1:1234 of type STARTED 2016-05-05 17:27:29,445 INFO [Thread-0] resourcemanager.ResourceTrackerService (ResourceTrackerService.java:registerNodeManager(393)) - received container statuses on node manager register :[[container_1462440433865_0002_03_000001, CreateTime: 0, State: COMPLETE, Capability: null, Diagnostics: , ExitStatus: -106, NodeLabelExpression: Priority: null]] 2016-05-05 17:27:29,445 INFO [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(630)) - 127.0.0.1:1234 Node Transitioned from NEW to RUNNING 2016-05-05 17:27:29,445 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeAddedSchedulerEvent.EventType: NODE_ADDED 2016-05-05 17:27:29,445 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.NodesListManagerEvent.EventType: NODE_USABLE 2016-05-05 17:27:29,445 INFO [Thread-0] resourcemanager.ResourceTrackerService (ResourceTrackerService.java:registerNodeManager(428)) - NodeManager from node 127.0.0.1(cmPort: 1234 httpPort: 2) registered with capability: , assigned nodeId 127.0.0.1:1234 2016-05-05 17:27:29,445 DEBUG [AsyncDispatcher event handler] resourcemanager.NodesListManager (NodesListManager.java:handle(315)) - 127.0.0.1:1234 reported usable 2016-05-05 17:27:29,445 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(259)) - AppAttempt : appattempt_1462440433865_0002_000003 State is : LAUNCHED Waiting for state : FAILED 2016-05-05 17:27:29,446 INFO [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:addNode(1490)) - Added node 127.0.0.1:1234 clusterResource: 2016-05-05 17:27:29,446 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptContainerFinishedEvent.EventType: CONTAINER_FINISHED 2016-05-05 17:27:29,446 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(859)) - Processing event for appattempt_1462440433865_0002_000003 of type CONTAINER_FINISHED 2016-05-05 17:27:29,447 DEBUG [AsyncDispatcher event handler] scheduler.AbstractYarnScheduler (AbstractYarnScheduler.java:getAppResourceUsageReport(281)) - Request for appInfo of unknown attempt appattempt_1462440433865_0002_000003 2016-05-05 17:27:29,447 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:rememberTargetTransitionsAndStoreState(1301)) - Updating application attempt appattempt_1462440433865_0002_000003 with final state: FAILED, and exit status: -106 2016-05-05 17:27:29,447 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(871)) - appattempt_1462440433865_0002_000003 State change from LAUNCHED to FINAL_SAVING 2016-05-05 17:27:29,447 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateUpdateAppAttemptEvent.EventType: UPDATE_APP_ATTEMPT 2016-05-05 17:27:29,447 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeLabelsUpdateSchedulerEvent.EventType: NODE_LABELS_UPDATE 2016-05-05 17:27:29,447 DEBUG [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:handleStoreEvent(1070)) - Processing event of type UPDATE_APP_ATTEMPT 2016-05-05 17:27:29,447 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppNodeUpdateEvent.EventType: NODE_UPDATE 2016-05-05 17:27:29,447 DEBUG [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:transition(331)) - Updating info for attempt: appattempt_1462440433865_0002_000003 2016-05-05 17:27:29,447 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(811)) - Processing event for application_1462440433865_0002 of type NODE_UPDATE 2016-05-05 17:27:29,447 INFO [AsyncDispatcher event handler] recovery.RMStateStore (MemoryRMStateStore.java:updateApplicationAttemptStateInternal(139)) - Updating final state FAILED for attempt: appattempt_1462440433865_0002_000003 2016-05-05 17:27:29,448 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:processNodeUpdate(913)) - Received node update event:NODE_USABLE for node:127.0.0.1:1234 with state:RUNNING 2016-05-05 17:27:29,448 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2016-05-05 17:27:29,448 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(618)) - Processing 127.0.0.1:1234 of type FINISHED_CONTAINERS_PULLED_BY_AM 2016-05-05 17:27:29,448 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: ATTEMPT_UPDATE_SAVED 2016-05-05 17:27:29,448 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(859)) - Processing event for appattempt_1462440433865_0002_000003 of type ATTEMPT_UPDATE_SAVED 2016-05-05 17:27:29,448 INFO [AsyncDispatcher event handler] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:unregisterAttempt(682)) - Unregistering app attempt : appattempt_1462440433865_0002_000003 2016-05-05 17:27:29,448 INFO [AsyncDispatcher event handler] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:applicationMasterFinished(124)) - Application finished, removing password for appattempt_1462440433865_0002_000003 2016-05-05 17:27:29,448 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(871)) - appattempt_1462440433865_0002_000003 State change from FINAL_SAVING to FAILED 2016-05-05 17:27:29,448 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppFailedAttemptEvent.EventType: ATTEMPT_FAILED 2016-05-05 17:27:29,448 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(811)) - Processing event for application_1462440433865_0002 of type ATTEMPT_FAILED 2016-05-05 17:27:29,448 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:transition(1354)) - The number of failed attempts in previous 10000 milliseconds is 1. The max attempts is 2 2016-05-05 17:27:29,448 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:removeExcessAttempts(1413)) - Remove attempt from state store : appattempt_1462440433865_0002_000001 2016-05-05 17:27:29,449 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStoreRemoveAppAttemptEvent.EventType: REMOVE_APP_ATTEMPT 2016-05-05 17:27:29,449 DEBUG [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:handleStoreEvent(1070)) - Processing event of type REMOVE_APP_ATTEMPT 2016-05-05 17:27:29,449 INFO [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:transition(574)) - Removing attempt appattempt_1462440433865_0002_000001 from app: application_1462440433865_0002 2016-05-05 17:27:29,449 INFO [AsyncDispatcher event handler] recovery.RMStateStore (MemoryRMStateStore.java:removeApplicationAttemptInternal(151)) - Removing state for attempt: appattempt_1462440433865_0002_000001 2016-05-05 17:27:29,449 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.AppAttemptRemovedSchedulerEvent.EventType: APP_ATTEMPT_REMOVED 2016-05-05 17:27:29,449 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppStartAttemptEvent.EventType: START 2016-05-05 17:27:29,449 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(859)) - Processing event for appattempt_1462440433865_0002_000004 of type START 2016-05-05 17:27:29,449 INFO [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:doneApplicationAttempt(864)) - Application Attempt appattempt_1462440433865_0002_000003 is done. finalState=FAILED 2016-05-05 17:27:29,449 INFO [AsyncDispatcher event handler] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:registerAppAttempt(676)) - Registering app attempt : appattempt_1462440433865_0002_000004 2016-05-05 17:27:29,449 INFO [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:doneApplicationAttempt(872)) - Unknown application appattempt_1462440433865_0002_000003 has completed! 2016-05-05 17:27:29,450 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(871)) - appattempt_1462440433865_0002_000004 State change from NEW to SUBMITTED 2016-05-05 17:27:29,450 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.AppAttemptAddedSchedulerEvent.EventType: APP_ATTEMPT_ADDED 2016-05-05 17:27:29,450 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:activateApplications(652)) - application application_1462440433865_0002 AMResource maxAMResourcePerQueuePercent 0.1 amLimit lastClusterResource amIfStarted AM node-partition name 2016-05-05 17:27:29,450 INFO [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:activateApplications(726)) - Application application_1462440433865_0002 from user: moonfang activated in queue: default 2016-05-05 17:27:29,450 INFO [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:addApplicationAttempt(741)) - Application added - appId: application_1462440433865_0002 user: moonfang, leaf-queue: default #user-pending-applications: 0 #user-active-applications: 1 #queue-pending-applications: 0 #queue-active-applications: 1 2016-05-05 17:27:29,450 INFO [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:addApplicationAttempt(825)) - Added Application Attempt appattempt_1462440433865_0002_000004 to scheduler from user moonfang in queue default 2016-05-05 17:27:29,450 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: ATTEMPT_ADDED 2016-05-05 17:27:29,450 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(859)) - Processing event for appattempt_1462440433865_0002_000004 of type ATTEMPT_ADDED 2016-05-05 17:27:29,450 WARN [AsyncDispatcher event handler] blacklist.SimpleBlacklistManager (SimpleBlacklistManager.java:getBlacklistUpdates(75)) - Ignoring Blacklists, blacklist size 1 is more than failure threshold ratio 0.800000011920929 out of total usable nodes 1 2016-05-05 17:27:29,451 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:transition(1041)) - Using blacklist for AM: additions([]) and removals([127.0.0.1]) 2016-05-05 17:27:29,451 DEBUG [AsyncDispatcher event handler] capacity.CapacityScheduler (CapacityScheduler.java:allocate(984)) - allocate: pre-update appattempt_1462440433865_0002_000004 ask size =1 2016-05-05 17:27:29,451 DEBUG [AsyncDispatcher event handler] scheduler.ActiveUsersManager (ActiveUsersManager.java:activateApplication(68)) - User moonfang added to activeUsers, currently: 1 2016-05-05 17:27:29,451 DEBUG [AsyncDispatcher event handler] capacity.CapacityScheduler (CapacityScheduler.java:allocate(996)) - allocate: post-update 2016-05-05 17:27:29,451 DEBUG [AsyncDispatcher event handler] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(480)) - showRequests: application=application_1462440433865_0002 headRoom= currentConsumption=0 2016-05-05 17:27:29,451 DEBUG [AsyncDispatcher event handler] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(484)) - showRequests: application=application_1462440433865_0002 request={Priority: 0, Capability: , # Containers: 1, Location: *, Relax Locality: true, Node Label Expression: } 2016-05-05 17:27:29,451 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(871)) - appattempt_1462440433865_0002_000004 State change from SUBMITTED to SCHEDULED 2016-05-05 17:27:29,455 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(265)) - Attempt State is : FAILED 2016-05-05 17:27:44,456 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(192)) - App State is : ACCEPTED 2016-05-05 17:27:44,456 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(192)) - App State is : ACCEPTED 2016-05-05 17:27:44,456 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(265)) - Attempt State is : SCHEDULED Launch AM appattempt_1462440433865_0002_000004 2016-05-05 17:27:44,457 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(259)) - AppAttempt : appattempt_1462440433865_0002_000004 State is : SCHEDULED Waiting for state : ALLOCATED 2016-05-05 17:27:44,457 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2016-05-05 17:27:44,457 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(618)) - Processing 127.0.0.1:1234 of type STATUS_UPDATE 2016-05-05 17:27:44,457 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2016-05-05 17:27:44,457 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1052)) - nodeUpdate: 127.0.0.1:1234 clusterResources: 2016-05-05 17:27:44,457 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1125)) - Node being looked for scheduling 127.0.0.1:1234 availableResource: 2016-05-05 17:27:44,458 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1276)) - Trying to schedule on node: 127.0.0.1, available: 2016-05-05 17:27:44,458 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(416)) - Trying to assign containers to child-queue of root 2016-05-05 17:27:44,458 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:printChildQueues(634)) - printChildQueues - queue: root child-queues: root.defaultusedCapacity=(0.0), label=(*) 2016-05-05 17:27:44,458 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(581)) - Trying to assign to queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=1, numContainers=0 2016-05-05 17:27:44,458 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:assignContainers(872)) - assignContainers: node=127.0.0.1 #applications=1 2016-05-05 17:27:44,458 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1190)) - User limit computation for moonfang in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-05-05 17:27:44,458 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimitAndSetHeadroom(1079)) - Headroom calculation for user moonfang: userLimit= queueMaxAvailRes= consumed= headroom= 2016-05-05 17:27:44,459 DEBUG [SchedulerEventDispatcher:Event Processor] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:assignContainers(496)) - pre-assignContainers for application application_1462440433865_0002 2016-05-05 17:27:44,459 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1190)) - User limit computation for moonfang in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2016-05-05 17:27:44,459 DEBUG [SchedulerEventDispatcher:Event Processor] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(480)) - showRequests: application=application_1462440433865_0002 headRoom= currentConsumption=0 2016-05-05 17:27:44,459 DEBUG [SchedulerEventDispatcher:Event Processor] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(484)) - showRequests: application=application_1462440433865_0002 request={Priority: 0, Capability: , # Containers: 1, Location: *, Relax Locality: true, Node Label Expression: } 2016-05-05 17:27:44,459 DEBUG [SchedulerEventDispatcher:Event Processor] allocator.IncreaseContainerAllocator (IncreaseContainerAllocator.java:assignContainers(184)) - Skip allocating increase request since we don't have any increase request on this node=127.0.0.1:1234 2016-05-05 17:27:44,459 DEBUG [SchedulerEventDispatcher:Event Processor] allocator.RegularContainerAllocator (RegularContainerAllocator.java:assignContainer(412)) - assignContainers: node=127.0.0.1 application=application_1462440433865_0002 priority=0 request={Priority: 0, Capability: , # Containers: 1, Location: *, Relax Locality: true, Node Label Expression: } type=OFF_SWITCH 2016-05-05 17:27:44,460 DEBUG [SchedulerEventDispatcher:Event Processor] scheduler.ActiveUsersManager (ActiveUsersManager.java:deactivateApplication(94)) - User moonfang removed from activeUsers, currently: 0 2016-05-05 17:27:44,460 DEBUG [SchedulerEventDispatcher:Event Processor] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:allocate(607)) - allocate: applicationId=application_1462440433865_0002 container=container_1462440433865_0002_04_000001 host=127.0.0.1:1234 user=moonfang resource= type=OFF_SWITCH 2016-05-05 17:27:44,460 DEBUG [SchedulerEventDispatcher:Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(439)) - Processing container_1462440433865_0002_04_000001 of type START 2016-05-05 17:27:44,460 INFO [SchedulerEventDispatcher:Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(453)) - container_1462440433865_0002_04_000001 Container Transitioned from NEW to ALLOCATED 2016-05-05 17:27:44,460 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: CONTAINER_ALLOCATED 2016-05-05 17:27:44,460 DEBUG [SchedulerEventDispatcher:Event Processor] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:allocate(225)) - allocate: applicationAttemptId=appattempt_1462440433865_0002_000004 container=container_1462440433865_0002_04_000001 host=127.0.0.1 type=OFF_SWITCH 2016-05-05 17:27:44,460 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(859)) - Processing event for appattempt_1462440433865_0002_000004 of type CONTAINER_ALLOCATED 2016-05-05 17:27:44,461 INFO [SchedulerEventDispatcher:Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(154)) - USER=moonfang OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1462440433865_0002 CONTAINERID=container_1462440433865_0002_04_000001 RESOURCE= 2016-05-05 17:27:44,461 DEBUG [SchedulerEventDispatcher:Event Processor] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(158)) - Assigned container container_1462440433865_0002_04_000001 of capacity on host 127.0.0.1:1234, which has 1 containers, used and available after allocation 2016-05-05 17:27:44,461 INFO [SchedulerEventDispatcher:Event Processor] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(89)) - assignedContainer application attempt=appattempt_1462440433865_0002_000004 container=container_1462440433865_0002_04_000001 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@3dc71daa clusterResource= type=OFF_SWITCH 2016-05-05 17:27:44,461 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:assignContainers(941)) - post-assignContainers for application application_1462440433865_0002 2016-05-05 17:27:44,461 DEBUG [AsyncDispatcher event handler] security.BaseContainerTokenSecretManager (BaseContainerTokenSecretManager.java:createPassword(90)) - Creating password for container_1462440433865_0002_04_000001 for user container_1462440433865_0002_04_000001 (auth:SIMPLE) to be run on NM 127.0.0.1:1234 2016-05-05 17:27:44,462 DEBUG [AsyncDispatcher event handler] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 2 cluster_timestamp: 1462440433865 } attemptId: 4 } id: 1 } nmHostAddr: "127.0.0.1:1234" appSubmitter: "moonfang" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1462441064461 masterKeyId: 1543136108 rmIdentifier: 1462440449422 priority { priority: 0 } creationTime: 1462440464460 nodeLabelExpression: "" containerType: APPLICATION_MASTER executionType: GUARANTEED 2016-05-05 17:27:44,463 DEBUG [AsyncDispatcher event handler] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(221)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 2 cluster_timestamp: 1462440433865 } attemptId: 4 } id: 1 } nmHostAddr: "127.0.0.1:1234" appSubmitter: "moonfang" resource { memory: 1024 virtual_cores: 1 } expiryTimeStamp: 1462441064461 masterKeyId: 1543136108 rmIdentifier: 1462440449422 priority { priority: 0 } creationTime: 1462440464460 nodeLabelExpression: "" containerType: APPLICATION_MASTER executionType: GUARANTEED 2016-05-05 17:27:44,463 INFO [AsyncDispatcher event handler] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:createAndGetNMToken(200)) - Sending NMToken for nodeId : 127.0.0.1:1234 for container : container_1462440433865_0002_04_000001 2016-05-05 17:27:44,463 DEBUG [AsyncDispatcher event handler] security.BaseNMTokenSecretManager (BaseNMTokenSecretManager.java:createPassword(75)) - creating password for appattempt_1462440433865_0002_000004 for user moonfang to run on NM 127.0.0.1:1234 2016-05-05 17:27:44,464 DEBUG [AsyncDispatcher event handler] security.NMTokenIdentifier (NMTokenIdentifier.java:write(96)) - Writing NMTokenIdentifier to RPC layer: appAttemptId { application_id { id: 2 cluster_timestamp: 1462440433865 } attemptId: 4 } nodeId { host: "127.0.0.1" port: 1234 } appSubmitter: "moonfang" keyId: -1949980208 2016-05-05 17:27:44,464 DEBUG [AsyncDispatcher event handler] security.NMTokenIdentifier (NMTokenIdentifier.java:write(96)) - Writing NMTokenIdentifier to RPC layer: appAttemptId { application_id { id: 2 cluster_timestamp: 1462440433865 } attemptId: 4 } nodeId { host: "127.0.0.1" port: 1234 } appSubmitter: "moonfang" keyId: -1949980208 2016-05-05 17:27:44,464 DEBUG [AsyncDispatcher event handler] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(439)) - Processing container_1462440433865_0002_04_000001 of type ACQUIRED 2016-05-05 17:27:44,465 INFO [AsyncDispatcher event handler] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(453)) - container_1462440433865_0002_04_000001 Container Transitioned from ALLOCATED to ACQUIRED 2016-05-05 17:27:44,465 DEBUG [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1190)) - User limit computation for moonfang in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-05-05 17:27:44,465 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1190)) - User limit computation for moonfang in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-05-05 17:27:44,465 DEBUG [SchedulerEventDispatcher:Event Processor] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(480)) - showRequests: application=application_1462440433865_0002 headRoom= currentConsumption=1024 2016-05-05 17:27:44,465 DEBUG [SchedulerEventDispatcher:Event Processor] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(484)) - showRequests: application=application_1462440433865_0002 request={Priority: 0, Capability: , # Containers: 0, Location: *, Relax Locality: true, Node Label Expression: } 2016-05-05 17:27:44,465 INFO [AsyncDispatcher event handler] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:clearNodeSetForAttempt(146)) - Clear node set for appattempt_1462440433865_0002_000004 2016-05-05 17:27:44,466 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1190)) - User limit computation for moonfang in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-05-05 17:27:44,466 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:storeAttempt(2075)) - Storing attempt: AppId: application_1462440433865_0002 AttemptId: appattempt_1462440433865_0002_000004 MasterContainer: Container: [ContainerId: container_1462440433865_0002_04_000001, NodeId: 127.0.0.1:1234, NodeHttpAddress: 127.0.0.1:2, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 127.0.0.1:1234 }, ] 2016-05-05 17:27:44,466 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1190)) - User limit computation for moonfang in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-05-05 17:27:44,466 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1190)) - User limit computation for moonfang in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-05-05 17:27:44,466 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStoreAppAttemptEvent.EventType: STORE_APP_ATTEMPT 2016-05-05 17:27:44,466 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(871)) - appattempt_1462440433865_0002_000004 State change from SCHEDULED to ALLOCATED_SAVING 2016-05-05 17:27:44,466 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppRunningOnNodeEvent.EventType: APP_RUNNING_ON_NODE 2016-05-05 17:27:44,466 DEBUG [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:handleStoreEvent(1070)) - Processing event of type STORE_APP_ATTEMPT 2016-05-05 17:27:44,466 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:allocateResource(1401)) - default user=moonfang used= numContainers=1 headroom = user-resources= 2016-05-05 17:27:44,467 DEBUG [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:transition(300)) - Storing info for attempt: appattempt_1462440433865_0002_000004 2016-05-05 17:27:44,467 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(811)) - Processing event for application_1462440433865_0002 of type APP_RUNNING_ON_NODE 2016-05-05 17:27:44,467 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(259)) - AppAttempt : appattempt_1462440433865_0002_000004 State is : ALLOCATED_SAVING Waiting for state : ALLOCATED 2016-05-05 17:27:44,467 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(592)) - Assigned to queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.128, absoluteUsedCapacity=0.128, numApps=1, numContainers=1 --> , OFF_SWITCH 2016-05-05 17:27:44,467 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: ATTEMPT_NEW_SAVED 2016-05-05 17:27:44,467 INFO [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(606)) - Re-sorting assigned queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.128, absoluteUsedCapacity=0.128, numApps=1, numContainers=1 2016-05-05 17:27:44,467 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(859)) - Processing event for appattempt_1462440433865_0002_000004 of type ATTEMPT_NEW_SAVED 2016-05-05 17:27:44,467 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:printChildQueues(634)) - printChildQueues - queue: root child-queues: root.defaultusedCapacity=(0.128), label=(*) 2016-05-05 17:27:44,468 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(871)) - appattempt_1462440433865_0002_000004 State change from ALLOCATED_SAVING to ALLOCATED 2016-05-05 17:27:44,468 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncherEvent.EventType: LAUNCH 2016-05-05 17:27:44,468 INFO [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(469)) - assignedContainer queue=root usedCapacity=0.128 absoluteUsedCapacity=0.128 used= cluster= 2016-05-05 17:27:44,468 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(481)) - ParentQ=root assignedSoFarInThisIteration= usedCapacity=0.128 absoluteUsedCapacity=0.128 2016-05-05 17:27:44,468 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(492)) - Not assigning more than one off-switch container, assignments so far: resource:; type:OFF_SWITCH; excessReservation:null; applicationid:null; skipped:false; fulfilled reservation:false; allocations(count/resource):1/; reservations(count/resource):0/ 2016-05-05 17:27:44,477 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(265)) - Attempt State is : ALLOCATED 2016-05-05 17:27:44,477 INFO [Thread-0] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:createAndGetAMRMToken(195)) - Create AMRMToken for ApplicationAttempt: appattempt_1462440433865_0002_000004 2016-05-05 17:27:44,477 INFO [Thread-0] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:createPassword(307)) - Creating password for appattempt_1462440433865_0002_000004 2016-05-05 17:27:44,478 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: LAUNCHED 2016-05-05 17:27:44,478 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(259)) - AppAttempt : appattempt_1462440433865_0002_000004 State is : ALLOCATED Waiting for state : LAUNCHED 2016-05-05 17:27:44,478 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(859)) - Processing event for appattempt_1462440433865_0002_000004 of type LAUNCHED 2016-05-05 17:27:44,478 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(871)) - appattempt_1462440433865_0002_000004 State change from ALLOCATED to LAUNCHED 2016-05-05 17:27:44,488 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(265)) - Attempt State is : LAUNCHED 2016-05-05 17:27:44,488 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(265)) - Attempt State is : LAUNCHED 2016-05-05 17:27:44,489 DEBUG [Thread-0] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1780)) - PrivilegedAction as:appattempt_1462440433865_0002_000004 (auth:SIMPLE) from:org.apache.hadoop.yarn.server.resourcemanager.MockAM.registerAppAttempt(MockAM.java:116) 2016-05-05 17:27:44,489 INFO [Thread-0] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:registerApplicationMaster(235)) - AM registration appattempt_1462440433865_0002_000004 2016-05-05 17:27:44,489 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptRegistrationEvent.EventType: REGISTERED 2016-05-05 17:27:44,489 INFO [Thread-0] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(175)) - USER=moonfang OPERATION=Register App Master TARGET=ApplicationMasterService RESULT=SUCCESS APPID=application_1462440433865_0002 APPATTEMPTID=appattempt_1462440433865_0002_000004 2016-05-05 17:27:44,490 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(859)) - Processing event for appattempt_1462440433865_0002_000004 of type REGISTERED 2016-05-05 17:27:44,490 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(186)) - App : application_1462440433865_0002 State is : ACCEPTED Waiting for state : RUNNING 2016-05-05 17:27:44,490 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(871)) - appattempt_1462440433865_0002_000004 State change from LAUNCHED to RUNNING 2016-05-05 17:27:44,491 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEvent.EventType: ATTEMPT_REGISTERED 2016-05-05 17:27:44,491 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(811)) - Processing event for application_1462440433865_0002 of type ATTEMPT_REGISTERED 2016-05-05 17:27:44,491 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(823)) - application_1462440433865_0002 State change from ACCEPTED to RUNNING 2016-05-05 17:27:44,501 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(192)) - App State is : RUNNING 2016-05-05 17:27:44,501 INFO [Thread-0] mortbay.log (Slf4jLog.java:info(67)) - ContainerStatus: ContainerStatus: [ContainerId: container_1462440433865_0002_04_000001, ExecutionType: GUARANTEED, State: COMPLETE, Capability: , Diagnostics: Success, ExitStatus: 0, ] 2016-05-05 17:27:44,502 INFO [Thread-0] mortbay.log (Slf4jLog.java:info(67)) - entry.getValue() [ContainerStatus: [ContainerId: container_1462440433865_0002_04_000001, ExecutionType: GUARANTEED, State: COMPLETE, Capability: , Diagnostics: Success, ExitStatus: 0, ]] 2016-05-05 17:27:44,502 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2016-05-05 17:27:44,502 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(259)) - AppAttempt : appattempt_1462440433865_0002_000004 State is : RUNNING Waiting for state : FAILED 2016-05-05 17:27:44,502 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(618)) - Processing 127.0.0.1:1234 of type STATUS_UPDATE 2016-05-05 17:27:44,503 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handleContainerStatus(1326)) - Container container_1462440433865_0002_04_000001 is the first container get launched for application application_1462440433865_0002 2016-05-05 17:27:44,504 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2016-05-05 17:27:44,504 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1052)) - nodeUpdate: 127.0.0.1:1234 clusterResources: 2016-05-05 17:27:44,504 DEBUG [SchedulerEventDispatcher:Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(439)) - Processing container_1462440433865_0002_04_000001 of type FINISHED 2016-05-05 17:27:44,505 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptContainerFinishedEvent.EventType: CONTAINER_FINISHED 2016-05-05 17:27:44,505 INFO [SchedulerEventDispatcher:Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(453)) - container_1462440433865_0002_04_000001 Container Transitioned from ACQUIRED to COMPLETED 2016-05-05 17:27:44,505 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(859)) - Processing event for appattempt_1462440433865_0002_000004 of type CONTAINER_FINISHED 2016-05-05 17:27:44,505 INFO [SchedulerEventDispatcher:Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(154)) - USER=moonfang OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1462440433865_0002 CONTAINERID=container_1462440433865_0002_04_000001 RESOURCE= 2016-05-05 17:27:44,506 DEBUG [SchedulerEventDispatcher:Event Processor] scheduler.SchedulerNode (SchedulerNode.java:releaseContainer(269)) - Released container container_1462440433865_0002_04_000001 of capacity on host 127.0.0.1:1234, which currently has 0 containers, used and available, release resources=true 2016-05-05 17:27:44,506 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1190)) - User limit computation for moonfang in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2016-05-05 17:27:44,507 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:releaseResource(1437)) - default used= numContainers=0 user=moonfang user-resources= 2016-05-05 17:27:44,507 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:rememberTargetTransitionsAndStoreState(1301)) - Updating application attempt appattempt_1462440433865_0002_000004 with final state: FAILED, and exit status: 0 2016-05-05 17:27:44,508 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:internalReleaseResource(647)) - completedContainer root: numChildQueue= 1, capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.0, numApps=1, numContainers=0, cluster= 2016-05-05 17:27:44,508 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(871)) - appattempt_1462440433865_0002_000004 State change from RUNNING to FINAL_SAVING 2016-05-05 17:27:44,508 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateUpdateAppAttemptEvent.EventType: UPDATE_APP_ATTEMPT 2016-05-05 17:27:44,508 DEBUG [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:handleStoreEvent(1070)) - Processing event of type UPDATE_APP_ATTEMPT 2016-05-05 17:27:44,508 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2016-05-05 17:27:44,508 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:internalReleaseResource(660)) - Re-sorting completed queue: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=1, numContainers=0 2016-05-05 17:27:44,509 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(618)) - Processing 127.0.0.1:1234 of type FINISHED_CONTAINERS_PULLED_BY_AM 2016-05-05 17:27:44,509 DEBUG [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:transition(331)) - Updating info for attempt: appattempt_1462440433865_0002_000004 2016-05-05 17:27:44,509 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1125)) - Node being looked for scheduling 127.0.0.1:1234 availableResource: 2016-05-05 17:27:44,510 INFO [AsyncDispatcher event handler] recovery.RMStateStore (MemoryRMStateStore.java:updateApplicationAttemptStateInternal(139)) - Updating final state FAILED for attempt: appattempt_1462440433865_0002_000004 2016-05-05 17:27:44,510 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1276)) - Trying to schedule on node: 127.0.0.1, available: 2016-05-05 17:27:44,510 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: ATTEMPT_UPDATE_SAVED 2016-05-05 17:27:44,510 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(404)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-05-05 17:27:44,510 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(859)) - Processing event for appattempt_1462440433865_0002_000004 of type ATTEMPT_UPDATE_SAVED 2016-05-05 17:27:44,511 INFO [AsyncDispatcher event handler] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:unregisterAttempt(682)) - Unregistering app attempt : appattempt_1462440433865_0002_000004 2016-05-05 17:27:44,511 INFO [AsyncDispatcher event handler] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:applicationMasterFinished(124)) - Application finished, removing password for appattempt_1462440433865_0002_000004 2016-05-05 17:27:44,511 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(871)) - appattempt_1462440433865_0002_000004 State change from FINAL_SAVING to FAILED 2016-05-05 17:27:44,512 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppFailedAttemptEvent.EventType: ATTEMPT_FAILED 2016-05-05 17:27:44,512 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(811)) - Processing event for application_1462440433865_0002 of type ATTEMPT_FAILED 2016-05-05 17:27:44,512 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:transition(1354)) - The number of failed attempts in previous 10000 milliseconds is 1. The max attempts is 2 2016-05-05 17:27:44,512 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:removeExcessAttempts(1413)) - Remove attempt from state store : appattempt_1462440433865_0002_000002 2016-05-05 17:27:44,513 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStoreRemoveAppAttemptEvent.EventType: REMOVE_APP_ATTEMPT 2016-05-05 17:27:44,513 INFO [Thread-0] resourcemanager.MockRM (MockRM.java:waitForState(265)) - Attempt State is : FAILED 2016-05-05 17:27:44,513 DEBUG [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:handleStoreEvent(1070)) - Processing event of type REMOVE_APP_ATTEMPT 2016-05-05 17:27:44,513 INFO [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:transition(574)) - Removing attempt appattempt_1462440433865_0002_000002 from app: application_1462440433865_0002 2016-05-05 17:27:44,514 INFO [AsyncDispatcher event handler] recovery.RMStateStore (MemoryRMStateStore.java:removeApplicationAttemptInternal(151)) - Removing state for attempt: appattempt_1462440433865_0002_000002 2016-05-05 17:27:44,514 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(823)) - application_1462440433865_0002 State change from RUNNING to ACCEPTED 2016-05-05 17:27:44,514 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.AppAttemptRemovedSchedulerEvent.EventType: APP_ATTEMPT_REMOVED 2016-05-05 17:27:44,514 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncherEvent.EventType: CLEANUP 2016-05-05 17:27:44,515 INFO [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:doneApplicationAttempt(864)) - Application Attempt appattempt_1462440433865_0002_000004 is done. finalState=FAILED 2016-05-05 17:27:44,515 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppStartAttemptEvent.EventType: START 2016-05-05 17:27:44,515 INFO [SchedulerEventDispatcher:Event Processor] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:clearRequests(135)) - Application application_1462440433865_0002 requests cleared 2016-05-05 17:27:44,515 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(859)) - Processing event for appattempt_1462440433865_0002_000005 of type START 2016-05-05 17:27:44,515 INFO [AsyncDispatcher event handler] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:registerAppAttempt(676)) - Registering app attempt : appattempt_1462440433865_0002_000005 2016-05-05 17:27:44,516 INFO [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:removeApplicationAttempt(793)) - Application removed - appId: application_1462440433865_0002 user: moonfang queue: default #user-pending-applications: 0 #user-active-applications: 0 #queue-pending-applications: 0 #queue-active-applications: 0 2016-05-05 17:27:44,516 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(871)) - appattempt_1462440433865_0002_000005 State change from NEW to SUBMITTED 2016-05-05 17:27:44,516 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.AppAttemptAddedSchedulerEvent.EventType: APP_ATTEMPT_ADDED 2016-05-05 17:27:44,517 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:activateApplications(652)) - application application_1462440433865_0002 AMResource maxAMResourcePerQueuePercent 0.1 amLimit lastClusterResource amIfStarted AM node-partition name 2016-05-05 17:27:44,517 INFO [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:activateApplications(726)) - Application application_1462440433865_0002 from user: moonfang activated in queue: default 2016-05-05 17:27:44,517 INFO [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue (LeafQueue.java:addApplicationAttempt(741)) - Application added - appId: application_1462440433865_0002 user: moonfang, leaf-queue: default #user-pending-applications: 0 #user-active-applications: 1 #queue-pending-applications: 0 #queue-active-applications: 1 2016-05-05 17:27:44,518 INFO [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:addApplicationAttempt(825)) - Added Application Attempt appattempt_1462440433865_0002_000005 to scheduler from user moonfang in queue default 2016-05-05 17:27:44,518 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: ATTEMPT_ADDED 2016-05-05 17:27:44,518 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(859)) - Processing event for appattempt_1462440433865_0002_000005 of type ATTEMPT_ADDED 2016-05-05 17:27:44,518 WARN [AsyncDispatcher event handler] blacklist.SimpleBlacklistManager (SimpleBlacklistManager.java:getBlacklistUpdates(75)) - Ignoring Blacklists, blacklist size 1 is more than failure threshold ratio 0.800000011920929 out of total usable nodes 1 2016-05-05 17:27:44,518 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:transition(1041)) - Using blacklist for AM: additions([]) and removals([127.0.0.1]) 2016-05-05 17:27:44,519 DEBUG [AsyncDispatcher event handler] capacity.CapacityScheduler (CapacityScheduler.java:allocate(984)) - allocate: pre-update appattempt_1462440433865_0002_000005 ask size =1 2016-05-05 17:27:44,519 DEBUG [AsyncDispatcher event handler] scheduler.ActiveUsersManager (ActiveUsersManager.java:activateApplication(68)) - User moonfang added to activeUsers, currently: 1 2016-05-05 17:27:44,519 DEBUG [AsyncDispatcher event handler] capacity.CapacityScheduler (CapacityScheduler.java:allocate(996)) - allocate: post-update 2016-05-05 17:27:44,519 DEBUG [AsyncDispatcher event handler] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(480)) - showRequests: application=application_1462440433865_0002 headRoom= currentConsumption=0 2016-05-05 17:27:44,519 DEBUG [AsyncDispatcher event handler] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(484)) - showRequests: application=application_1462440433865_0002 request={Priority: 0, Capability: , # Containers: 1, Location: *, Relax Locality: true, Node Label Expression: } 2016-05-05 17:27:44,520 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(871)) - appattempt_1462440433865_0002_000005 State change from SUBMITTED to SCHEDULED