2015-11-22 19:23:04,929 INFO [main] distributedshell.TestDistributedShell (TestDistributedShell.java:setupInternal(77)) - Starting up YARN cluster 2015-11-22 19:23:05,235 WARN [main] conf.Configuration (Configuration.java:loadProperty(2687)) - yarn-site.xml:an attempt to override final parameter: hadoop.tmp.dir; Ignoring. 2015-11-22 19:23:05,251 INFO [main] net.ServerSocketUtil (ServerSocketUtil.java:getPort(52)) - Using port 10021 2015-11-22 19:23:05,413 WARN [main] conf.Configuration (Configuration.java:loadProperty(2687)) - yarn-site.xml:an attempt to override final parameter: hadoop.tmp.dir; Ignoring. 2015-11-22 19:23:05,477 WARN [main] conf.Configuration (Configuration.java:loadProperty(2687)) - yarn-site.xml:an attempt to override final parameter: hadoop.tmp.dir; Ignoring. 2015-11-22 19:23:05,513 WARN [main] util.NativeCodeLoader (NativeCodeLoader.java:(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2015-11-22 19:23:05,609 WARN [main] conf.Configuration (Configuration.java:loadProperty(2687)) - yarn-site.xml:an attempt to override final parameter: hadoop.tmp.dir; Ignoring. 2015-11-22 19:23:05,722 INFO [main] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2363)) - found resource core-site.xml at file:/home/ubuntu/hadoop-dev/hadoop-common-project/hadoop-common/target/test-classes/core-site.xml 2015-11-22 19:23:05,776 WARN [main] conf.Configuration (Configuration.java:loadProperty(2687)) - yarn-site.xml:an attempt to override final parameter: hadoop.tmp.dir; Ignoring. 2015-11-22 19:23:05,779 WARN [main] conf.Configuration (Configuration.java:loadProperty(2687)) - core-site.xml:an attempt to override final parameter: hadoop.tmp.dir; Ignoring. 2015-11-22 19:23:05,780 INFO [main] security.Groups (Groups.java:refresh(256)) - clearing userToGroupsMap cache 2015-11-22 19:23:05,810 WARN [main] conf.Configuration (Configuration.java:loadProperty(2687)) - yarn-site.xml:an attempt to override final parameter: hadoop.tmp.dir; Ignoring. 2015-11-22 19:23:05,843 INFO [main] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2363)) - found resource yarn-site.xml at file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/test-classes/yarn-site.xml 2015-11-22 19:23:05,866 WARN [main] conf.Configuration (Configuration.java:loadProperty(2687)) - yarn-site.xml:an attempt to override final parameter: hadoop.tmp.dir; Ignoring. 2015-11-22 19:23:05,879 WARN [main] conf.Configuration (Configuration.java:loadProperty(2687)) - yarn-site.xml:an attempt to override final parameter: hadoop.tmp.dir; Ignoring. 2015-11-22 19:23:05,884 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.RMFatalEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMFatalEventDispatcher 2015-11-22 19:23:06,076 INFO [main] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:(75)) - NMTokenKeyRollingInterval: 86400000ms and NMTokenKeyActivationDelay: 900000ms 2015-11-22 19:23:06,081 INFO [main] security.RMContainerTokenSecretManager (RMContainerTokenSecretManager.java:(78)) - ContainerTokenKeyRollingInterval: 86400000ms and ContainerTokenKeyActivationDelay: 900000ms 2015-11-22 19:23:06,088 INFO [main] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:(94)) - AMRMTokenKeyRollingInterval: 86400000ms and AMRMTokenKeyActivationDelay: 900000 ms 2015-11-22 19:23:06,133 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStoreEventType for class org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$ForwardingEventHandler 2015-11-22 19:23:06,136 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.NodesListManagerEventType for class org.apache.hadoop.yarn.server.resourcemanager.NodesListManager 2015-11-22 19:23:06,137 INFO [main] resourcemanager.ResourceManager (ResourceManager.java:createScheduler(295)) - Using Scheduler: org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler 2015-11-22 19:23:06,158 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.SchedulerEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher 2015-11-22 19:23:06,159 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationEventDispatcher 2015-11-22 19:23:06,161 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationAttemptEventDispatcher 2015-11-22 19:23:06,162 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$NodeEventDispatcher 2015-11-22 19:23:06,222 INFO [main] impl.MetricsConfig (MetricsConfig.java:loadFirst(115)) - loaded properties from hadoop-metrics2.properties 2015-11-22 19:23:06,283 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:startTimer(374)) - Scheduled snapshot period at 10 second(s). 2015-11-22 19:23:06,284 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:start(192)) - ResourceManager metrics system started 2015-11-22 19:23:06,302 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.RMAppManagerEventType for class org.apache.hadoop.yarn.server.resourcemanager.RMAppManager 2015-11-22 19:23:06,308 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncherEventType for class org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher 2015-11-22 19:23:06,310 INFO [main] resourcemanager.RMNMInfo (RMNMInfo.java:(63)) - Registered RMNMInfo MBean 2015-11-22 19:23:06,316 INFO [main] security.YarnAuthorizationProvider (YarnAuthorizationProvider.java:getInstance(57)) - org.apache.hadoop.yarn.security.ConfiguredYarnAuthorizer is instiantiated. 2015-11-22 19:23:06,317 INFO [main] util.HostsFileReader (HostsFileReader.java:refresh(132)) - Refreshing hosts (include/exclude) list 2015-11-22 19:23:06,324 INFO [main] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2363)) - found resource capacity-scheduler.xml at jar:file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/target/hadoop-yarn-server-tests-2.8.0-SNAPSHOT-tests.jar!/capacity-scheduler.xml 2015-11-22 19:23:06,342 WARN [main] conf.Configuration (Configuration.java:loadProperty(2687)) - yarn-site.xml:an attempt to override final parameter: hadoop.tmp.dir; Ignoring. 2015-11-22 19:23:06,375 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(647)) - max alloc mb per queue for root is undefined 2015-11-22 19:23:06,376 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(651)) - max alloc vcore per queue for root is undefined 2015-11-22 19:23:06,385 INFO [main] capacity.ParentQueue (ParentQueue.java:setupQueueConfigs(125)) - root, capacity=1.0, asboluteCapacity=1.0, maxCapacity=1.0, asboluteMaxCapacity=1.0, state=RUNNING, acls=ADMINISTER_QUEUE:*SUBMIT_APP:*, labels=*, , reservationsContinueLooking=true 2015-11-22 19:23:06,385 INFO [main] capacity.ParentQueue (ParentQueue.java:(104)) - Initialized parent-queue root name=root, fullname=root 2015-11-22 19:23:06,395 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(647)) - max alloc mb per queue for root.default is undefined 2015-11-22 19:23:06,395 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(651)) - max alloc vcore per queue for root.default is undefined 2015-11-22 19:23:06,399 INFO [main] capacity.LeafQueue (LeafQueue.java:setupQueueConfigs(217)) - Initializing default capacity = 1.0 [= (float) configuredCapacity / 100 ] asboluteCapacity = 1.0 [= parentAbsoluteCapacity * capacity ] maxCapacity = 1.0 [= configuredMaxCapacity ] absoluteMaxCapacity = 1.0 [= 1.0 maximumCapacity undefined, (parentAbsoluteMaxCapacity * maximumCapacity) / 100 otherwise ] userLimit = 100 [= configuredUserLimit ] userLimitFactor = 1.0 [= configuredUserLimitFactor ] maxApplications = 10000 [= configuredMaximumSystemApplicationsPerQueue or (int)(configuredMaximumSystemApplications * absoluteCapacity)] maxApplicationsPerUser = 10000 [= (int)(maxApplications * (userLimit / 100.0f) * userLimitFactor) ] usedCapacity = 0.0 [= usedResourcesMemory / (clusterResourceMemory * absoluteCapacity)] absoluteUsedCapacity = 0.0 [= usedResourcesMemory / clusterResourceMemory] maxAMResourcePerQueuePercent = 0.1 [= configuredMaximumAMResourcePercent ] minimumAllocationFactor = 0.984375 [= (float)(maximumAllocationMemory - minimumAllocationMemory) / maximumAllocationMemory ] maximumAllocation = [= configuredMaxAllocation ] numContainers = 0 [= currentNumContainers ] state = RUNNING [= configuredState ] acls = ADMINISTER_QUEUE:*SUBMIT_APP:* [= configuredAcls ] nodeLocalityDelay = -1 labels=*, nodeLocalityDelay = -1 reservationsContinueLooking = true preemptionDisabled = true defaultAppPriorityPerQueue = 0 2015-11-22 19:23:06,400 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:parseQueue(657)) - Initialized queue: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=0, numContainers=0 2015-11-22 19:23:06,400 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:parseQueue(657)) - Initialized queue: root: numChildQueue= 1, capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.0, numApps=0, numContainers=0 2015-11-22 19:23:06,401 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:initializeQueues(501)) - Initialized root queue root: numChildQueue= 1, capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.0, numApps=0, numContainers=0 2015-11-22 19:23:06,401 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:getUserGroupMappingPlacementRule(449)) - Initialized queue mappings, override: false 2015-11-22 19:23:06,402 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:initScheduler(314)) - Initialized CapacityScheduler with calculator=class org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator, minimumAllocation=<>, maximumAllocation=<>, asynchronousScheduling=false, asyncScheduleInterval=5ms 2015-11-22 19:23:06,412 INFO [main] metrics.SystemMetricsPublisher (SystemMetricsPublisher.java:serviceInit(95)) - YARN system metrics publishing service is not enabled 2015-11-22 19:23:06,413 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEventType for class org.apache.hadoop.yarn.server.MiniYARNCluster$1 2015-11-22 19:23:06,414 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:prepareDirs(558)) - Created localDir in /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0 2015-11-22 19:23:06,414 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:prepareDirs(558)) - Created logDir in /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-logDir-nm-0_0 2015-11-22 19:23:06,415 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceInit(540)) - Starting NM: 0 2015-11-22 19:23:06,443 WARN [main] conf.Configuration (Configuration.java:loadProperty(2687)) - yarn-site.xml:an attempt to override final parameter: hadoop.tmp.dir; Ignoring. 2015-11-22 19:23:06,464 WARN [main] conf.Configuration (Configuration.java:loadProperty(2687)) - yarn-site.xml:an attempt to override final parameter: hadoop.tmp.dir; Ignoring. 2015-11-22 19:23:06,470 INFO [main] nodemanager.NodeManager (NodeManager.java:getNodeHealthScriptRunner(255)) - Node Manager health check script is not available or doesn't have execute permission, so not starting the node health script runner. 2015-11-22 19:23:06,511 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ContainerEventDispatcher 2015-11-22 19:23:06,512 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ApplicationEventDispatcher 2015-11-22 19:23:06,513 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizationEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService 2015-11-22 19:23:06,513 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServicesEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices 2015-11-22 19:23:06,514 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl 2015-11-22 19:23:06,515 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncherEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncher 2015-11-22 19:23:06,523 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.ContainerManagerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl 2015-11-22 19:23:06,523 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.NodeManagerEventType for class org.apache.hadoop.yarn.server.MiniYARNCluster$ShortCircuitedNodeManager 2015-11-22 19:23:06,524 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:init(159)) - NodeManager metrics system started (again) 2015-11-22 19:23:06,553 INFO [main] nodemanager.NodeResourceMonitorImpl (NodeResourceMonitorImpl.java:serviceInit(76)) - Using ResourceCalculatorPlugin : org.apache.hadoop.yarn.util.ResourceCalculatorPlugin@c8c12ac 2015-11-22 19:23:06,555 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.event.LogHandlerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.NonAggregatingLogHandler 2015-11-22 19:23:06,556 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.sharedcache.SharedCacheUploadEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.sharedcache.SharedCacheUploadService 2015-11-22 19:23:06,557 INFO [main] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:serviceInit(261)) - AMRMProxyService is disabled 2015-11-22 19:23:06,557 INFO [main] localizer.ResourceLocalizationService (ResourceLocalizationService.java:validateConf(224)) - per directory file limit = 8192 2015-11-22 19:23:06,586 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerTracker 2015-11-22 19:23:06,587 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(116)) - Using ResourceCalculatorPlugin : org.apache.hadoop.yarn.util.ResourceCalculatorPlugin@223191a6 2015-11-22 19:23:06,588 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(121)) - Using ResourceCalculatorProcessTree : null 2015-11-22 19:23:06,589 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(158)) - Physical memory check enabled: false 2015-11-22 19:23:06,589 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(159)) - Virtual memory check enabled: false 2015-11-22 19:23:06,592 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:isEnabled(206)) - Neither virutal-memory nor physical-memory monitoring is needed. Not running the monitor-thread 2015-11-22 19:23:06,592 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(162)) - ContainersMonitor enabled: false 2015-11-22 19:23:06,594 INFO [main] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:serviceInit(175)) - Nodemanager resources: memory set to 4096MB. 2015-11-22 19:23:06,594 INFO [main] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:serviceInit(176)) - Nodemanager resources: vcores set to 8. 2015-11-22 19:23:06,595 INFO [main] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:serviceInit(208)) - Initialized nodemanager with : physical-memory=4096 virtual-memory=8602 virtual-cores=8 2015-11-22 19:23:06,609 WARN [main] applicationhistoryservice.ApplicationHistoryServer (ApplicationHistoryServer.java:createApplicationHistoryManager(218)) - The filesystem based application history store is deprecated. 2015-11-22 19:23:06,612 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:init(159)) - ApplicationHistoryServer metrics system started (again) 2015-11-22 19:23:06,619 INFO [main] applicationhistoryservice.ApplicationHistoryManagerImpl (ApplicationHistoryManagerImpl.java:serviceInit(61)) - ApplicationHistory Init 2015-11-22 19:23:06,621 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:startResourceManager(326)) - Waiting for RM to start... 2015-11-22 19:23:06,621 INFO [RM-0] resourcemanager.ResourceManager (ResourceManager.java:transitionToActive(1024)) - Transitioning to active state 2015-11-22 19:23:06,640 INFO [RM-0] recovery.RMStateStore (RMStateStore.java:transition(482)) - Updating AMRMToken 2015-11-22 19:23:06,642 INFO [RM-0] security.RMContainerTokenSecretManager (RMContainerTokenSecretManager.java:rollMasterKey(106)) - Rolling master-key for container-tokens 2015-11-22 19:23:06,643 INFO [RM-0] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:rollMasterKey(95)) - Rolling master-key for nm-tokens 2015-11-22 19:23:06,644 INFO [RM-0] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:23:06,645 INFO [RM-0] security.RMDelegationTokenSecretManager (RMDelegationTokenSecretManager.java:storeNewMasterKey(87)) - storing master key with keyID 1 2015-11-22 19:23:06,645 INFO [RM-0] recovery.RMStateStore (RMStateStore.java:transition(433)) - Storing RMDTMasterKey. 2015-11-22 19:23:06,647 INFO [Thread[Thread-24,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2015-11-22 19:23:06,647 INFO [Thread[Thread-24,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:23:06,648 INFO [Thread[Thread-24,5,main]] security.RMDelegationTokenSecretManager (RMDelegationTokenSecretManager.java:storeNewMasterKey(87)) - storing master key with keyID 2 2015-11-22 19:23:06,653 INFO [Thread[Thread-24,5,main]] recovery.RMStateStore (RMStateStore.java:transition(433)) - Storing RMDTMasterKey. 2015-11-22 19:23:06,670 INFO [RM-0] nodelabels.CommonNodeLabelsManager (RMNodeLabelsManager.java:replaceLabelsOnNode(170)) - No Modified Node label Mapping to replace 2015-11-22 19:23:06,675 INFO [RM-0] nodelabels.CommonNodeLabelsManager (CommonNodeLabelsManager.java:addToCluserNodeLabels(309)) - Add labels: [] 2015-11-22 19:23:06,682 INFO [RM-0] nodelabels.CommonNodeLabelsManager (CommonNodeLabelsManager.java:internalUpdateLabelsOnNodes(638)) - REPLACE labels on nodes: 2015-11-22 19:23:06,682 INFO [RM-0] nodelabels.CommonNodeLabelsManager (CommonNodeLabelsManager.java:internalUpdateLabelsOnNodes(640)) - NM=ip-172-31-20-42.ap-northeast-1.compute.internal:58694, labels=[x] 2015-11-22 19:23:06,691 INFO [RM-0] nodelabels.FileSystemNodeLabelsStore (FileSystemNodeLabelsStore.java:recover(279)) - Finished write mirror at:file:/tmp/hadoop-yarn-ubuntu/node-labels/nodelabel.mirror 2015-11-22 19:23:06,691 INFO [RM-0] nodelabels.FileSystemNodeLabelsStore (FileSystemNodeLabelsStore.java:recover(280)) - Finished create editlog file at:file:/tmp/hadoop-yarn-ubuntu/node-labels/nodelabel.editlog 2015-11-22 19:23:06,692 INFO [RM-0] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.nodelabels.event.NodeLabelsStoreEventType for class org.apache.hadoop.yarn.nodelabels.CommonNodeLabelsManager$ForwardingEventHandler 2015-11-22 19:23:06,714 INFO [RM-0] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:23:06,723 INFO [Socket Reader #1 for port 43240] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 43240 2015-11-22 19:23:06,735 INFO [RM-0] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.server.api.ResourceTrackerPB to the server 2015-11-22 19:23:06,736 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:23:06,736 INFO [IPC Server listener on 43240] ipc.Server (Server.java:run(794)) - IPC Server listener on 43240: starting 2015-11-22 19:23:06,776 INFO [RM-0] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:23:06,783 INFO [Socket Reader #1 for port 51338] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 51338 2015-11-22 19:23:06,798 INFO [RM-0] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB to the server 2015-11-22 19:23:06,799 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:23:06,799 INFO [IPC Server listener on 51338] ipc.Server (Server.java:run(794)) - IPC Server listener on 51338: starting 2015-11-22 19:23:06,877 INFO [RM-0] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:23:06,878 INFO [Socket Reader #1 for port 35548] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 35548 2015-11-22 19:23:06,880 INFO [RM-0] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.api.ApplicationClientProtocolPB to the server 2015-11-22 19:23:06,880 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:23:06,880 INFO [IPC Server listener on 35548] ipc.Server (Server.java:run(794)) - IPC Server listener on 35548: starting 2015-11-22 19:23:06,888 INFO [RM-0] resourcemanager.ResourceManager (ResourceManager.java:transitionToActive(1040)) - Transitioned to active state 2015-11-22 19:23:06,959 INFO [RM-0] mortbay.log (Slf4jLog.java:info(67)) - Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2015-11-22 19:23:06,968 INFO [RM-0] server.AuthenticationFilter (AuthenticationFilter.java:constructSecretProvider(304)) - Unable to initialize FileSignerSecretProvider, falling back to use random secrets. 2015-11-22 19:23:06,974 INFO [RM-0] http.HttpRequestLog (HttpRequestLog.java:getRequestLog(80)) - Http request log for http.requests.resourcemanager is not defined 2015-11-22 19:23:06,979 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter) 2015-11-22 19:23:06,982 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context cluster 2015-11-22 19:23:06,982 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context logs 2015-11-22 19:23:06,983 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context static 2015-11-22 19:23:06,983 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context cluster 2015-11-22 19:23:06,983 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2015-11-22 19:23:06,983 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2015-11-22 19:23:06,986 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'Timeline Authentication Filter' (class=org.apache.hadoop.yarn.server.timeline.security.TimelineAuthenticationFilter) 2015-11-22 19:23:06,990 INFO [RM-0] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /cluster/* 2015-11-22 19:23:06,991 INFO [RM-0] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /ws/* 2015-11-22 19:23:07,304 INFO [RM-0] webapp.WebApps (WebApps.java:build(288)) - Registered webapp guice modules 2015-11-22 19:23:07,307 INFO [RM-0] http.HttpServer2 (HttpServer2.java:openListeners(917)) - Jetty bound to port 59535 2015-11-22 19:23:07,307 INFO [RM-0] mortbay.log (Slf4jLog.java:info(67)) - jetty-6.1.26 2015-11-22 19:23:07,406 INFO [RM-0] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:23:07,407 INFO [Thread[Thread-195,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2015-11-22 19:23:07,407 INFO [Thread[Thread-195,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:23:07,409 INFO [RM-0] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:23:07,410 INFO [Thread[Thread-197,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2015-11-22 19:23:07,410 INFO [Thread[Thread-197,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens Nov 22, 2015 7:23:07 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.resourcemanager.webapp.JAXBContextResolver as a provider class Nov 22, 2015 7:23:07 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices as a root resource class Nov 22, 2015 7:23:07 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a provider class Nov 22, 2015 7:23:07 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate INFO: Initiating Jersey application, version 'Jersey: 1.9 09/02/2011 11:17 AM' Nov 22, 2015 7:23:07 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.resourcemanager.webapp.JAXBContextResolver to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:23:08 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.webapp.GenericExceptionHandler to GuiceManagedComponentProvider with the scope "Singleton" 2015-11-22 19:23:08,121 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:startResourceManager(339)) - MiniYARN ResourceManager address: ip-172-31-20-42.ap-northeast-1.compute.internal:35548 2015-11-22 19:23:08,121 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:startResourceManager(341)) - MiniYARN ResourceManager web address: ip-172-31-20-42.ap-northeast-1.compute.internal:0 2015-11-22 19:23:08,122 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(459)) - MiniYARN ResourceManager address: ip-172-31-20-42.ap-northeast-1.compute.internal:35548 2015-11-22 19:23:08,122 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(461)) - MiniYARN ResourceManager web address: ip-172-31-20-42.ap-northeast-1.compute.internal:0 2015-11-22 19:23:08,124 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(575)) - Waiting for NM 0 to start... 2015-11-22 19:23:08,155 INFO [Thread-199] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:23:08,167 INFO [Socket Reader #1 for port 52784] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 52784 2015-11-22 19:23:08,175 INFO [Thread-199] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.api.ContainerManagementProtocolPB to the server 2015-11-22 19:23:08,175 INFO [Thread-199] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:serviceStart(464)) - Blocking new container-requests as container manager rpc server is still starting. 2015-11-22 19:23:08,181 INFO [IPC Server listener on 52784] ipc.Server (Server.java:run(794)) - IPC Server listener on 52784: starting 2015-11-22 19:23:08,181 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:23:08,187 INFO [Thread-199] security.NMContainerTokenSecretManager (NMContainerTokenSecretManager.java:setNodeId(260)) - Updating node address : ip-172-31-20-42.ap-northeast-1.compute.internal:52784 2015-11-22 19:23:08,198 INFO [Thread-199] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:23:08,204 INFO [Socket Reader #1 for port 44577] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 44577 2015-11-22 19:23:08,207 INFO [Thread-199] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB to the server 2015-11-22 19:23:08,208 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:23:08,212 INFO [IPC Server listener on 44577] ipc.Server (Server.java:run(794)) - IPC Server listener on 44577: starting 2015-11-22 19:23:08,253 INFO [Thread-199] localizer.ResourceLocalizationService (ResourceLocalizationService.java:serviceStart(363)) - Localizer started on port 44577 2015-11-22 19:23:08,284 INFO [Thread-199] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:serviceStart(509)) - ContainerManager started at /172.31.20.42:52784 2015-11-22 19:23:08,284 INFO [Thread-199] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:serviceStart(510)) - ContainerManager bound to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:0 2015-11-22 19:23:08,284 INFO [Thread-199] webapp.WebServer (WebServer.java:serviceStart(71)) - Instantiating NMWebApp at ip-172-31-20-42.ap-northeast-1.compute.internal:0 2015-11-22 19:23:08,286 INFO [Thread-199] server.AuthenticationFilter (AuthenticationFilter.java:constructSecretProvider(304)) - Unable to initialize FileSignerSecretProvider, falling back to use random secrets. 2015-11-22 19:23:08,287 INFO [Thread-199] http.HttpRequestLog (HttpRequestLog.java:getRequestLog(80)) - Http request log for http.requests.nodemanager is not defined 2015-11-22 19:23:08,288 INFO [Thread-199] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter) 2015-11-22 19:23:08,289 INFO [Thread-199] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context node 2015-11-22 19:23:08,290 INFO [Thread-199] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context logs 2015-11-22 19:23:08,290 INFO [Thread-199] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context static 2015-11-22 19:23:08,290 INFO [Thread-199] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context node 2015-11-22 19:23:08,290 INFO [Thread-199] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2015-11-22 19:23:08,291 INFO [Thread-199] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2015-11-22 19:23:08,292 INFO [Thread-199] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'Timeline Authentication Filter' (class=org.apache.hadoop.yarn.server.timeline.security.TimelineAuthenticationFilter) 2015-11-22 19:23:08,293 INFO [Thread-199] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /node/* 2015-11-22 19:23:08,294 INFO [Thread-199] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /ws/* Nov 22, 2015 7:23:08 PM com.google.inject.servlet.GuiceFilter setPipeline WARNING: Multiple Servlet injectors detected. This is a warning indicating that you have more than one GuiceFilter running in your web application. If this is deliberate, you may safely ignore this message. If this is NOT deliberate however, your application may not work as expected. 2015-11-22 19:23:08,319 INFO [Thread-199] webapp.WebApps (WebApps.java:build(288)) - Registered webapp guice modules 2015-11-22 19:23:08,320 INFO [Thread-199] http.HttpServer2 (HttpServer2.java:openListeners(917)) - Jetty bound to port 46999 2015-11-22 19:23:08,320 INFO [Thread-199] mortbay.log (Slf4jLog.java:info(67)) - jetty-6.1.26 2015-11-22 19:23:08,353 INFO [Thread-199] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:23:08,361 INFO [Thread-199] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:23:08,361 INFO [Thread[Thread-240,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2015-11-22 19:23:08,362 INFO [Thread[Thread-240,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:23:08,365 INFO [Thread[Thread-242,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2015-11-22 19:23:08,365 INFO [Thread[Thread-242,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens Nov 22, 2015 7:23:08 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.nodemanager.webapp.NMWebServices as a root resource class Nov 22, 2015 7:23:08 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a provider class Nov 22, 2015 7:23:08 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.nodemanager.webapp.JAXBContextResolver as a provider class Nov 22, 2015 7:23:08 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate INFO: Initiating Jersey application, version 'Jersey: 1.9 09/02/2011 11:17 AM' Nov 22, 2015 7:23:08 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.nodemanager.webapp.JAXBContextResolver to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:23:08 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.webapp.GenericExceptionHandler to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:23:08 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices to GuiceManagedComponentProvider with the scope "Singleton" 2015-11-22 19:23:08,665 INFO [RM-0] mortbay.log (Slf4jLog.java:info(67)) - Started HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:59535 2015-11-22 19:23:08,666 INFO [RM-0] webapp.WebApps (WebApps.java:start(307)) - Web app cluster started at 59535 2015-11-22 19:23:08,694 INFO [RM-0] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:23:08,695 INFO [Socket Reader #1 for port 47823] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 47823 2015-11-22 19:23:08,697 INFO [RM-0] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.server.api.ResourceManagerAdministrationProtocolPB to the server 2015-11-22 19:23:08,698 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:23:08,705 INFO [IPC Server listener on 47823] ipc.Server (Server.java:run(794)) - IPC Server listener on 47823: starting Nov 22, 2015 7:23:08 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.nodemanager.webapp.NMWebServices to GuiceManagedComponentProvider with the scope "Singleton" 2015-11-22 19:23:08,730 INFO [Thread-199] mortbay.log (Slf4jLog.java:info(67)) - Started HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:46999 2015-11-22 19:23:08,731 INFO [Thread-199] webapp.WebApps (WebApps.java:start(307)) - Web app node started at 46999 2015-11-22 19:23:08,731 INFO [Thread-199] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:serviceStart(222)) - Node ID assigned is : ip-172-31-20-42.ap-northeast-1.compute.internal:52784 2015-11-22 19:23:08,734 INFO [Thread-199] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNMContainerStatuses(550)) - Sending out 0 NM container statuses: [] 2015-11-22 19:23:08,741 INFO [Thread-199] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:registerWithRM(330)) - Registering with RM using containers :[] 2015-11-22 19:23:08,765 INFO [Thread-199] resourcemanager.ResourceTrackerService (ResourceTrackerService.java:registerNodeManager(375)) - NodeManager from node ip-172-31-20-42.ap-northeast-1.compute.internal(cmPort: 52784 httpPort: 46999) registered with capability: , assigned nodeId ip-172-31-20-42.ap-northeast-1.compute.internal:52784 2015-11-22 19:23:08,766 INFO [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(566)) - ip-172-31-20-42.ap-northeast-1.compute.internal:52784 Node Transitioned from NEW to RUNNING 2015-11-22 19:23:08,768 INFO [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:addNode(1427)) - Added node ip-172-31-20-42.ap-northeast-1.compute.internal:52784 clusterResource: 2015-11-22 19:23:08,769 INFO [Thread-199] security.NMContainerTokenSecretManager (NMContainerTokenSecretManager.java:setMasterKey(138)) - Rolling master-key for container-tokens, got key with id 408356595 2015-11-22 19:23:08,771 INFO [Thread-199] security.NMTokenSecretManagerInNM (NMTokenSecretManagerInNM.java:setMasterKey(135)) - Rolling master-key for container-tokens, got key with id -1682079204 2015-11-22 19:23:08,771 INFO [Thread-199] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:registerWithRM(391)) - Registered with ResourceManager as ip-172-31-20-42.ap-northeast-1.compute.internal:52784 with total resource of 2015-11-22 19:23:08,771 INFO [Thread-199] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:registerWithRM(392)) - Notifying ContainerManager to unblock new container-requests 2015-11-22 19:23:09,125 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(728)) - Waiting for Timeline Server to start... 2015-11-22 19:23:09,126 INFO [Thread-250] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:23:09,128 INFO [Thread[Thread-252,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2015-11-22 19:23:09,128 INFO [Thread[Thread-252,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:23:09,130 INFO [Thread-250] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:23:09,131 INFO [Socket Reader #1 for port 57607] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 57607 2015-11-22 19:23:09,134 INFO [Thread-250] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.api.ApplicationHistoryProtocolPB to the server 2015-11-22 19:23:09,135 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:23:09,135 INFO [IPC Server listener on 57607] ipc.Server (Server.java:run(794)) - IPC Server listener on 57607: starting 2015-11-22 19:23:09,138 INFO [Thread-250] applicationhistoryservice.ApplicationHistoryClientService (ApplicationHistoryClientService.java:serviceStart(111)) - Instantiated ApplicationHistoryClientService at ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:57607 2015-11-22 19:23:09,138 INFO [Thread-250] applicationhistoryservice.ApplicationHistoryManagerImpl (ApplicationHistoryManagerImpl.java:serviceStart(71)) - Starting ApplicationHistory 2015-11-22 19:23:09,142 INFO [Thread-250] server.AuthenticationFilter (AuthenticationFilter.java:constructSecretProvider(304)) - Unable to initialize FileSignerSecretProvider, falling back to use random secrets. 2015-11-22 19:23:09,145 INFO [Thread-250] http.HttpRequestLog (HttpRequestLog.java:getRequestLog(80)) - Http request log for http.requests.applicationhistory is not defined 2015-11-22 19:23:09,146 INFO [Thread-250] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter) 2015-11-22 19:23:09,147 INFO [Thread-250] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context applicationhistory 2015-11-22 19:23:09,147 INFO [Thread-250] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context logs 2015-11-22 19:23:09,147 INFO [Thread-250] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context static 2015-11-22 19:23:09,148 INFO [Thread-250] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context applicationhistory 2015-11-22 19:23:09,148 INFO [Thread-250] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2015-11-22 19:23:09,148 INFO [Thread-250] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2015-11-22 19:23:09,149 INFO [Thread-250] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'Timeline Authentication Filter' (class=org.apache.hadoop.yarn.server.timeline.security.TimelineAuthenticationFilter) 2015-11-22 19:23:09,150 INFO [Thread-250] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /applicationhistory/* 2015-11-22 19:23:09,150 INFO [Thread-250] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /ws/* Nov 22, 2015 7:23:09 PM com.google.inject.servlet.GuiceFilter setPipeline WARNING: Multiple Servlet injectors detected. This is a warning indicating that you have more than one GuiceFilter running in your web application. If this is deliberate, you may safely ignore this message. If this is NOT deliberate however, your application may not work as expected. 2015-11-22 19:23:09,190 INFO [Thread-250] webapp.WebApps (WebApps.java:build(288)) - Registered webapp guice modules 2015-11-22 19:23:09,191 INFO [Thread-250] http.HttpServer2 (HttpServer2.java:openListeners(917)) - Jetty bound to port 51636 2015-11-22 19:23:09,191 INFO [Thread-250] mortbay.log (Slf4jLog.java:info(67)) - jetty-6.1.26 2015-11-22 19:23:09,221 INFO [Thread-250] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:23:09,222 INFO [Thread[Thread-270,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2015-11-22 19:23:09,223 INFO [Thread[Thread-270,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:23:09,224 INFO [Thread-250] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:23:09,225 INFO [Thread[Thread-272,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) Nov 22, 2015 7:23:09 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider as a provider class Nov 22, 2015 7:23:09 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.AHSWebServices as a root resource class Nov 22, 2015 7:23:09 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.timeline.webapp.TimelineWebServices as a root resource class Nov 22, 2015 7:23:09 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a provider class Nov 22, 2015 7:23:09 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate INFO: Initiating Jersey application, version 'Jersey: 1.9 09/02/2011 11:17 AM' Nov 22, 2015 7:23:09 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.webapp.GenericExceptionHandler to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:23:09 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:23:09 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.AHSWebServices to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:23:09 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.timeline.webapp.TimelineWebServices to GuiceManagedComponentProvider with the scope "Singleton" 2015-11-22 19:23:09,508 INFO [Thread-250] mortbay.log (Slf4jLog.java:info(67)) - Started HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:51636 2015-11-22 19:23:09,509 INFO [Thread-250] applicationhistoryservice.ApplicationHistoryServer (ApplicationHistoryServer.java:startWebApp(342)) - Instantiating AHSWebApp at 51636 2015-11-22 19:23:10,626 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(739)) - MiniYARN ApplicationHistoryServer address: ip-172-31-20-42.ap-northeast-1.compute.internal:57607 2015-11-22 19:23:10,626 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(741)) - MiniYARN ApplicationHistoryServer web address: ip-172-31-20-42.ap-northeast-1.compute.internal:51636 2015-11-22 19:23:13,057 INFO [Thread-274] impl.TimelineClientImpl (TimelineClientImpl.java:serviceInit(296)) - Timeline service address: http://ip-172-31-20-42.ap-northeast-1.compute.internal:51636/ws/v1/timeline/ 2015-11-22 19:23:13,298 INFO [Thread-274] distributedshell.TestDistributedShell (TestDistributedShell.java:testDSShellWithInvalidArgs(630)) - Initializing DS Client with no args 2015-11-22 19:23:13,301 INFO [Thread-274] distributedshell.TestDistributedShell (TestDistributedShell.java:testDSShellWithInvalidArgs(639)) - Initializing DS Client with no jar file 2015-11-22 19:23:13,302 INFO [Thread-274] distributedshell.TestDistributedShell (TestDistributedShell.java:testDSShellWithInvalidArgs(658)) - Initializing DS Client with no shell command 2015-11-22 19:23:13,302 INFO [Thread-274] distributedshell.TestDistributedShell (TestDistributedShell.java:testDSShellWithInvalidArgs(677)) - Initializing DS Client with invalid no. of containers 2015-11-22 19:23:13,303 INFO [Thread-274] distributedshell.TestDistributedShell (TestDistributedShell.java:testDSShellWithInvalidArgs(698)) - Initializing DS Client with invalid no. of vcores 2015-11-22 19:23:13,303 INFO [Thread-274] distributedshell.TestDistributedShell (TestDistributedShell.java:testDSShellWithInvalidArgs(723)) - Initializing DS Client with --shell_command and --shell_script 2015-11-22 19:23:13,303 INFO [Thread-274] distributedshell.TestDistributedShell (TestDistributedShell.java:testDSShellWithInvalidArgs(751)) - Initializing DS Client without --shell_command and --shell_script 2015-11-22 19:23:13,327 ERROR [Thread[Thread-270,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(670)) - ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted 2015-11-22 19:23:13,332 INFO [main] mortbay.log (Slf4jLog.java:info(67)) - Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:0 2015-11-22 19:23:13,435 INFO [main] applicationhistoryservice.ApplicationHistoryManagerImpl (ApplicationHistoryManagerImpl.java:serviceStop(78)) - Stopping ApplicationHistory 2015-11-22 19:23:13,436 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 57607 2015-11-22 19:23:13,439 INFO [IPC Server listener on 57607] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 57607 2015-11-22 19:23:13,439 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:23:13,440 ERROR [Thread[Thread-252,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(670)) - ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted 2015-11-22 19:23:13,446 INFO [main] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:unRegisterNM(268)) - Successfully Unregistered the Node ip-172-31-20-42.ap-northeast-1.compute.internal:52784 with ResourceManager. 2015-11-22 19:23:13,449 ERROR [Thread[Thread-242,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(670)) - ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted 2015-11-22 19:23:13,449 ERROR [Thread[Thread-240,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(670)) - ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted 2015-11-22 19:23:13,451 INFO [main] mortbay.log (Slf4jLog.java:info(67)) - Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:0 2015-11-22 19:23:13,552 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 52784 2015-11-22 19:23:13,553 INFO [IPC Server listener on 52784] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 52784 2015-11-22 19:23:13,554 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:23:13,554 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 44577 2015-11-22 19:23:13,555 INFO [IPC Server listener on 44577] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 44577 2015-11-22 19:23:13,555 INFO [Public Localizer] localizer.ResourceLocalizationService (ResourceLocalizationService.java:run(908)) - Public cache exiting 2015-11-22 19:23:13,555 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:23:13,555 WARN [Node Resource Monitor] nodemanager.NodeResourceMonitorImpl (NodeResourceMonitorImpl.java:run(154)) - org.apache.hadoop.yarn.server.nodemanager.NodeResourceMonitorImpl is interrupted. Exiting. 2015-11-22 19:23:13,561 ERROR [Thread[Thread-197,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(670)) - ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted 2015-11-22 19:23:13,561 ERROR [Thread[Thread-195,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(670)) - ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted 2015-11-22 19:23:13,565 INFO [main] mortbay.log (Slf4jLog.java:info(67)) - Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:0 2015-11-22 19:23:13,666 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 35548 2015-11-22 19:23:13,667 INFO [IPC Server listener on 35548] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 35548 2015-11-22 19:23:13,667 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:23:13,668 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 47823 2015-11-22 19:23:13,668 INFO [IPC Server listener on 47823] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 47823 2015-11-22 19:23:13,669 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:23:13,671 INFO [main] resourcemanager.ResourceManager (ResourceManager.java:transitionToStandby(1051)) - Transitioning to standby state 2015-11-22 19:23:13,671 WARN [ApplicationMaster Launcher] amlauncher.ApplicationMasterLauncher (ApplicationMasterLauncher.java:run(122)) - org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher$LauncherThread interrupted. Returning. 2015-11-22 19:23:13,672 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 51338 2015-11-22 19:23:13,673 INFO [IPC Server listener on 51338] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 51338 2015-11-22 19:23:13,673 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 43240 2015-11-22 19:23:13,673 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:23:13,674 INFO [IPC Server listener on 43240] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 43240 2015-11-22 19:23:13,674 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:23:13,674 ERROR [ResourceManager Event Processor] resourcemanager.ResourceManager (ResourceManager.java:run(691)) - Returning, interrupted : java.lang.InterruptedException 2015-11-22 19:23:13,674 INFO [Ping Checker] util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(135)) - NMLivelinessMonitor thread interrupted 2015-11-22 19:23:13,675 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:serviceStop(141)) - AsyncDispatcher is draining to stop, igonring any new events. 2015-11-22 19:23:13,675 INFO [Ping Checker] util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(135)) - AMLivelinessMonitor thread interrupted 2015-11-22 19:23:13,676 INFO [Ping Checker] util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(135)) - AMLivelinessMonitor thread interrupted 2015-11-22 19:23:13,676 INFO [Ping Checker] util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(135)) - org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.ContainerAllocationExpirer thread interrupted 2015-11-22 19:23:13,676 ERROR [Thread[Thread-24,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(670)) - ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted 2015-11-22 19:23:13,680 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stop(211)) - Stopping ApplicationHistoryServer metrics system... 2015-11-22 19:23:13,683 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stop(217)) - ApplicationHistoryServer metrics system stopped. 2015-11-22 19:23:13,683 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:shutdown(608)) - ApplicationHistoryServer metrics system shutdown complete. 2015-11-22 19:23:13,684 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:serviceStop(141)) - AsyncDispatcher is draining to stop, igonring any new events. 2015-11-22 19:23:13,684 INFO [main] resourcemanager.ResourceManager (ResourceManager.java:transitionToStandby(1058)) - Transitioned to standby state 2015-11-22 19:23:13,706 INFO [main] distributedshell.TestDistributedShell (TestDistributedShell.java:setupInternal(77)) - Starting up YARN cluster 2015-11-22 19:23:13,724 INFO [main] net.ServerSocketUtil (ServerSocketUtil.java:getPort(52)) - Using port 10021 2015-11-22 19:23:13,742 INFO [main] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2363)) - found resource core-site.xml at file:/home/ubuntu/hadoop-dev/hadoop-common-project/hadoop-common/target/test-classes/core-site.xml 2015-11-22 19:23:13,756 WARN [main] conf.Configuration (Configuration.java:loadProperty(2687)) - core-site.xml:an attempt to override final parameter: hadoop.tmp.dir; Ignoring. 2015-11-22 19:23:13,757 INFO [main] security.Groups (Groups.java:refresh(256)) - clearing userToGroupsMap cache 2015-11-22 19:23:13,758 INFO [main] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2363)) - found resource yarn-site.xml at file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/test-classes/yarn-site.xml 2015-11-22 19:23:13,784 WARN [main] conf.Configuration (Configuration.java:loadProperty(2687)) - yarn-site.xml:an attempt to override final parameter: hadoop.tmp.dir; Ignoring. 2015-11-22 19:23:13,785 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.RMFatalEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMFatalEventDispatcher 2015-11-22 19:23:13,786 INFO [main] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:(75)) - NMTokenKeyRollingInterval: 86400000ms and NMTokenKeyActivationDelay: 900000ms 2015-11-22 19:23:13,787 INFO [main] security.RMContainerTokenSecretManager (RMContainerTokenSecretManager.java:(78)) - ContainerTokenKeyRollingInterval: 86400000ms and ContainerTokenKeyActivationDelay: 900000ms 2015-11-22 19:23:13,792 INFO [main] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:(94)) - AMRMTokenKeyRollingInterval: 86400000ms and AMRMTokenKeyActivationDelay: 900000 ms 2015-11-22 19:23:13,792 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStoreEventType for class org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$ForwardingEventHandler 2015-11-22 19:23:13,792 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.NodesListManagerEventType for class org.apache.hadoop.yarn.server.resourcemanager.NodesListManager 2015-11-22 19:23:13,793 INFO [main] resourcemanager.ResourceManager (ResourceManager.java:createScheduler(295)) - Using Scheduler: org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler 2015-11-22 19:23:13,793 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.SchedulerEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher 2015-11-22 19:23:13,793 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationEventDispatcher 2015-11-22 19:23:13,794 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationAttemptEventDispatcher 2015-11-22 19:23:13,794 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$NodeEventDispatcher 2015-11-22 19:23:13,796 INFO [main] impl.MetricsConfig (MetricsConfig.java:loadFirst(115)) - loaded properties from hadoop-metrics2.properties 2015-11-22 19:23:13,800 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:startTimer(374)) - Scheduled snapshot period at 10 second(s). 2015-11-22 19:23:13,800 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:start(192)) - ResourceManager metrics system started 2015-11-22 19:23:13,802 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.RMAppManagerEventType for class org.apache.hadoop.yarn.server.resourcemanager.RMAppManager 2015-11-22 19:23:13,802 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncherEventType for class org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher 2015-11-22 19:23:13,803 WARN [main] util.MBeans (MBeans.java:register(73)) - Failed to register MBean "Hadoop:service=ResourceManager,name=RMNMInfo": Instance already exists. 2015-11-22 19:23:13,803 INFO [main] resourcemanager.RMNMInfo (RMNMInfo.java:(63)) - Registered RMNMInfo MBean 2015-11-22 19:23:13,803 INFO [main] util.HostsFileReader (HostsFileReader.java:refresh(132)) - Refreshing hosts (include/exclude) list 2015-11-22 19:23:13,805 INFO [main] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2363)) - found resource capacity-scheduler.xml at jar:file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/target/hadoop-yarn-server-tests-2.8.0-SNAPSHOT-tests.jar!/capacity-scheduler.xml 2015-11-22 19:23:13,828 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(647)) - max alloc mb per queue for root is undefined 2015-11-22 19:23:13,828 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(651)) - max alloc vcore per queue for root is undefined 2015-11-22 19:23:13,829 INFO [main] capacity.ParentQueue (ParentQueue.java:setupQueueConfigs(125)) - root, capacity=1.0, asboluteCapacity=1.0, maxCapacity=1.0, asboluteMaxCapacity=1.0, state=RUNNING, acls=ADMINISTER_QUEUE:*SUBMIT_APP:*, labels=*, , reservationsContinueLooking=true 2015-11-22 19:23:13,829 INFO [main] capacity.ParentQueue (ParentQueue.java:(104)) - Initialized parent-queue root name=root, fullname=root 2015-11-22 19:23:13,831 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(647)) - max alloc mb per queue for root.default is undefined 2015-11-22 19:23:13,832 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(651)) - max alloc vcore per queue for root.default is undefined 2015-11-22 19:23:13,833 INFO [main] capacity.LeafQueue (LeafQueue.java:setupQueueConfigs(217)) - Initializing default capacity = 1.0 [= (float) configuredCapacity / 100 ] asboluteCapacity = 1.0 [= parentAbsoluteCapacity * capacity ] maxCapacity = 1.0 [= configuredMaxCapacity ] absoluteMaxCapacity = 1.0 [= 1.0 maximumCapacity undefined, (parentAbsoluteMaxCapacity * maximumCapacity) / 100 otherwise ] userLimit = 100 [= configuredUserLimit ] userLimitFactor = 1.0 [= configuredUserLimitFactor ] maxApplications = 10000 [= configuredMaximumSystemApplicationsPerQueue or (int)(configuredMaximumSystemApplications * absoluteCapacity)] maxApplicationsPerUser = 10000 [= (int)(maxApplications * (userLimit / 100.0f) * userLimitFactor) ] usedCapacity = 0.0 [= usedResourcesMemory / (clusterResourceMemory * absoluteCapacity)] absoluteUsedCapacity = 0.0 [= usedResourcesMemory / clusterResourceMemory] maxAMResourcePerQueuePercent = 0.1 [= configuredMaximumAMResourcePercent ] minimumAllocationFactor = 0.984375 [= (float)(maximumAllocationMemory - minimumAllocationMemory) / maximumAllocationMemory ] maximumAllocation = [= configuredMaxAllocation ] numContainers = 0 [= currentNumContainers ] state = RUNNING [= configuredState ] acls = ADMINISTER_QUEUE:*SUBMIT_APP:* [= configuredAcls ] nodeLocalityDelay = -1 labels=*, nodeLocalityDelay = -1 reservationsContinueLooking = true preemptionDisabled = true defaultAppPriorityPerQueue = 0 2015-11-22 19:23:13,834 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:parseQueue(657)) - Initialized queue: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=0, numContainers=0 2015-11-22 19:23:13,834 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:parseQueue(657)) - Initialized queue: root: numChildQueue= 1, capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.0, numApps=0, numContainers=0 2015-11-22 19:23:13,834 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:initializeQueues(501)) - Initialized root queue root: numChildQueue= 1, capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.0, numApps=0, numContainers=0 2015-11-22 19:23:13,835 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:getUserGroupMappingPlacementRule(449)) - Initialized queue mappings, override: false 2015-11-22 19:23:13,835 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:initScheduler(314)) - Initialized CapacityScheduler with calculator=class org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator, minimumAllocation=<>, maximumAllocation=<>, asynchronousScheduling=false, asyncScheduleInterval=5ms 2015-11-22 19:23:13,837 INFO [main] metrics.SystemMetricsPublisher (SystemMetricsPublisher.java:serviceInit(95)) - YARN system metrics publishing service is not enabled 2015-11-22 19:23:13,837 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEventType for class org.apache.hadoop.yarn.server.MiniYARNCluster$1 2015-11-22 19:23:13,838 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:prepareDirs(558)) - Created localDir in /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0 2015-11-22 19:23:13,838 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:prepareDirs(558)) - Created logDir in /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-logDir-nm-0_0 2015-11-22 19:23:13,839 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceInit(540)) - Starting NM: 0 2015-11-22 19:23:13,850 INFO [main] nodemanager.NodeManager (NodeManager.java:getNodeHealthScriptRunner(255)) - Node Manager health check script is not available or doesn't have execute permission, so not starting the node health script runner. 2015-11-22 19:23:13,850 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ContainerEventDispatcher 2015-11-22 19:23:13,851 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ApplicationEventDispatcher 2015-11-22 19:23:13,851 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizationEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService 2015-11-22 19:23:13,851 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServicesEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices 2015-11-22 19:23:13,851 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl 2015-11-22 19:23:13,851 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncherEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncher 2015-11-22 19:23:13,851 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.ContainerManagerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl 2015-11-22 19:23:13,852 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.NodeManagerEventType for class org.apache.hadoop.yarn.server.MiniYARNCluster$ShortCircuitedNodeManager 2015-11-22 19:23:13,852 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:init(159)) - NodeManager metrics system started (again) 2015-11-22 19:23:13,854 INFO [main] nodemanager.NodeResourceMonitorImpl (NodeResourceMonitorImpl.java:serviceInit(76)) - Using ResourceCalculatorPlugin : org.apache.hadoop.yarn.util.ResourceCalculatorPlugin@45752059 2015-11-22 19:23:13,854 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.event.LogHandlerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.NonAggregatingLogHandler 2015-11-22 19:23:13,854 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.sharedcache.SharedCacheUploadEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.sharedcache.SharedCacheUploadService 2015-11-22 19:23:13,854 INFO [main] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:serviceInit(261)) - AMRMProxyService is disabled 2015-11-22 19:23:13,855 INFO [main] localizer.ResourceLocalizationService (ResourceLocalizationService.java:validateConf(224)) - per directory file limit = 8192 2015-11-22 19:23:13,906 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerTracker 2015-11-22 19:23:13,906 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(116)) - Using ResourceCalculatorPlugin : org.apache.hadoop.yarn.util.ResourceCalculatorPlugin@34e9fd99 2015-11-22 19:23:13,906 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(121)) - Using ResourceCalculatorProcessTree : null 2015-11-22 19:23:13,907 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(158)) - Physical memory check enabled: false 2015-11-22 19:23:13,907 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(159)) - Virtual memory check enabled: false 2015-11-22 19:23:13,907 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:isEnabled(206)) - Neither virutal-memory nor physical-memory monitoring is needed. Not running the monitor-thread 2015-11-22 19:23:13,907 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(162)) - ContainersMonitor enabled: false 2015-11-22 19:23:13,908 INFO [main] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:serviceInit(175)) - Nodemanager resources: memory set to 4096MB. 2015-11-22 19:23:13,908 INFO [main] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:serviceInit(176)) - Nodemanager resources: vcores set to 8. 2015-11-22 19:23:13,908 INFO [main] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:serviceInit(208)) - Initialized nodemanager with : physical-memory=4096 virtual-memory=8602 virtual-cores=8 2015-11-22 19:23:13,909 WARN [main] applicationhistoryservice.ApplicationHistoryServer (ApplicationHistoryServer.java:createApplicationHistoryManager(218)) - The filesystem based application history store is deprecated. 2015-11-22 19:23:13,909 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:init(159)) - ApplicationHistoryServer metrics system started (again) 2015-11-22 19:23:13,911 INFO [main] applicationhistoryservice.ApplicationHistoryManagerImpl (ApplicationHistoryManagerImpl.java:serviceInit(61)) - ApplicationHistory Init 2015-11-22 19:23:13,911 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:startResourceManager(326)) - Waiting for RM to start... 2015-11-22 19:23:13,912 INFO [RM-0] resourcemanager.ResourceManager (ResourceManager.java:transitionToActive(1024)) - Transitioning to active state 2015-11-22 19:23:13,917 INFO [RM-0] recovery.RMStateStore (RMStateStore.java:transition(482)) - Updating AMRMToken 2015-11-22 19:23:13,917 INFO [RM-0] security.RMContainerTokenSecretManager (RMContainerTokenSecretManager.java:rollMasterKey(106)) - Rolling master-key for container-tokens 2015-11-22 19:23:13,917 INFO [RM-0] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:rollMasterKey(95)) - Rolling master-key for nm-tokens 2015-11-22 19:23:13,918 INFO [RM-0] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:23:13,918 INFO [RM-0] security.RMDelegationTokenSecretManager (RMDelegationTokenSecretManager.java:storeNewMasterKey(87)) - storing master key with keyID 1 2015-11-22 19:23:13,918 INFO [RM-0] recovery.RMStateStore (RMStateStore.java:transition(433)) - Storing RMDTMasterKey. 2015-11-22 19:23:13,920 INFO [Thread[Thread-293,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2015-11-22 19:23:13,927 INFO [Thread[Thread-293,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:23:13,927 INFO [Thread[Thread-293,5,main]] security.RMDelegationTokenSecretManager (RMDelegationTokenSecretManager.java:storeNewMasterKey(87)) - storing master key with keyID 2 2015-11-22 19:23:13,931 INFO [Thread[Thread-293,5,main]] recovery.RMStateStore (RMStateStore.java:transition(433)) - Storing RMDTMasterKey. 2015-11-22 19:23:13,934 INFO [RM-0] nodelabels.CommonNodeLabelsManager (CommonNodeLabelsManager.java:addToCluserNodeLabels(309)) - Add labels: [] 2015-11-22 19:23:13,934 INFO [RM-0] nodelabels.CommonNodeLabelsManager (CommonNodeLabelsManager.java:internalUpdateLabelsOnNodes(638)) - REPLACE labels on nodes: 2015-11-22 19:23:13,934 INFO [RM-0] nodelabels.CommonNodeLabelsManager (CommonNodeLabelsManager.java:internalUpdateLabelsOnNodes(640)) - NM=ip-172-31-20-42.ap-northeast-1.compute.internal:58694, labels=[x] 2015-11-22 19:23:13,936 INFO [RM-0] nodelabels.FileSystemNodeLabelsStore (FileSystemNodeLabelsStore.java:recover(279)) - Finished write mirror at:file:/tmp/hadoop-yarn-ubuntu/node-labels/nodelabel.mirror 2015-11-22 19:23:13,936 INFO [RM-0] nodelabels.FileSystemNodeLabelsStore (FileSystemNodeLabelsStore.java:recover(280)) - Finished create editlog file at:file:/tmp/hadoop-yarn-ubuntu/node-labels/nodelabel.editlog 2015-11-22 19:23:13,936 INFO [RM-0] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.nodelabels.event.NodeLabelsStoreEventType for class org.apache.hadoop.yarn.nodelabels.CommonNodeLabelsManager$ForwardingEventHandler 2015-11-22 19:23:13,948 INFO [RM-0] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:23:13,949 INFO [Socket Reader #1 for port 42633] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 42633 2015-11-22 19:23:13,952 INFO [RM-0] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.server.api.ResourceTrackerPB to the server 2015-11-22 19:23:13,952 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:23:13,956 INFO [IPC Server listener on 42633] ipc.Server (Server.java:run(794)) - IPC Server listener on 42633: starting 2015-11-22 19:23:13,993 INFO [RM-0] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:23:13,994 INFO [Socket Reader #1 for port 48969] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 48969 2015-11-22 19:23:13,998 INFO [RM-0] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB to the server 2015-11-22 19:23:13,998 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:23:13,998 INFO [IPC Server listener on 48969] ipc.Server (Server.java:run(794)) - IPC Server listener on 48969: starting 2015-11-22 19:23:14,022 INFO [RM-0] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:23:14,023 INFO [Socket Reader #1 for port 59307] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 59307 2015-11-22 19:23:14,025 INFO [RM-0] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.api.ApplicationClientProtocolPB to the server 2015-11-22 19:23:14,025 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:23:14,025 INFO [IPC Server listener on 59307] ipc.Server (Server.java:run(794)) - IPC Server listener on 59307: starting 2015-11-22 19:23:14,035 INFO [RM-0] resourcemanager.ResourceManager (ResourceManager.java:transitionToActive(1040)) - Transitioned to active state 2015-11-22 19:23:14,037 INFO [RM-0] server.AuthenticationFilter (AuthenticationFilter.java:constructSecretProvider(304)) - Unable to initialize FileSignerSecretProvider, falling back to use random secrets. 2015-11-22 19:23:14,038 INFO [RM-0] http.HttpRequestLog (HttpRequestLog.java:getRequestLog(80)) - Http request log for http.requests.resourcemanager is not defined 2015-11-22 19:23:14,039 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter) 2015-11-22 19:23:14,040 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context cluster 2015-11-22 19:23:14,040 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context logs 2015-11-22 19:23:14,040 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context static 2015-11-22 19:23:14,040 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context cluster 2015-11-22 19:23:14,041 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2015-11-22 19:23:14,041 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2015-11-22 19:23:14,042 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'Timeline Authentication Filter' (class=org.apache.hadoop.yarn.server.timeline.security.TimelineAuthenticationFilter) 2015-11-22 19:23:14,043 INFO [RM-0] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /cluster/* 2015-11-22 19:23:14,043 INFO [RM-0] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /ws/* 2015-11-22 19:23:14,051 INFO [RM-0] webapp.WebApps (WebApps.java:build(288)) - Registered webapp guice modules 2015-11-22 19:23:14,052 INFO [RM-0] http.HttpServer2 (HttpServer2.java:openListeners(917)) - Jetty bound to port 41686 2015-11-22 19:23:14,052 INFO [RM-0] mortbay.log (Slf4jLog.java:info(67)) - jetty-6.1.26 2015-11-22 19:23:14,067 INFO [RM-0] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:23:14,067 INFO [Thread[Thread-462,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2015-11-22 19:23:14,068 INFO [Thread[Thread-462,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:23:14,069 INFO [RM-0] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:23:14,069 INFO [Thread[Thread-464,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) Nov 22, 2015 7:23:14 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.resourcemanager.webapp.JAXBContextResolver as a provider class Nov 22, 2015 7:23:14 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices as a root resource class Nov 22, 2015 7:23:14 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a provider class Nov 22, 2015 7:23:14 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate INFO: Initiating Jersey application, version 'Jersey: 1.9 09/02/2011 11:17 AM' Nov 22, 2015 7:23:14 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.resourcemanager.webapp.JAXBContextResolver to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:23:14 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.webapp.GenericExceptionHandler to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:23:14 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices to GuiceManagedComponentProvider with the scope "Singleton" 2015-11-22 19:23:14,324 INFO [RM-0] mortbay.log (Slf4jLog.java:info(67)) - Started HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:41686 2015-11-22 19:23:14,324 INFO [RM-0] webapp.WebApps (WebApps.java:start(307)) - Web app cluster started at 41686 2015-11-22 19:23:14,329 INFO [RM-0] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:23:14,329 INFO [Socket Reader #1 for port 60680] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 60680 2015-11-22 19:23:14,331 INFO [RM-0] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.server.api.ResourceManagerAdministrationProtocolPB to the server 2015-11-22 19:23:14,331 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:23:14,336 INFO [IPC Server listener on 60680] ipc.Server (Server.java:run(794)) - IPC Server listener on 60680: starting 2015-11-22 19:23:15,412 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:startResourceManager(339)) - MiniYARN ResourceManager address: ip-172-31-20-42.ap-northeast-1.compute.internal:59307 2015-11-22 19:23:15,412 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:startResourceManager(341)) - MiniYARN ResourceManager web address: ip-172-31-20-42.ap-northeast-1.compute.internal:41686 2015-11-22 19:23:15,412 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(459)) - MiniYARN ResourceManager address: ip-172-31-20-42.ap-northeast-1.compute.internal:59307 2015-11-22 19:23:15,412 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(461)) - MiniYARN ResourceManager web address: ip-172-31-20-42.ap-northeast-1.compute.internal:41686 2015-11-22 19:23:15,413 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(575)) - Waiting for NM 0 to start... 2015-11-22 19:23:15,415 INFO [Thread-471] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:23:15,418 INFO [Socket Reader #1 for port 45634] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 45634 2015-11-22 19:23:15,420 INFO [Thread-471] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.api.ContainerManagementProtocolPB to the server 2015-11-22 19:23:15,420 INFO [Thread-471] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:serviceStart(464)) - Blocking new container-requests as container manager rpc server is still starting. 2015-11-22 19:23:15,420 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:23:15,421 INFO [IPC Server listener on 45634] ipc.Server (Server.java:run(794)) - IPC Server listener on 45634: starting 2015-11-22 19:23:15,423 INFO [Thread-471] security.NMContainerTokenSecretManager (NMContainerTokenSecretManager.java:setNodeId(260)) - Updating node address : ip-172-31-20-42.ap-northeast-1.compute.internal:45634 2015-11-22 19:23:15,424 INFO [Thread-471] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:23:15,425 INFO [Socket Reader #1 for port 42804] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 42804 2015-11-22 19:23:15,426 INFO [Thread-471] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB to the server 2015-11-22 19:23:15,426 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:23:15,426 INFO [IPC Server listener on 42804] ipc.Server (Server.java:run(794)) - IPC Server listener on 42804: starting 2015-11-22 19:23:15,429 INFO [Thread-471] localizer.ResourceLocalizationService (ResourceLocalizationService.java:serviceStart(363)) - Localizer started on port 42804 2015-11-22 19:23:15,445 INFO [Thread-471] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:serviceStart(509)) - ContainerManager started at /172.31.20.42:45634 2015-11-22 19:23:15,445 INFO [Thread-471] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:serviceStart(510)) - ContainerManager bound to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:0 2015-11-22 19:23:15,445 INFO [Thread-471] webapp.WebServer (WebServer.java:serviceStart(71)) - Instantiating NMWebApp at ip-172-31-20-42.ap-northeast-1.compute.internal:0 2015-11-22 19:23:15,446 INFO [Thread-471] server.AuthenticationFilter (AuthenticationFilter.java:constructSecretProvider(304)) - Unable to initialize FileSignerSecretProvider, falling back to use random secrets. 2015-11-22 19:23:15,447 INFO [Thread-471] http.HttpRequestLog (HttpRequestLog.java:getRequestLog(80)) - Http request log for http.requests.nodemanager is not defined 2015-11-22 19:23:15,447 INFO [Thread-471] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter) 2015-11-22 19:23:15,448 INFO [Thread-471] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context node 2015-11-22 19:23:15,448 INFO [Thread-471] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context static 2015-11-22 19:23:15,448 INFO [Thread-471] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context logs 2015-11-22 19:23:15,449 INFO [Thread-471] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context node 2015-11-22 19:23:15,449 INFO [Thread-471] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2015-11-22 19:23:15,449 INFO [Thread-471] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2015-11-22 19:23:15,450 INFO [Thread-471] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'Timeline Authentication Filter' (class=org.apache.hadoop.yarn.server.timeline.security.TimelineAuthenticationFilter) 2015-11-22 19:23:15,450 INFO [Thread-471] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /node/* 2015-11-22 19:23:15,450 INFO [Thread-471] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /ws/* Nov 22, 2015 7:23:15 PM com.google.inject.servlet.GuiceFilter setPipeline WARNING: Multiple Servlet injectors detected. This is a warning indicating that you have more than one GuiceFilter running in your web application. If this is deliberate, you may safely ignore this message. If this is NOT deliberate however, your application may not work as expected. 2015-11-22 19:23:15,456 INFO [Thread-471] webapp.WebApps (WebApps.java:build(288)) - Registered webapp guice modules 2015-11-22 19:23:15,457 INFO [Thread-471] http.HttpServer2 (HttpServer2.java:openListeners(917)) - Jetty bound to port 41860 2015-11-22 19:23:15,457 INFO [Thread-471] mortbay.log (Slf4jLog.java:info(67)) - jetty-6.1.26 2015-11-22 19:23:15,478 INFO [Thread-471] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:23:15,479 INFO [Thread[Thread-512,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2015-11-22 19:23:15,479 INFO [Thread[Thread-512,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:23:15,480 INFO [Thread-471] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:23:15,480 INFO [Thread[Thread-514,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) Nov 22, 2015 7:23:15 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.nodemanager.webapp.NMWebServices as a root resource class Nov 22, 2015 7:23:15 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a provider class Nov 22, 2015 7:23:15 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.nodemanager.webapp.JAXBContextResolver as a provider class Nov 22, 2015 7:23:15 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate INFO: Initiating Jersey application, version 'Jersey: 1.9 09/02/2011 11:17 AM' Nov 22, 2015 7:23:15 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.nodemanager.webapp.JAXBContextResolver to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:23:15 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.webapp.GenericExceptionHandler to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:23:15 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.nodemanager.webapp.NMWebServices to GuiceManagedComponentProvider with the scope "Singleton" 2015-11-22 19:23:15,672 INFO [Thread-471] mortbay.log (Slf4jLog.java:info(67)) - Started HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:41860 2015-11-22 19:23:15,673 INFO [Thread-471] webapp.WebApps (WebApps.java:start(307)) - Web app node started at 41860 2015-11-22 19:23:15,674 INFO [Thread-471] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:serviceStart(222)) - Node ID assigned is : ip-172-31-20-42.ap-northeast-1.compute.internal:45634 2015-11-22 19:23:15,674 INFO [Thread-471] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNMContainerStatuses(550)) - Sending out 0 NM container statuses: [] 2015-11-22 19:23:15,675 INFO [Thread-471] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:registerWithRM(330)) - Registering with RM using containers :[] 2015-11-22 19:23:15,675 INFO [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(566)) - ip-172-31-20-42.ap-northeast-1.compute.internal:45634 Node Transitioned from NEW to RUNNING 2015-11-22 19:23:15,677 INFO [Thread-471] resourcemanager.ResourceTrackerService (ResourceTrackerService.java:registerNodeManager(375)) - NodeManager from node ip-172-31-20-42.ap-northeast-1.compute.internal(cmPort: 45634 httpPort: 41860) registered with capability: , assigned nodeId ip-172-31-20-42.ap-northeast-1.compute.internal:45634 2015-11-22 19:23:15,678 INFO [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:addNode(1427)) - Added node ip-172-31-20-42.ap-northeast-1.compute.internal:45634 clusterResource: 2015-11-22 19:23:15,679 INFO [Thread-471] security.NMContainerTokenSecretManager (NMContainerTokenSecretManager.java:setMasterKey(138)) - Rolling master-key for container-tokens, got key with id -1924226884 2015-11-22 19:23:15,679 INFO [Thread-471] security.NMTokenSecretManagerInNM (NMTokenSecretManagerInNM.java:setMasterKey(135)) - Rolling master-key for container-tokens, got key with id -463412845 2015-11-22 19:23:15,679 INFO [Thread-471] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:registerWithRM(391)) - Registered with ResourceManager as ip-172-31-20-42.ap-northeast-1.compute.internal:45634 with total resource of 2015-11-22 19:23:15,679 INFO [Thread-471] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:registerWithRM(392)) - Notifying ContainerManager to unblock new container-requests 2015-11-22 19:23:16,413 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(728)) - Waiting for Timeline Server to start... 2015-11-22 19:23:16,414 INFO [Thread-517] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:23:16,414 INFO [Thread[Thread-519,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2015-11-22 19:23:16,415 INFO [Thread[Thread-519,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:23:16,415 INFO [Thread-517] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:23:16,416 INFO [Socket Reader #1 for port 59513] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 59513 2015-11-22 19:23:16,418 INFO [Thread-517] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.api.ApplicationHistoryProtocolPB to the server 2015-11-22 19:23:16,418 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:23:16,418 INFO [IPC Server listener on 59513] ipc.Server (Server.java:run(794)) - IPC Server listener on 59513: starting 2015-11-22 19:23:16,425 INFO [Thread-517] applicationhistoryservice.ApplicationHistoryClientService (ApplicationHistoryClientService.java:serviceStart(111)) - Instantiated ApplicationHistoryClientService at ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:59513 2015-11-22 19:23:16,425 INFO [Thread-517] applicationhistoryservice.ApplicationHistoryManagerImpl (ApplicationHistoryManagerImpl.java:serviceStart(71)) - Starting ApplicationHistory 2015-11-22 19:23:16,426 INFO [Thread-517] server.AuthenticationFilter (AuthenticationFilter.java:constructSecretProvider(304)) - Unable to initialize FileSignerSecretProvider, falling back to use random secrets. 2015-11-22 19:23:16,432 INFO [Thread-517] http.HttpRequestLog (HttpRequestLog.java:getRequestLog(80)) - Http request log for http.requests.applicationhistory is not defined 2015-11-22 19:23:16,433 INFO [Thread-517] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter) 2015-11-22 19:23:16,434 INFO [Thread-517] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context applicationhistory 2015-11-22 19:23:16,434 INFO [Thread-517] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context static 2015-11-22 19:23:16,434 INFO [Thread-517] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context logs 2015-11-22 19:23:16,435 INFO [Thread-517] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context applicationhistory 2015-11-22 19:23:16,435 INFO [Thread-517] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2015-11-22 19:23:16,435 INFO [Thread-517] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2015-11-22 19:23:16,436 INFO [Thread-517] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'Timeline Authentication Filter' (class=org.apache.hadoop.yarn.server.timeline.security.TimelineAuthenticationFilter) 2015-11-22 19:23:16,437 INFO [Thread-517] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /applicationhistory/* 2015-11-22 19:23:16,437 INFO [Thread-517] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /ws/* Nov 22, 2015 7:23:16 PM com.google.inject.servlet.GuiceFilter setPipeline WARNING: Multiple Servlet injectors detected. This is a warning indicating that you have more than one GuiceFilter running in your web application. If this is deliberate, you may safely ignore this message. If this is NOT deliberate however, your application may not work as expected. 2015-11-22 19:23:16,450 INFO [Thread-517] webapp.WebApps (WebApps.java:build(288)) - Registered webapp guice modules 2015-11-22 19:23:16,451 INFO [Thread-517] http.HttpServer2 (HttpServer2.java:openListeners(917)) - Jetty bound to port 47967 2015-11-22 19:23:16,451 INFO [Thread-517] mortbay.log (Slf4jLog.java:info(67)) - jetty-6.1.26 2015-11-22 19:23:16,469 INFO [Thread-517] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:23:16,469 INFO [Thread[Thread-537,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2015-11-22 19:23:16,470 INFO [Thread[Thread-537,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:23:16,471 INFO [Thread-517] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:23:16,471 INFO [Thread[Thread-539,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) Nov 22, 2015 7:23:16 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider as a provider class Nov 22, 2015 7:23:16 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.AHSWebServices as a root resource class Nov 22, 2015 7:23:16 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.timeline.webapp.TimelineWebServices as a root resource class Nov 22, 2015 7:23:16 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a provider class Nov 22, 2015 7:23:16 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate INFO: Initiating Jersey application, version 'Jersey: 1.9 09/02/2011 11:17 AM' Nov 22, 2015 7:23:16 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.webapp.GenericExceptionHandler to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:23:16 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:23:16 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.AHSWebServices to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:23:16 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.timeline.webapp.TimelineWebServices to GuiceManagedComponentProvider with the scope "Singleton" 2015-11-22 19:23:16,610 INFO [Thread-517] mortbay.log (Slf4jLog.java:info(67)) - Started HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:47967 2015-11-22 19:23:16,611 INFO [Thread-517] applicationhistoryservice.ApplicationHistoryServer (ApplicationHistoryServer.java:startWebApp(342)) - Instantiating AHSWebApp at 47967 2015-11-22 19:23:17,913 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(739)) - MiniYARN ApplicationHistoryServer address: ip-172-31-20-42.ap-northeast-1.compute.internal:59513 2015-11-22 19:23:17,914 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(741)) - MiniYARN ApplicationHistoryServer web address: ip-172-31-20-42.ap-northeast-1.compute.internal:47967 2015-11-22 19:23:19,966 INFO [Thread-541] distributedshell.TestDistributedShell (TestDistributedShell.java:testDSShell(191)) - Initializing DS Client 2015-11-22 19:23:20,033 INFO [Thread-541] impl.TimelineClientImpl (TimelineClientImpl.java:serviceInit(296)) - Timeline service address: http://ip-172-31-20-42.ap-northeast-1.compute.internal:47967/ws/v1/timeline/ 2015-11-22 19:23:20,034 INFO [Thread-541] distributedshell.TestDistributedShell (TestDistributedShell.java:testDSShell(195)) - Running DS Client 2015-11-22 19:23:20,035 INFO [Thread-542] distributedshell.Client (Client.java:run(443)) - Running Client 2015-11-22 19:23:20,060 INFO [Thread-542] client.RMProxy (RMProxy.java:createRMProxy(98)) - Connecting to ResourceManager at ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:59307 2015-11-22 19:23:20,126 INFO [Thread-541] impl.TimelineClientImpl (TimelineClientImpl.java:serviceInit(296)) - Timeline service address: http://ip-172-31-20-42.ap-northeast-1.compute.internal:47967/ws/v1/timeline/ 2015-11-22 19:23:20,144 INFO [Thread-541] client.RMProxy (RMProxy.java:createRMProxy(98)) - Connecting to ResourceManager at ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:59307 2015-11-22 19:23:20,285 INFO [Thread-542] distributedshell.Client (Client.java:run(447)) - Got Cluster metric info from ASM, numNodeManagers=1 2015-11-22 19:23:20,299 INFO [Thread-542] distributedshell.Client (Client.java:run(452)) - Got Cluster node info from ASM 2015-11-22 19:23:20,299 INFO [Thread-542] distributedshell.Client (Client.java:run(454)) - Got node report from ASM for, nodeId=ip-172-31-20-42.ap-northeast-1.compute.internal:45634, nodeAddress=ip-172-31-20-42.ap-northeast-1.compute.internal:41860, nodeRackName=/default-rack, nodeNumContainers=0 2015-11-22 19:23:20,315 INFO [Thread-542] distributedshell.Client (Client.java:run(462)) - Queue info, queueName=default, queueCurrentCapacity=0.0, queueMaxCapacity=1.0, queueApplicationCount=0, queueChildQueueCount=0 2015-11-22 19:23:20,325 INFO [Thread-542] distributedshell.Client (Client.java:run(472)) - User ACL Info for Queue, queueName=root, userAcl=SUBMIT_APPLICATIONS 2015-11-22 19:23:20,325 INFO [Thread-542] distributedshell.Client (Client.java:run(472)) - User ACL Info for Queue, queueName=root, userAcl=ADMINISTER_QUEUE 2015-11-22 19:23:20,325 INFO [Thread-542] distributedshell.Client (Client.java:run(472)) - User ACL Info for Queue, queueName=default, userAcl=SUBMIT_APPLICATIONS 2015-11-22 19:23:20,325 INFO [Thread-542] distributedshell.Client (Client.java:run(472)) - User ACL Info for Queue, queueName=default, userAcl=ADMINISTER_QUEUE 2015-11-22 19:23:20,335 INFO [IPC Server handler 10 on 59307] resourcemanager.ClientRMService (ClientRMService.java:getNewApplicationId(305)) - Allocated new applicationId: 1 2015-11-22 19:23:20,336 INFO [Thread-542] distributedshell.Client (Client.java:run(491)) - Max mem capability of resources in this cluster 8192 2015-11-22 19:23:20,336 INFO [Thread-542] distributedshell.Client (Client.java:run(502)) - Max virtual cores capability of resources in this cluster 4 2015-11-22 19:23:20,337 INFO [Thread-542] distributedshell.Client (Client.java:run(528)) - Copy App Master jar from local filesystem and add to local environment 2015-11-22 19:23:20,373 INFO [Thread-542] distributedshell.Client (Client.java:run(577)) - Set the environment for the application master 2015-11-22 19:23:20,373 INFO [Thread-542] distributedshell.Client (Client.java:run(619)) - Setting up app master command 2015-11-22 19:23:20,373 INFO [Thread-542] distributedshell.Client (Client.java:run(650)) - Completed setting up app master command {{JAVA_HOME}}/bin/java -Xmx512m org.apache.hadoop.yarn.applications.distributedshell.ApplicationMaster --container_memory 128 --container_vcores 1 --num_containers 2 --priority 0 1>/AppMaster.stdout 2>/AppMaster.stderr 2015-11-22 19:23:20,379 INFO [Thread-542] distributedshell.Client (Client.java:run(706)) - Submitting application to ASM 2015-11-22 19:23:20,404 INFO [IPC Server handler 15 on 59307] capacity.CapacityScheduler (CapacityScheduler.java:checkAndGetApplicationPriority(1941)) - Priority '0' is acceptable in queue : default for application: application_1448220193913_0001 for the user: ubuntu 2015-11-22 19:23:20,414 WARN [IPC Server handler 15 on 59307] rmapp.RMAppImpl (RMAppImpl.java:(420)) - The specific max attempts: 0 for application: 1 is invalid, because it is out of the range [1, 2]. Use the global max attempts instead. 2015-11-22 19:23:20,415 INFO [IPC Server handler 15 on 59307] resourcemanager.ClientRMService (ClientRMService.java:submitApplication(602)) - Application with id 1 submitted by user ubuntu 2015-11-22 19:23:20,415 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:transition(1079)) - Storing application with id application_1448220193913_0001 2015-11-22 19:23:20,417 INFO [IPC Server handler 15 on 59307] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(154)) - USER=ubuntu IP=172.31.20.42 OPERATION=Submit Application Request TARGET=ClientRMService RESULT=SUCCESS APPID=application_1448220193913_0001 2015-11-22 19:23:20,426 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220193913_0001 State change from NEW to NEW_SAVING 2015-11-22 19:23:20,427 INFO [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:transition(208)) - Storing info for app: application_1448220193913_0001 2015-11-22 19:23:20,445 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220193913_0001 State change from NEW_SAVING to SUBMITTED 2015-11-22 19:23:20,447 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:addApplication(349)) - Application added - appId: application_1448220193913_0001 user: ubuntu leaf-queue of parent: root #applications: 1 2015-11-22 19:23:20,447 INFO [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:addApplication(765)) - Accepted application application_1448220193913_0001 from user: ubuntu, in queue: default 2015-11-22 19:23:20,450 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220193913_0001 State change from SUBMITTED to ACCEPTED 2015-11-22 19:23:20,463 INFO [AsyncDispatcher event handler] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:registerAppAttempt(661)) - Registering app attempt : appattempt_1448220193913_0001_000001 2015-11-22 19:23:20,464 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220193913_0001_000001 State change from NEW to SUBMITTED 2015-11-22 19:23:20,482 INFO [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:activateApplications(705)) - Application application_1448220193913_0001 from user: ubuntu activated in queue: default 2015-11-22 19:23:20,483 INFO [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:addApplicationAttempt(720)) - Application added - appId: application_1448220193913_0001 user: ubuntu, leaf-queue: default #user-pending-applications: 0 #user-active-applications: 1 #queue-pending-applications: 0 #queue-active-applications: 1 2015-11-22 19:23:20,483 INFO [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:addApplicationAttempt(805)) - Added Application Attempt appattempt_1448220193913_0001_000001 to scheduler from user ubuntu in queue default 2015-11-22 19:23:20,486 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220193913_0001_000001 State change from SUBMITTED to SCHEDULED 2015-11-22 19:23:20,652 INFO [Thread-542] impl.YarnClientImpl (YarnClientImpl.java:submitApplication(279)) - Submitted application application_1448220193913_0001 2015-11-22 19:23:20,706 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220193913_0001_01_000001 Container Transitioned from NEW to ALLOCATED 2015-11-22 19:23:20,707 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220193913_0001 CONTAINERID=container_1448220193913_0001_01_000001 2015-11-22 19:23:20,707 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(154)) - Assigned container container_1448220193913_0001_01_000001 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:45634, which has 1 containers, used and available after allocation 2015-11-22 19:23:20,708 INFO [ResourceManager Event Processor] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(89)) - assignedContainer application attempt=appattempt_1448220193913_0001_000001 container=container_1448220193913_0001_01_000001 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@22d397e4 clusterResource= type=OFF_SWITCH 2015-11-22 19:23:20,709 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(589)) - Re-sorting assigned queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.125, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 2015-11-22 19:23:20,709 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(461)) - assignedContainer queue=root usedCapacity=0.125 absoluteUsedCapacity=0.125 used= cluster= 2015-11-22 19:23:20,725 INFO [AsyncDispatcher event handler] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:createAndGetNMToken(200)) - Sending NMToken for nodeId : ip-172-31-20-42.ap-northeast-1.compute.internal:45634 for container : container_1448220193913_0001_01_000001 2015-11-22 19:23:20,733 INFO [AsyncDispatcher event handler] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220193913_0001_01_000001 Container Transitioned from ALLOCATED to ACQUIRED 2015-11-22 19:23:20,734 INFO [AsyncDispatcher event handler] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:clearNodeSetForAttempt(146)) - Clear node set for appattempt_1448220193913_0001_000001 2015-11-22 19:23:20,734 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:storeAttempt(2026)) - Storing attempt: AppId: application_1448220193913_0001 AttemptId: appattempt_1448220193913_0001_000001 MasterContainer: Container: [ContainerId: container_1448220193913_0001_01_000001, NodeId: ip-172-31-20-42.ap-northeast-1.compute.internal:45634, NodeHttpAddress: ip-172-31-20-42.ap-northeast-1.compute.internal:41860, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.31.20.42:45634 }, ] 2015-11-22 19:23:20,742 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220193913_0001_000001 State change from SCHEDULED to ALLOCATED_SAVING 2015-11-22 19:23:20,746 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220193913_0001_000001 State change from ALLOCATED_SAVING to ALLOCATED 2015-11-22 19:23:20,748 INFO [ApplicationMasterLauncher #0] amlauncher.AMLauncher (AMLauncher.java:run(254)) - Launching masterappattempt_1448220193913_0001_000001 2015-11-22 19:23:20,765 INFO [ApplicationMasterLauncher #0] amlauncher.AMLauncher (AMLauncher.java:launch(106)) - Setting up container Container: [ContainerId: container_1448220193913_0001_01_000001, NodeId: ip-172-31-20-42.ap-northeast-1.compute.internal:45634, NodeHttpAddress: ip-172-31-20-42.ap-northeast-1.compute.internal:41860, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.31.20.42:45634 }, ] for AM appattempt_1448220193913_0001_000001 2015-11-22 19:23:20,765 INFO [ApplicationMasterLauncher #0] amlauncher.AMLauncher (AMLauncher.java:createAMContainerLaunchContext(188)) - Command to launch container container_1448220193913_0001_01_000001 : {{JAVA_HOME}}/bin/java -Xmx512m org.apache.hadoop.yarn.applications.distributedshell.ApplicationMaster --container_memory 128 --container_vcores 1 --num_containers 2 --priority 0 1>/AppMaster.stdout 2>/AppMaster.stderr 2015-11-22 19:23:20,765 INFO [ApplicationMasterLauncher #0] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:createAndGetAMRMToken(195)) - Create AMRMToken for ApplicationAttempt: appattempt_1448220193913_0001_000001 2015-11-22 19:23:20,769 INFO [ApplicationMasterLauncher #0] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:createPassword(307)) - Creating password for appattempt_1448220193913_0001_000001 2015-11-22 19:23:20,795 INFO [Socket Reader #1 for port 45634] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220193913_0001_000001 (auth:SIMPLE) 2015-11-22 19:23:20,815 INFO [IPC Server handler 0 on 45634] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:startContainerInternal(889)) - Start request for container_1448220193913_0001_01_000001 by user ubuntu 2015-11-22 19:23:20,835 INFO [IPC Server handler 0 on 45634] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:startContainerInternal(930)) - Creating a new application reference for app application_1448220193913_0001 2015-11-22 19:23:20,841 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(463)) - Application application_1448220193913_0001 transitioned from NEW to INITING 2015-11-22 19:23:20,841 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(303)) - Adding container_1448220193913_0001_01_000001 to application application_1448220193913_0001 2015-11-22 19:23:20,842 INFO [IPC Server handler 0 on 45634] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu IP=172.31.20.42 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1448220193913_0001 CONTAINERID=container_1448220193913_0001_01_000001 2015-11-22 19:23:20,847 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(463)) - Application application_1448220193913_0001 transitioned from INITING to RUNNING 2015-11-22 19:23:20,852 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220193913_0001_01_000001 transitioned from NEW to LOCALIZING 2015-11-22 19:23:20,852 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_INIT for appId application_1448220193913_0001 2015-11-22 19:23:20,853 INFO [ApplicationMasterLauncher #0] amlauncher.AMLauncher (AMLauncher.java:launch(127)) - Done launching container Container: [ContainerId: container_1448220193913_0001_01_000001, NodeId: ip-172-31-20-42.ap-northeast-1.compute.internal:45634, NodeHttpAddress: ip-172-31-20-42.ap-northeast-1.compute.internal:41860, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.31.20.42:45634 }, ] for AM appattempt_1448220193913_0001_000001 2015-11-22 19:23:20,853 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220193913_0001_000001 State change from ALLOCATED to LAUNCHED 2015-11-22 19:23:20,861 INFO [AsyncDispatcher event handler] localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource file:/home/ubuntu/DistributedShell/application_1448220193913_0001/AppMaster.jar transitioned from INIT to DOWNLOADING 2015-11-22 19:23:20,861 INFO [AsyncDispatcher event handler] localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource file:/home/ubuntu/DistributedShell/application_1448220193913_0001/shellCommands transitioned from INIT to DOWNLOADING 2015-11-22 19:23:20,862 INFO [AsyncDispatcher event handler] localizer.ResourceLocalizationService (ResourceLocalizationService.java:handle(742)) - Created localizer for container_1448220193913_0001_01_000001 2015-11-22 19:23:20,876 INFO [LocalizerRunner for container_1448220193913_0001_01_000001] localizer.ResourceLocalizationService (ResourceLocalizationService.java:writeCredentials(1221)) - Writing credentials to the nmPrivate file /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/container_1448220193913_0001_01_000001.tokens. Credentials list: 2015-11-22 19:23:20,883 INFO [LocalizerRunner for container_1448220193913_0001_01_000001] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:createUserCacheDirs(632)) - Initializing user ubuntu 2015-11-22 19:23:20,908 INFO [LocalizerRunner for container_1448220193913_0001_01_000001] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:startLocalizer(126)) - Copying from /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/container_1448220193913_0001_01_000001.tokens to /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220193913_0001/container_1448220193913_0001_01_000001.tokens 2015-11-22 19:23:20,909 INFO [LocalizerRunner for container_1448220193913_0001_01_000001] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:startLocalizer(133)) - Localizer CWD set to /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220193913_0001 = file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220193913_0001 2015-11-22 19:23:21,005 INFO [IPC Server handler 4 on 42804] localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource file:/home/ubuntu/DistributedShell/application_1448220193913_0001/AppMaster.jar(->/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220193913_0001/filecache/10/AppMaster.jar) transitioned from DOWNLOADING to LOCALIZED 2015-11-22 19:23:21,038 INFO [IPC Server handler 3 on 42804] localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource file:/home/ubuntu/DistributedShell/application_1448220193913_0001/shellCommands(->/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220193913_0001/filecache/11/shellCommands) transitioned from DOWNLOADING to LOCALIZED 2015-11-22 19:23:21,040 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220193913_0001_01_000001 transitioned from LOCALIZING to LOCALIZED 2015-11-22 19:23:21,074 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220193913_0001_01_000001 transitioned from LOCALIZED to RUNNING 2015-11-22 19:23:21,102 INFO [ContainersLauncher #0] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:buildCommandExecutor(281)) - launchContainer: [nice, -n, 0, bash, /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220193913_0001/container_1448220193913_0001_01_000001/default_container_executor.sh] 2015-11-22 19:23:21,654 INFO [Thread-542] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=N/A, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220200414, yarnAppState=ACCEPTED, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:41686/proxy/application_1448220193913_0001/, appUser=ubuntu 2015-11-22 19:23:21,710 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220193913_0001_01_000001 Container Transitioned from ACQUIRED to RUNNING 2015-11-22 19:23:22,655 INFO [Thread-542] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=N/A, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220200414, yarnAppState=ACCEPTED, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:41686/proxy/application_1448220193913_0001/, appUser=ubuntu 2015-11-22 19:23:23,658 INFO [Thread-542] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=N/A, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220200414, yarnAppState=ACCEPTED, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:41686/proxy/application_1448220193913_0001/, appUser=ubuntu 2015-11-22 19:23:24,083 INFO [Socket Reader #1 for port 48969] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220193913_0001_000001 (auth:SIMPLE) 2015-11-22 19:23:24,095 INFO [IPC Server handler 0 on 48969] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:registerApplicationMaster(220)) - AM registration appattempt_1448220193913_0001_000001 2015-11-22 19:23:24,095 INFO [IPC Server handler 0 on 48969] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(133)) - USER=ubuntu IP=172.31.20.42 OPERATION=Register App Master TARGET=ApplicationMasterService RESULT=SUCCESS APPID=application_1448220193913_0001 APPATTEMPTID=appattempt_1448220193913_0001_000001 2015-11-22 19:23:24,096 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220193913_0001_000001 State change from LAUNCHED to RUNNING 2015-11-22 19:23:24,096 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220193913_0001 State change from ACCEPTED to RUNNING 2015-11-22 19:23:24,660 INFO [Thread-542] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220200414, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:41686/proxy/application_1448220193913_0001/, appUser=ubuntu 2015-11-22 19:23:25,662 INFO [Thread-542] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220200414, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:41686/proxy/application_1448220193913_0001/, appUser=ubuntu 2015-11-22 19:23:25,701 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220193913_0001_01_000002 Container Transitioned from NEW to ALLOCATED 2015-11-22 19:23:25,701 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220193913_0001 CONTAINERID=container_1448220193913_0001_01_000002 2015-11-22 19:23:25,701 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(154)) - Assigned container container_1448220193913_0001_01_000002 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:45634, which has 2 containers, used and available after allocation 2015-11-22 19:23:25,702 INFO [ResourceManager Event Processor] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(89)) - assignedContainer application attempt=appattempt_1448220193913_0001_000001 container=container_1448220193913_0001_01_000002 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@22d397e4 clusterResource= type=OFF_SWITCH 2015-11-22 19:23:25,702 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(589)) - Re-sorting assigned queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.15625, absoluteUsedCapacity=0.15625, numApps=1, numContainers=2 2015-11-22 19:23:25,702 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(461)) - assignedContainer queue=root usedCapacity=0.15625 absoluteUsedCapacity=0.15625 used= cluster= 2015-11-22 19:23:26,164 INFO [IPC Server handler 2 on 48969] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:createAndGetNMToken(200)) - Sending NMToken for nodeId : ip-172-31-20-42.ap-northeast-1.compute.internal:45634 for container : container_1448220193913_0001_01_000002 2015-11-22 19:23:26,165 INFO [IPC Server handler 2 on 48969] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220193913_0001_01_000002 Container Transitioned from ALLOCATED to ACQUIRED 2015-11-22 19:23:26,228 INFO [Socket Reader #1 for port 45634] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220193913_0001_000001 (auth:SIMPLE) 2015-11-22 19:23:26,236 INFO [IPC Server handler 1 on 45634] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:startContainerInternal(889)) - Start request for container_1448220193913_0001_01_000002 by user ubuntu 2015-11-22 19:23:26,237 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(303)) - Adding container_1448220193913_0001_01_000002 to application application_1448220193913_0001 2015-11-22 19:23:26,237 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220193913_0001_01_000002 transitioned from NEW to LOCALIZED 2015-11-22 19:23:26,237 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_INIT for appId application_1448220193913_0001 2015-11-22 19:23:26,237 INFO [IPC Server handler 1 on 45634] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu IP=172.31.20.42 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1448220193913_0001 CONTAINERID=container_1448220193913_0001_01_000002 2015-11-22 19:23:26,267 INFO [Socket Reader #1 for port 45634] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220193913_0001_000001 (auth:SIMPLE) 2015-11-22 19:23:26,289 INFO [IPC Server handler 2 on 45634] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:getContainerStatusInternal(1221)) - Getting container-status for container_1448220193913_0001_01_000002 2015-11-22 19:23:26,289 INFO [IPC Server handler 2 on 45634] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:getContainerStatusInternal(1235)) - Returning ContainerStatus: [ContainerId: container_1448220193913_0001_01_000002, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ] 2015-11-22 19:23:26,310 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220193913_0001_01_000002 transitioned from LOCALIZED to RUNNING 2015-11-22 19:23:26,381 INFO [ContainersLauncher #1] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:buildCommandExecutor(281)) - launchContainer: [nice, -n, 0, bash, /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220193913_0001/container_1448220193913_0001_01_000002/default_container_executor.sh] 2015-11-22 19:23:26,392 INFO [ContainersLauncher #1] launcher.ContainerLaunch (ContainerLaunch.java:call(356)) - Container container_1448220193913_0001_01_000002 succeeded 2015-11-22 19:23:26,393 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220193913_0001_01_000002 transitioned from RUNNING to EXITED_WITH_SUCCESS 2015-11-22 19:23:26,393 INFO [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(380)) - Cleaning up container container_1448220193913_0001_01_000002 2015-11-22 19:23:26,410 INFO [AsyncDispatcher event handler] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu OPERATION=Container Finished - Succeeded TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1448220193913_0001 CONTAINERID=container_1448220193913_0001_01_000002 2015-11-22 19:23:26,411 INFO [DeletionService #0] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(480)) - Deleting absolute path : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220193913_0001/container_1448220193913_0001_01_000002 2015-11-22 19:23:26,412 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220193913_0001_01_000002 transitioned from EXITED_WITH_SUCCESS to DONE 2015-11-22 19:23:26,412 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(346)) - Removing container_1448220193913_0001_01_000002 from application application_1448220193913_0001 2015-11-22 19:23:26,412 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_STOP for appId application_1448220193913_0001 2015-11-22 19:23:26,664 INFO [Thread-542] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220200414, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:41686/proxy/application_1448220193913_0001/, appUser=ubuntu 2015-11-22 19:23:26,702 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220193913_0001_01_000002 Container Transitioned from ACQUIRED to COMPLETED 2015-11-22 19:23:26,702 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220193913_0001 CONTAINERID=container_1448220193913_0001_01_000002 2015-11-22 19:23:26,703 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:releaseContainer(248)) - Released container container_1448220193913_0001_01_000002 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:45634, which currently has 1 containers, used and available, release resources=true 2015-11-22 19:23:26,703 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220193913_0001_01_000003 Container Transitioned from NEW to ALLOCATED 2015-11-22 19:23:26,703 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220193913_0001 CONTAINERID=container_1448220193913_0001_01_000003 2015-11-22 19:23:26,704 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(154)) - Assigned container container_1448220193913_0001_01_000003 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:45634, which has 2 containers, used and available after allocation 2015-11-22 19:23:26,704 INFO [ResourceManager Event Processor] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(89)) - assignedContainer application attempt=appattempt_1448220193913_0001_000001 container=container_1448220193913_0001_01_000003 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@22d397e4 clusterResource= type=OFF_SWITCH 2015-11-22 19:23:26,704 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(589)) - Re-sorting assigned queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.15625, absoluteUsedCapacity=0.15625, numApps=1, numContainers=2 2015-11-22 19:23:26,704 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(461)) - assignedContainer queue=root usedCapacity=0.15625 absoluteUsedCapacity=0.15625 used= cluster= 2015-11-22 19:23:27,180 INFO [IPC Server handler 4 on 48969] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220193913_0001_01_000003 Container Transitioned from ALLOCATED to ACQUIRED 2015-11-22 19:23:27,245 INFO [Socket Reader #1 for port 45634] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220193913_0001_000001 (auth:SIMPLE) 2015-11-22 19:23:27,249 INFO [IPC Server handler 2 on 45634] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:startContainerInternal(889)) - Start request for container_1448220193913_0001_01_000003 by user ubuntu 2015-11-22 19:23:27,250 INFO [IPC Server handler 2 on 45634] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu IP=172.31.20.42 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1448220193913_0001 CONTAINERID=container_1448220193913_0001_01_000003 2015-11-22 19:23:27,250 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(303)) - Adding container_1448220193913_0001_01_000003 to application application_1448220193913_0001 2015-11-22 19:23:27,251 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220193913_0001_01_000003 transitioned from NEW to LOCALIZED 2015-11-22 19:23:27,251 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_INIT for appId application_1448220193913_0001 2015-11-22 19:23:27,270 INFO [Socket Reader #1 for port 45634] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220193913_0001_000001 (auth:SIMPLE) 2015-11-22 19:23:27,290 INFO [IPC Server handler 4 on 45634] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:getContainerStatusInternal(1221)) - Getting container-status for container_1448220193913_0001_01_000003 2015-11-22 19:23:27,290 INFO [IPC Server handler 4 on 45634] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:getContainerStatusInternal(1235)) - Returning ContainerStatus: [ContainerId: container_1448220193913_0001_01_000003, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ] 2015-11-22 19:23:27,302 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220193913_0001_01_000003 transitioned from LOCALIZED to RUNNING 2015-11-22 19:23:27,340 INFO [ContainersLauncher #1] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:buildCommandExecutor(281)) - launchContainer: [nice, -n, 0, bash, /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220193913_0001/container_1448220193913_0001_01_000003/default_container_executor.sh] 2015-11-22 19:23:27,349 INFO [ContainersLauncher #1] launcher.ContainerLaunch (ContainerLaunch.java:call(356)) - Container container_1448220193913_0001_01_000003 succeeded 2015-11-22 19:23:27,350 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220193913_0001_01_000003 transitioned from RUNNING to EXITED_WITH_SUCCESS 2015-11-22 19:23:27,350 INFO [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(380)) - Cleaning up container container_1448220193913_0001_01_000003 2015-11-22 19:23:27,375 INFO [DeletionService #1] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(480)) - Deleting absolute path : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220193913_0001/container_1448220193913_0001_01_000003 2015-11-22 19:23:27,375 INFO [AsyncDispatcher event handler] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu OPERATION=Container Finished - Succeeded TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1448220193913_0001 CONTAINERID=container_1448220193913_0001_01_000003 2015-11-22 19:23:27,376 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220193913_0001_01_000003 transitioned from EXITED_WITH_SUCCESS to DONE 2015-11-22 19:23:27,376 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(346)) - Removing container_1448220193913_0001_01_000003 from application application_1448220193913_0001 2015-11-22 19:23:27,376 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_STOP for appId application_1448220193913_0001 2015-11-22 19:23:27,666 INFO [Thread-542] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220200414, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:41686/proxy/application_1448220193913_0001/, appUser=ubuntu 2015-11-22 19:23:27,703 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220193913_0001_01_000003 Container Transitioned from ACQUIRED to COMPLETED 2015-11-22 19:23:27,703 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220193913_0001 CONTAINERID=container_1448220193913_0001_01_000003 2015-11-22 19:23:27,703 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:releaseContainer(248)) - Released container container_1448220193913_0001_01_000003 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:45634, which currently has 1 containers, used and available, release resources=true 2015-11-22 19:23:28,411 INFO [Socket Reader #1 for port 45634] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220193913_0001_000001 (auth:SIMPLE) 2015-11-22 19:23:28,420 INFO [IPC Server handler 3 on 45634] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:stopContainerInternal(1163)) - Stopping container with container Id: container_1448220193913_0001_01_000003 2015-11-22 19:23:28,421 INFO [IPC Server handler 3 on 45634] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu IP=172.31.20.42 OPERATION=Stop Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1448220193913_0001 CONTAINERID=container_1448220193913_0001_01_000003 2015-11-22 19:23:28,422 INFO [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:removeOrTrackCompletedContainersFromContext(605)) - Removed completed containers from NM context: [container_1448220193913_0001_01_000002] 2015-11-22 19:23:28,436 INFO [Socket Reader #1 for port 45634] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220193913_0001_000001 (auth:SIMPLE) 2015-11-22 19:23:28,446 INFO [IPC Server handler 0 on 45634] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:stopContainerInternal(1163)) - Stopping container with container Id: container_1448220193913_0001_01_000002 2015-11-22 19:23:28,465 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:rememberTargetTransitionsAndStoreState(1258)) - Updating application attempt appattempt_1448220193913_0001_000001 with final state: FINISHING, and exit status: -1000 2015-11-22 19:23:28,466 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220193913_0001_000001 State change from RUNNING to FINAL_SAVING 2015-11-22 19:23:28,467 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:rememberTargetTransitionsAndStoreState(1098)) - Updating application application_1448220193913_0001 with final state: FINISHING 2015-11-22 19:23:28,467 INFO [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:transition(237)) - Updating info for app: application_1448220193913_0001 2015-11-22 19:23:28,468 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220193913_0001 State change from RUNNING to FINAL_SAVING 2015-11-22 19:23:28,471 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220193913_0001_000001 State change from FINAL_SAVING to FINISHING 2015-11-22 19:23:28,471 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220193913_0001 State change from FINAL_SAVING to FINISHING 2015-11-22 19:23:28,572 INFO [IPC Server handler 6 on 48969] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:finishApplicationMaster(300)) - application_1448220193913_0001 unregistered successfully. 2015-11-22 19:23:28,591 INFO [ContainersLauncher #0] launcher.ContainerLaunch (ContainerLaunch.java:call(356)) - Container container_1448220193913_0001_01_000001 succeeded 2015-11-22 19:23:28,592 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220193913_0001_01_000001 transitioned from RUNNING to EXITED_WITH_SUCCESS 2015-11-22 19:23:28,592 INFO [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(380)) - Cleaning up container container_1448220193913_0001_01_000001 2015-11-22 19:23:28,609 INFO [DeletionService #2] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(480)) - Deleting absolute path : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220193913_0001/container_1448220193913_0001_01_000001 2015-11-22 19:23:28,609 INFO [AsyncDispatcher event handler] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu OPERATION=Container Finished - Succeeded TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1448220193913_0001 CONTAINERID=container_1448220193913_0001_01_000001 2015-11-22 19:23:28,610 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220193913_0001_01_000001 transitioned from EXITED_WITH_SUCCESS to DONE 2015-11-22 19:23:28,610 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(346)) - Removing container_1448220193913_0001_01_000001 from application application_1448220193913_0001 2015-11-22 19:23:28,610 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_STOP for appId application_1448220193913_0001 2015-11-22 19:23:28,668 INFO [Thread-542] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220200414, yarnAppState=FINISHED, distributedFinalState=SUCCEEDED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:41686/proxy/application_1448220193913_0001/, appUser=ubuntu 2015-11-22 19:23:28,668 INFO [Thread-542] distributedshell.Client (Client.java:monitorApplication(759)) - Application has completed successfully. Breaking monitoring loop 2015-11-22 19:23:28,669 INFO [Thread-541] distributedshell.TestDistributedShell (TestDistributedShell.java:testDSShell(239)) - Client run completed. Result=true 2015-11-22 19:23:28,673 ERROR [Thread[Thread-537,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(670)) - ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted 2015-11-22 19:23:28,675 INFO [main] mortbay.log (Slf4jLog.java:info(67)) - Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:0 2015-11-22 19:23:28,778 INFO [main] applicationhistoryservice.ApplicationHistoryManagerImpl (ApplicationHistoryManagerImpl.java:serviceStop(78)) - Stopping ApplicationHistory 2015-11-22 19:23:28,780 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 59513 2015-11-22 19:23:28,781 INFO [IPC Server listener on 59513] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 59513 2015-11-22 19:23:28,781 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:23:28,784 ERROR [Thread[Thread-519,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(670)) - ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted 2015-11-22 19:23:28,786 INFO [main] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:unRegisterNM(268)) - Successfully Unregistered the Node ip-172-31-20-42.ap-northeast-1.compute.internal:45634 with ResourceManager. 2015-11-22 19:23:28,793 ERROR [Thread[Thread-512,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(670)) - ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted 2015-11-22 19:23:28,795 INFO [main] mortbay.log (Slf4jLog.java:info(67)) - Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:0 2015-11-22 19:23:28,898 INFO [main] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:cleanUpApplicationsOnNMShutDown(556)) - Applications still running : [application_1448220193913_0001] 2015-11-22 19:23:28,900 INFO [main] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:cleanUpApplicationsOnNMShutDown(572)) - Waiting for Applications to be Finished 2015-11-22 19:23:28,900 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(463)) - Application application_1448220193913_0001 transitioned from RUNNING to APPLICATION_RESOURCES_CLEANINGUP 2015-11-22 19:23:28,901 INFO [DeletionService #1] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(480)) - Deleting absolute path : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220193913_0001 2015-11-22 19:23:28,901 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event APPLICATION_STOP for appId application_1448220193913_0001 2015-11-22 19:23:28,902 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(463)) - Application application_1448220193913_0001 transitioned from APPLICATION_RESOURCES_CLEANINGUP to FINISHED 2015-11-22 19:23:28,902 INFO [AsyncDispatcher event handler] loghandler.NonAggregatingLogHandler (NonAggregatingLogHandler.java:handle(167)) - Scheduling Log Deletion for application: application_1448220193913_0001, with delay of 10800 seconds 2015-11-22 19:23:32,901 INFO [main] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:cleanUpApplicationsOnNMShutDown(589)) - Done waiting for Applications to be Finished. Still alive: [application_1448220193913_0001] 2015-11-22 19:23:32,902 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 45634 2015-11-22 19:23:32,904 INFO [IPC Server listener on 45634] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 45634 2015-11-22 19:23:32,904 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:23:42,910 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 42804 2015-11-22 19:23:42,911 INFO [IPC Server listener on 42804] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 42804 2015-11-22 19:23:42,912 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:23:42,912 INFO [Public Localizer] localizer.ResourceLocalizationService (ResourceLocalizationService.java:run(908)) - Public cache exiting 2015-11-22 19:23:42,915 WARN [Node Resource Monitor] nodemanager.NodeResourceMonitorImpl (NodeResourceMonitorImpl.java:run(154)) - org.apache.hadoop.yarn.server.nodemanager.NodeResourceMonitorImpl is interrupted. Exiting. 2015-11-22 19:23:42,922 ERROR [Thread[Thread-462,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(670)) - ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted 2015-11-22 19:23:42,923 INFO [main] mortbay.log (Slf4jLog.java:info(67)) - Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:0 2015-11-22 19:23:43,025 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 59307 2015-11-22 19:23:43,028 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:23:43,028 INFO [IPC Server listener on 59307] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 59307 2015-11-22 19:23:43,029 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 60680 2015-11-22 19:23:43,030 INFO [IPC Server listener on 60680] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 60680 2015-11-22 19:23:43,031 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:23:43,030 INFO [main] resourcemanager.ResourceManager (ResourceManager.java:transitionToStandby(1051)) - Transitioning to standby state 2015-11-22 19:23:43,037 WARN [ApplicationMaster Launcher] amlauncher.ApplicationMasterLauncher (ApplicationMasterLauncher.java:run(122)) - org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher$LauncherThread interrupted. Returning. 2015-11-22 19:23:43,037 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 48969 2015-11-22 19:23:43,040 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:23:43,041 INFO [IPC Server listener on 48969] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 48969 2015-11-22 19:23:43,041 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 42633 2015-11-22 19:23:43,046 ERROR [ResourceManager Event Processor] resourcemanager.ResourceManager (ResourceManager.java:run(691)) - Returning, interrupted : java.lang.InterruptedException 2015-11-22 19:23:43,046 INFO [IPC Server listener on 42633] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 42633 2015-11-22 19:23:43,046 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:23:43,046 INFO [Ping Checker] util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(135)) - NMLivelinessMonitor thread interrupted 2015-11-22 19:23:43,048 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:serviceStop(141)) - AsyncDispatcher is draining to stop, igonring any new events. 2015-11-22 19:23:43,049 INFO [Ping Checker] util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(135)) - AMLivelinessMonitor thread interrupted 2015-11-22 19:23:43,050 ERROR [Thread[Thread-293,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(670)) - ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted 2015-11-22 19:23:43,050 INFO [Ping Checker] util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(135)) - org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.ContainerAllocationExpirer thread interrupted 2015-11-22 19:23:43,050 INFO [Ping Checker] util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(135)) - AMLivelinessMonitor thread interrupted 2015-11-22 19:23:43,050 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stop(211)) - Stopping ApplicationHistoryServer metrics system... 2015-11-22 19:23:43,053 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stop(217)) - ApplicationHistoryServer metrics system stopped. 2015-11-22 19:23:43,053 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:shutdown(608)) - ApplicationHistoryServer metrics system shutdown complete. 2015-11-22 19:23:43,054 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:serviceStop(141)) - AsyncDispatcher is draining to stop, igonring any new events. 2015-11-22 19:23:43,054 INFO [main] resourcemanager.ResourceManager (ResourceManager.java:transitionToStandby(1058)) - Transitioned to standby state 2015-11-22 19:23:43,068 INFO [main] distributedshell.TestDistributedShell (TestDistributedShell.java:setupInternal(77)) - Starting up YARN cluster 2015-11-22 19:23:43,080 INFO [main] net.ServerSocketUtil (ServerSocketUtil.java:getPort(52)) - Using port 10021 2015-11-22 19:23:43,096 INFO [main] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2363)) - found resource core-site.xml at file:/home/ubuntu/hadoop-dev/hadoop-common-project/hadoop-common/target/test-classes/core-site.xml 2015-11-22 19:23:43,108 WARN [main] conf.Configuration (Configuration.java:loadProperty(2687)) - core-site.xml:an attempt to override final parameter: hadoop.tmp.dir; Ignoring. 2015-11-22 19:23:43,108 INFO [main] security.Groups (Groups.java:refresh(256)) - clearing userToGroupsMap cache 2015-11-22 19:23:43,110 INFO [main] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2363)) - found resource yarn-site.xml at file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/test-classes/yarn-site.xml 2015-11-22 19:23:43,126 WARN [main] conf.Configuration (Configuration.java:loadProperty(2687)) - yarn-site.xml:an attempt to override final parameter: hadoop.tmp.dir; Ignoring. 2015-11-22 19:23:43,126 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.RMFatalEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMFatalEventDispatcher 2015-11-22 19:23:43,127 INFO [main] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:(75)) - NMTokenKeyRollingInterval: 86400000ms and NMTokenKeyActivationDelay: 900000ms 2015-11-22 19:23:43,127 INFO [main] security.RMContainerTokenSecretManager (RMContainerTokenSecretManager.java:(78)) - ContainerTokenKeyRollingInterval: 86400000ms and ContainerTokenKeyActivationDelay: 900000ms 2015-11-22 19:23:43,127 INFO [main] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:(94)) - AMRMTokenKeyRollingInterval: 86400000ms and AMRMTokenKeyActivationDelay: 900000 ms 2015-11-22 19:23:43,128 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStoreEventType for class org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$ForwardingEventHandler 2015-11-22 19:23:43,128 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.NodesListManagerEventType for class org.apache.hadoop.yarn.server.resourcemanager.NodesListManager 2015-11-22 19:23:43,128 INFO [main] resourcemanager.ResourceManager (ResourceManager.java:createScheduler(295)) - Using Scheduler: org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler 2015-11-22 19:23:43,128 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.SchedulerEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher 2015-11-22 19:23:43,128 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationEventDispatcher 2015-11-22 19:23:43,128 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationAttemptEventDispatcher 2015-11-22 19:23:43,129 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$NodeEventDispatcher 2015-11-22 19:23:43,130 INFO [main] impl.MetricsConfig (MetricsConfig.java:loadFirst(115)) - loaded properties from hadoop-metrics2.properties 2015-11-22 19:23:43,132 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:startTimer(374)) - Scheduled snapshot period at 10 second(s). 2015-11-22 19:23:43,132 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:start(192)) - ResourceManager metrics system started 2015-11-22 19:23:43,133 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.RMAppManagerEventType for class org.apache.hadoop.yarn.server.resourcemanager.RMAppManager 2015-11-22 19:23:43,133 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncherEventType for class org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher 2015-11-22 19:23:43,134 WARN [main] util.MBeans (MBeans.java:register(73)) - Failed to register MBean "Hadoop:service=ResourceManager,name=RMNMInfo": Instance already exists. 2015-11-22 19:23:43,134 INFO [main] resourcemanager.RMNMInfo (RMNMInfo.java:(63)) - Registered RMNMInfo MBean 2015-11-22 19:23:43,134 INFO [main] util.HostsFileReader (HostsFileReader.java:refresh(132)) - Refreshing hosts (include/exclude) list 2015-11-22 19:23:43,135 INFO [main] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2363)) - found resource capacity-scheduler.xml at jar:file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/target/hadoop-yarn-server-tests-2.8.0-SNAPSHOT-tests.jar!/capacity-scheduler.xml 2015-11-22 19:23:43,149 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(647)) - max alloc mb per queue for root is undefined 2015-11-22 19:23:43,149 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(651)) - max alloc vcore per queue for root is undefined 2015-11-22 19:23:43,150 INFO [main] capacity.ParentQueue (ParentQueue.java:setupQueueConfigs(125)) - root, capacity=1.0, asboluteCapacity=1.0, maxCapacity=1.0, asboluteMaxCapacity=1.0, state=RUNNING, acls=ADMINISTER_QUEUE:*SUBMIT_APP:*, labels=*, , reservationsContinueLooking=true 2015-11-22 19:23:43,150 INFO [main] capacity.ParentQueue (ParentQueue.java:(104)) - Initialized parent-queue root name=root, fullname=root 2015-11-22 19:23:43,151 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(647)) - max alloc mb per queue for root.default is undefined 2015-11-22 19:23:43,151 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(651)) - max alloc vcore per queue for root.default is undefined 2015-11-22 19:23:43,152 INFO [main] capacity.LeafQueue (LeafQueue.java:setupQueueConfigs(217)) - Initializing default capacity = 1.0 [= (float) configuredCapacity / 100 ] asboluteCapacity = 1.0 [= parentAbsoluteCapacity * capacity ] maxCapacity = 1.0 [= configuredMaxCapacity ] absoluteMaxCapacity = 1.0 [= 1.0 maximumCapacity undefined, (parentAbsoluteMaxCapacity * maximumCapacity) / 100 otherwise ] userLimit = 100 [= configuredUserLimit ] userLimitFactor = 1.0 [= configuredUserLimitFactor ] maxApplications = 10000 [= configuredMaximumSystemApplicationsPerQueue or (int)(configuredMaximumSystemApplications * absoluteCapacity)] maxApplicationsPerUser = 10000 [= (int)(maxApplications * (userLimit / 100.0f) * userLimitFactor) ] usedCapacity = 0.0 [= usedResourcesMemory / (clusterResourceMemory * absoluteCapacity)] absoluteUsedCapacity = 0.0 [= usedResourcesMemory / clusterResourceMemory] maxAMResourcePerQueuePercent = 0.1 [= configuredMaximumAMResourcePercent ] minimumAllocationFactor = 0.984375 [= (float)(maximumAllocationMemory - minimumAllocationMemory) / maximumAllocationMemory ] maximumAllocation = [= configuredMaxAllocation ] numContainers = 0 [= currentNumContainers ] state = RUNNING [= configuredState ] acls = ADMINISTER_QUEUE:*SUBMIT_APP:* [= configuredAcls ] nodeLocalityDelay = -1 labels=*, nodeLocalityDelay = -1 reservationsContinueLooking = true preemptionDisabled = true defaultAppPriorityPerQueue = 0 2015-11-22 19:23:43,153 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:parseQueue(657)) - Initialized queue: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=0, numContainers=0 2015-11-22 19:23:43,153 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:parseQueue(657)) - Initialized queue: root: numChildQueue= 1, capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.0, numApps=0, numContainers=0 2015-11-22 19:23:43,153 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:initializeQueues(501)) - Initialized root queue root: numChildQueue= 1, capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.0, numApps=0, numContainers=0 2015-11-22 19:23:43,153 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:getUserGroupMappingPlacementRule(449)) - Initialized queue mappings, override: false 2015-11-22 19:23:43,153 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:initScheduler(314)) - Initialized CapacityScheduler with calculator=class org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator, minimumAllocation=<>, maximumAllocation=<>, asynchronousScheduling=false, asyncScheduleInterval=5ms 2015-11-22 19:23:43,154 INFO [main] metrics.SystemMetricsPublisher (SystemMetricsPublisher.java:serviceInit(95)) - YARN system metrics publishing service is not enabled 2015-11-22 19:23:43,154 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEventType for class org.apache.hadoop.yarn.server.MiniYARNCluster$1 2015-11-22 19:23:43,155 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:prepareDirs(558)) - Created localDir in /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0 2015-11-22 19:23:43,155 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:prepareDirs(558)) - Created logDir in /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-logDir-nm-0_0 2015-11-22 19:23:43,155 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceInit(540)) - Starting NM: 0 2015-11-22 19:23:43,166 INFO [main] nodemanager.NodeManager (NodeManager.java:getNodeHealthScriptRunner(255)) - Node Manager health check script is not available or doesn't have execute permission, so not starting the node health script runner. 2015-11-22 19:23:43,166 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ContainerEventDispatcher 2015-11-22 19:23:43,166 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ApplicationEventDispatcher 2015-11-22 19:23:43,167 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizationEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService 2015-11-22 19:23:43,167 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServicesEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices 2015-11-22 19:23:43,167 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl 2015-11-22 19:23:43,167 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncherEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncher 2015-11-22 19:23:43,167 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.ContainerManagerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl 2015-11-22 19:23:43,167 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.NodeManagerEventType for class org.apache.hadoop.yarn.server.MiniYARNCluster$ShortCircuitedNodeManager 2015-11-22 19:23:43,168 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:init(159)) - NodeManager metrics system started (again) 2015-11-22 19:23:43,170 INFO [main] nodemanager.NodeResourceMonitorImpl (NodeResourceMonitorImpl.java:serviceInit(76)) - Using ResourceCalculatorPlugin : org.apache.hadoop.yarn.util.ResourceCalculatorPlugin@9d5509a 2015-11-22 19:23:43,170 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.event.LogHandlerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.NonAggregatingLogHandler 2015-11-22 19:23:43,170 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.sharedcache.SharedCacheUploadEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.sharedcache.SharedCacheUploadService 2015-11-22 19:23:43,170 INFO [main] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:serviceInit(261)) - AMRMProxyService is disabled 2015-11-22 19:23:43,171 INFO [main] localizer.ResourceLocalizationService (ResourceLocalizationService.java:validateConf(224)) - per directory file limit = 8192 2015-11-22 19:23:43,184 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerTracker 2015-11-22 19:23:43,185 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(116)) - Using ResourceCalculatorPlugin : org.apache.hadoop.yarn.util.ResourceCalculatorPlugin@179ece50 2015-11-22 19:23:43,185 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(121)) - Using ResourceCalculatorProcessTree : null 2015-11-22 19:23:43,185 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(158)) - Physical memory check enabled: false 2015-11-22 19:23:43,185 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(159)) - Virtual memory check enabled: false 2015-11-22 19:23:43,185 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:isEnabled(206)) - Neither virutal-memory nor physical-memory monitoring is needed. Not running the monitor-thread 2015-11-22 19:23:43,186 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(162)) - ContainersMonitor enabled: false 2015-11-22 19:23:43,186 INFO [main] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:serviceInit(175)) - Nodemanager resources: memory set to 4096MB. 2015-11-22 19:23:43,186 INFO [main] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:serviceInit(176)) - Nodemanager resources: vcores set to 8. 2015-11-22 19:23:43,186 INFO [main] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:serviceInit(208)) - Initialized nodemanager with : physical-memory=4096 virtual-memory=8602 virtual-cores=8 2015-11-22 19:23:43,186 WARN [main] applicationhistoryservice.ApplicationHistoryServer (ApplicationHistoryServer.java:createApplicationHistoryManager(218)) - The filesystem based application history store is deprecated. 2015-11-22 19:23:43,186 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:init(159)) - ApplicationHistoryServer metrics system started (again) 2015-11-22 19:23:43,187 INFO [main] applicationhistoryservice.ApplicationHistoryManagerImpl (ApplicationHistoryManagerImpl.java:serviceInit(61)) - ApplicationHistory Init 2015-11-22 19:23:43,188 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:startResourceManager(326)) - Waiting for RM to start... 2015-11-22 19:23:43,188 INFO [RM-0] resourcemanager.ResourceManager (ResourceManager.java:transitionToActive(1024)) - Transitioning to active state 2015-11-22 19:23:43,190 INFO [RM-0] recovery.RMStateStore (RMStateStore.java:transition(482)) - Updating AMRMToken 2015-11-22 19:23:43,190 INFO [RM-0] security.RMContainerTokenSecretManager (RMContainerTokenSecretManager.java:rollMasterKey(106)) - Rolling master-key for container-tokens 2015-11-22 19:23:43,191 INFO [RM-0] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:rollMasterKey(95)) - Rolling master-key for nm-tokens 2015-11-22 19:23:43,191 INFO [RM-0] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:23:43,191 INFO [RM-0] security.RMDelegationTokenSecretManager (RMDelegationTokenSecretManager.java:storeNewMasterKey(87)) - storing master key with keyID 1 2015-11-22 19:23:43,191 INFO [RM-0] recovery.RMStateStore (RMStateStore.java:transition(433)) - Storing RMDTMasterKey. 2015-11-22 19:23:43,191 INFO [Thread[Thread-671,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2015-11-22 19:23:43,192 INFO [Thread[Thread-671,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:23:43,192 INFO [Thread[Thread-671,5,main]] security.RMDelegationTokenSecretManager (RMDelegationTokenSecretManager.java:storeNewMasterKey(87)) - storing master key with keyID 2 2015-11-22 19:23:43,193 INFO [Thread[Thread-671,5,main]] recovery.RMStateStore (RMStateStore.java:transition(433)) - Storing RMDTMasterKey. 2015-11-22 19:23:43,194 INFO [RM-0] nodelabels.CommonNodeLabelsManager (CommonNodeLabelsManager.java:addToCluserNodeLabels(309)) - Add labels: [] 2015-11-22 19:23:43,194 INFO [RM-0] nodelabels.CommonNodeLabelsManager (CommonNodeLabelsManager.java:internalUpdateLabelsOnNodes(638)) - REPLACE labels on nodes: 2015-11-22 19:23:43,195 INFO [RM-0] nodelabels.CommonNodeLabelsManager (CommonNodeLabelsManager.java:internalUpdateLabelsOnNodes(640)) - NM=ip-172-31-20-42.ap-northeast-1.compute.internal:58694, labels=[x] 2015-11-22 19:23:43,196 INFO [RM-0] nodelabels.FileSystemNodeLabelsStore (FileSystemNodeLabelsStore.java:recover(279)) - Finished write mirror at:file:/tmp/hadoop-yarn-ubuntu/node-labels/nodelabel.mirror 2015-11-22 19:23:43,196 INFO [RM-0] nodelabels.FileSystemNodeLabelsStore (FileSystemNodeLabelsStore.java:recover(280)) - Finished create editlog file at:file:/tmp/hadoop-yarn-ubuntu/node-labels/nodelabel.editlog 2015-11-22 19:23:43,197 INFO [RM-0] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.nodelabels.event.NodeLabelsStoreEventType for class org.apache.hadoop.yarn.nodelabels.CommonNodeLabelsManager$ForwardingEventHandler 2015-11-22 19:23:43,199 INFO [RM-0] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:23:43,200 INFO [Socket Reader #1 for port 49532] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 49532 2015-11-22 19:23:43,202 INFO [RM-0] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.server.api.ResourceTrackerPB to the server 2015-11-22 19:23:43,202 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:23:43,202 INFO [IPC Server listener on 49532] ipc.Server (Server.java:run(794)) - IPC Server listener on 49532: starting 2015-11-22 19:23:43,213 INFO [RM-0] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:23:43,214 INFO [Socket Reader #1 for port 50491] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 50491 2015-11-22 19:23:43,216 INFO [RM-0] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB to the server 2015-11-22 19:23:43,216 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:23:43,217 INFO [IPC Server listener on 50491] ipc.Server (Server.java:run(794)) - IPC Server listener on 50491: starting 2015-11-22 19:23:43,234 INFO [RM-0] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:23:43,234 INFO [Socket Reader #1 for port 35934] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 35934 2015-11-22 19:23:43,236 INFO [RM-0] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.api.ApplicationClientProtocolPB to the server 2015-11-22 19:23:43,237 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:23:43,237 INFO [IPC Server listener on 35934] ipc.Server (Server.java:run(794)) - IPC Server listener on 35934: starting 2015-11-22 19:23:43,243 INFO [RM-0] resourcemanager.ResourceManager (ResourceManager.java:transitionToActive(1040)) - Transitioned to active state 2015-11-22 19:23:43,245 INFO [RM-0] server.AuthenticationFilter (AuthenticationFilter.java:constructSecretProvider(304)) - Unable to initialize FileSignerSecretProvider, falling back to use random secrets. 2015-11-22 19:23:43,245 INFO [RM-0] http.HttpRequestLog (HttpRequestLog.java:getRequestLog(80)) - Http request log for http.requests.resourcemanager is not defined 2015-11-22 19:23:43,246 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter) 2015-11-22 19:23:43,247 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context cluster 2015-11-22 19:23:43,247 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context static 2015-11-22 19:23:43,247 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context logs 2015-11-22 19:23:43,247 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context cluster 2015-11-22 19:23:43,247 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2015-11-22 19:23:43,247 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2015-11-22 19:23:43,248 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'Timeline Authentication Filter' (class=org.apache.hadoop.yarn.server.timeline.security.TimelineAuthenticationFilter) 2015-11-22 19:23:43,248 INFO [RM-0] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /cluster/* 2015-11-22 19:23:43,249 INFO [RM-0] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /ws/* 2015-11-22 19:23:43,258 INFO [RM-0] webapp.WebApps (WebApps.java:build(288)) - Registered webapp guice modules 2015-11-22 19:23:43,259 INFO [RM-0] http.HttpServer2 (HttpServer2.java:openListeners(917)) - Jetty bound to port 52325 2015-11-22 19:23:43,259 INFO [RM-0] mortbay.log (Slf4jLog.java:info(67)) - jetty-6.1.26 2015-11-22 19:23:43,284 INFO [RM-0] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:23:43,284 INFO [Thread[Thread-840,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2015-11-22 19:23:43,286 INFO [RM-0] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:23:43,287 INFO [Thread[Thread-842,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2015-11-22 19:23:43,291 INFO [Thread[Thread-840,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens Nov 22, 2015 7:23:43 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.resourcemanager.webapp.JAXBContextResolver as a provider class Nov 22, 2015 7:23:43 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices as a root resource class Nov 22, 2015 7:23:43 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a provider class Nov 22, 2015 7:23:43 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate INFO: Initiating Jersey application, version 'Jersey: 1.9 09/02/2011 11:17 AM' Nov 22, 2015 7:23:43 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.resourcemanager.webapp.JAXBContextResolver to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:23:43 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.webapp.GenericExceptionHandler to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:23:43 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices to GuiceManagedComponentProvider with the scope "Singleton" 2015-11-22 19:23:43,532 INFO [RM-0] mortbay.log (Slf4jLog.java:info(67)) - Started HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:52325 2015-11-22 19:23:43,532 INFO [RM-0] webapp.WebApps (WebApps.java:start(307)) - Web app cluster started at 52325 2015-11-22 19:23:43,533 INFO [RM-0] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:23:43,533 INFO [Socket Reader #1 for port 40969] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 40969 2015-11-22 19:23:43,535 INFO [RM-0] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.server.api.ResourceManagerAdministrationProtocolPB to the server 2015-11-22 19:23:43,535 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:23:43,535 INFO [IPC Server listener on 40969] ipc.Server (Server.java:run(794)) - IPC Server listener on 40969: starting 2015-11-22 19:23:44,689 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:startResourceManager(339)) - MiniYARN ResourceManager address: ip-172-31-20-42.ap-northeast-1.compute.internal:35934 2015-11-22 19:23:44,689 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:startResourceManager(341)) - MiniYARN ResourceManager web address: ip-172-31-20-42.ap-northeast-1.compute.internal:52325 2015-11-22 19:23:44,689 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(459)) - MiniYARN ResourceManager address: ip-172-31-20-42.ap-northeast-1.compute.internal:35934 2015-11-22 19:23:44,689 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(461)) - MiniYARN ResourceManager web address: ip-172-31-20-42.ap-northeast-1.compute.internal:52325 2015-11-22 19:23:44,689 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(575)) - Waiting for NM 0 to start... 2015-11-22 19:23:44,692 INFO [Thread-849] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:23:44,694 INFO [Socket Reader #1 for port 39971] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 39971 2015-11-22 19:23:44,696 INFO [Thread-849] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.api.ContainerManagementProtocolPB to the server 2015-11-22 19:23:44,696 INFO [Thread-849] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:serviceStart(464)) - Blocking new container-requests as container manager rpc server is still starting. 2015-11-22 19:23:44,696 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:23:44,696 INFO [IPC Server listener on 39971] ipc.Server (Server.java:run(794)) - IPC Server listener on 39971: starting 2015-11-22 19:23:44,705 INFO [Thread-849] security.NMContainerTokenSecretManager (NMContainerTokenSecretManager.java:setNodeId(260)) - Updating node address : ip-172-31-20-42.ap-northeast-1.compute.internal:39971 2015-11-22 19:23:44,707 INFO [Thread-849] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:23:44,707 INFO [Socket Reader #1 for port 48479] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 48479 2015-11-22 19:23:44,708 INFO [Thread-849] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB to the server 2015-11-22 19:23:44,709 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:23:44,709 INFO [IPC Server listener on 48479] ipc.Server (Server.java:run(794)) - IPC Server listener on 48479: starting 2015-11-22 19:23:44,710 INFO [Thread-849] localizer.ResourceLocalizationService (ResourceLocalizationService.java:serviceStart(363)) - Localizer started on port 48479 2015-11-22 19:23:44,717 INFO [Thread-849] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:serviceStart(509)) - ContainerManager started at /172.31.20.42:39971 2015-11-22 19:23:44,717 INFO [Thread-849] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:serviceStart(510)) - ContainerManager bound to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:0 2015-11-22 19:23:44,717 INFO [Thread-849] webapp.WebServer (WebServer.java:serviceStart(71)) - Instantiating NMWebApp at ip-172-31-20-42.ap-northeast-1.compute.internal:0 2015-11-22 19:23:44,718 INFO [Thread-849] server.AuthenticationFilter (AuthenticationFilter.java:constructSecretProvider(304)) - Unable to initialize FileSignerSecretProvider, falling back to use random secrets. 2015-11-22 19:23:44,718 INFO [Thread-849] http.HttpRequestLog (HttpRequestLog.java:getRequestLog(80)) - Http request log for http.requests.nodemanager is not defined 2015-11-22 19:23:44,719 INFO [Thread-849] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter) 2015-11-22 19:23:44,720 INFO [Thread-849] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context node 2015-11-22 19:23:44,720 INFO [Thread-849] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context logs 2015-11-22 19:23:44,720 INFO [Thread-849] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context static 2015-11-22 19:23:44,720 INFO [Thread-849] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context node 2015-11-22 19:23:44,720 INFO [Thread-849] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2015-11-22 19:23:44,720 INFO [Thread-849] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2015-11-22 19:23:44,721 INFO [Thread-849] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'Timeline Authentication Filter' (class=org.apache.hadoop.yarn.server.timeline.security.TimelineAuthenticationFilter) 2015-11-22 19:23:44,722 INFO [Thread-849] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /node/* 2015-11-22 19:23:44,723 INFO [Thread-849] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /ws/* Nov 22, 2015 7:23:44 PM com.google.inject.servlet.GuiceFilter setPipeline WARNING: Multiple Servlet injectors detected. This is a warning indicating that you have more than one GuiceFilter running in your web application. If this is deliberate, you may safely ignore this message. If this is NOT deliberate however, your application may not work as expected. 2015-11-22 19:23:44,730 INFO [Thread-849] webapp.WebApps (WebApps.java:build(288)) - Registered webapp guice modules 2015-11-22 19:23:44,730 INFO [Thread-849] http.HttpServer2 (HttpServer2.java:openListeners(917)) - Jetty bound to port 41774 2015-11-22 19:23:44,731 INFO [Thread-849] mortbay.log (Slf4jLog.java:info(67)) - jetty-6.1.26 2015-11-22 19:23:44,757 INFO [Thread-849] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:23:44,757 INFO [Thread[Thread-890,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2015-11-22 19:23:44,758 INFO [Thread[Thread-890,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:23:44,759 INFO [Thread-849] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:23:44,760 INFO [Thread[Thread-892,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) Nov 22, 2015 7:23:44 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.nodemanager.webapp.NMWebServices as a root resource class Nov 22, 2015 7:23:44 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a provider class Nov 22, 2015 7:23:44 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.nodemanager.webapp.JAXBContextResolver as a provider class Nov 22, 2015 7:23:44 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate INFO: Initiating Jersey application, version 'Jersey: 1.9 09/02/2011 11:17 AM' Nov 22, 2015 7:23:44 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.nodemanager.webapp.JAXBContextResolver to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:23:44 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.webapp.GenericExceptionHandler to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:23:44 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.nodemanager.webapp.NMWebServices to GuiceManagedComponentProvider with the scope "Singleton" 2015-11-22 19:23:44,871 INFO [Thread-849] mortbay.log (Slf4jLog.java:info(67)) - Started HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:41774 2015-11-22 19:23:44,872 INFO [Thread-849] webapp.WebApps (WebApps.java:start(307)) - Web app node started at 41774 2015-11-22 19:23:44,872 INFO [Thread-849] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:serviceStart(222)) - Node ID assigned is : ip-172-31-20-42.ap-northeast-1.compute.internal:39971 2015-11-22 19:23:44,873 INFO [Thread-849] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNMContainerStatuses(550)) - Sending out 0 NM container statuses: [] 2015-11-22 19:23:44,873 INFO [Thread-849] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:registerWithRM(330)) - Registering with RM using containers :[] 2015-11-22 19:23:44,873 INFO [Thread-849] resourcemanager.ResourceTrackerService (ResourceTrackerService.java:registerNodeManager(375)) - NodeManager from node ip-172-31-20-42.ap-northeast-1.compute.internal(cmPort: 39971 httpPort: 41774) registered with capability: , assigned nodeId ip-172-31-20-42.ap-northeast-1.compute.internal:39971 2015-11-22 19:23:44,873 INFO [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(566)) - ip-172-31-20-42.ap-northeast-1.compute.internal:39971 Node Transitioned from NEW to RUNNING 2015-11-22 19:23:44,873 INFO [Thread-849] security.NMContainerTokenSecretManager (NMContainerTokenSecretManager.java:setMasterKey(138)) - Rolling master-key for container-tokens, got key with id -853791043 2015-11-22 19:23:44,874 INFO [Thread-849] security.NMTokenSecretManagerInNM (NMTokenSecretManagerInNM.java:setMasterKey(135)) - Rolling master-key for container-tokens, got key with id -840304764 2015-11-22 19:23:44,874 INFO [Thread-849] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:registerWithRM(391)) - Registered with ResourceManager as ip-172-31-20-42.ap-northeast-1.compute.internal:39971 with total resource of 2015-11-22 19:23:44,875 INFO [Thread-849] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:registerWithRM(392)) - Notifying ContainerManager to unblock new container-requests 2015-11-22 19:23:44,875 INFO [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:addNode(1427)) - Added node ip-172-31-20-42.ap-northeast-1.compute.internal:39971 clusterResource: 2015-11-22 19:23:45,690 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(728)) - Waiting for Timeline Server to start... 2015-11-22 19:23:45,690 INFO [Thread-895] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:23:45,690 INFO [Thread[Thread-897,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2015-11-22 19:23:45,692 INFO [Thread[Thread-897,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:23:45,692 INFO [Thread-895] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:23:45,693 INFO [Socket Reader #1 for port 47761] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 47761 2015-11-22 19:23:45,694 INFO [Thread-895] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.api.ApplicationHistoryProtocolPB to the server 2015-11-22 19:23:45,695 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:23:45,695 INFO [IPC Server listener on 47761] ipc.Server (Server.java:run(794)) - IPC Server listener on 47761: starting 2015-11-22 19:23:45,696 INFO [Thread-895] applicationhistoryservice.ApplicationHistoryClientService (ApplicationHistoryClientService.java:serviceStart(111)) - Instantiated ApplicationHistoryClientService at ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:47761 2015-11-22 19:23:45,696 INFO [Thread-895] applicationhistoryservice.ApplicationHistoryManagerImpl (ApplicationHistoryManagerImpl.java:serviceStart(71)) - Starting ApplicationHistory 2015-11-22 19:23:45,698 INFO [Thread-895] server.AuthenticationFilter (AuthenticationFilter.java:constructSecretProvider(304)) - Unable to initialize FileSignerSecretProvider, falling back to use random secrets. 2015-11-22 19:23:45,698 INFO [Thread-895] http.HttpRequestLog (HttpRequestLog.java:getRequestLog(80)) - Http request log for http.requests.applicationhistory is not defined 2015-11-22 19:23:45,698 INFO [Thread-895] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter) 2015-11-22 19:23:45,699 INFO [Thread-895] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context applicationhistory 2015-11-22 19:23:45,699 INFO [Thread-895] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context static 2015-11-22 19:23:45,699 INFO [Thread-895] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context logs 2015-11-22 19:23:45,699 INFO [Thread-895] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context applicationhistory 2015-11-22 19:23:45,699 INFO [Thread-895] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2015-11-22 19:23:45,699 INFO [Thread-895] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2015-11-22 19:23:45,700 INFO [Thread-895] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'Timeline Authentication Filter' (class=org.apache.hadoop.yarn.server.timeline.security.TimelineAuthenticationFilter) 2015-11-22 19:23:45,701 INFO [Thread-895] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /applicationhistory/* 2015-11-22 19:23:45,701 INFO [Thread-895] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /ws/* Nov 22, 2015 7:23:45 PM com.google.inject.servlet.GuiceFilter setPipeline WARNING: Multiple Servlet injectors detected. This is a warning indicating that you have more than one GuiceFilter running in your web application. If this is deliberate, you may safely ignore this message. If this is NOT deliberate however, your application may not work as expected. 2015-11-22 19:23:45,707 INFO [Thread-895] webapp.WebApps (WebApps.java:build(288)) - Registered webapp guice modules 2015-11-22 19:23:45,707 INFO [Thread-895] http.HttpServer2 (HttpServer2.java:openListeners(917)) - Jetty bound to port 54125 2015-11-22 19:23:45,707 INFO [Thread-895] mortbay.log (Slf4jLog.java:info(67)) - jetty-6.1.26 2015-11-22 19:23:45,719 INFO [Thread-895] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:23:45,719 INFO [Thread[Thread-915,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2015-11-22 19:23:45,720 INFO [Thread[Thread-915,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:23:45,721 INFO [Thread-895] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:23:45,721 INFO [Thread[Thread-917,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) Nov 22, 2015 7:23:45 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider as a provider class Nov 22, 2015 7:23:45 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.AHSWebServices as a root resource class Nov 22, 2015 7:23:45 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.timeline.webapp.TimelineWebServices as a root resource class Nov 22, 2015 7:23:45 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a provider class Nov 22, 2015 7:23:45 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate INFO: Initiating Jersey application, version 'Jersey: 1.9 09/02/2011 11:17 AM' Nov 22, 2015 7:23:45 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.webapp.GenericExceptionHandler to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:23:45 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:23:45 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.AHSWebServices to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:23:45 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.timeline.webapp.TimelineWebServices to GuiceManagedComponentProvider with the scope "Singleton" 2015-11-22 19:23:45,809 INFO [Thread-895] mortbay.log (Slf4jLog.java:info(67)) - Started HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:54125 2015-11-22 19:23:45,810 INFO [Thread-895] applicationhistoryservice.ApplicationHistoryServer (ApplicationHistoryServer.java:startWebApp(342)) - Instantiating AHSWebApp at 54125 2015-11-22 19:23:47,190 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(739)) - MiniYARN ApplicationHistoryServer address: ip-172-31-20-42.ap-northeast-1.compute.internal:47761 2015-11-22 19:23:47,190 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(741)) - MiniYARN ApplicationHistoryServer web address: ip-172-31-20-42.ap-northeast-1.compute.internal:54125 2015-11-22 19:23:49,224 INFO [Thread-919] distributedshell.TestDistributedShell (TestDistributedShell.java:testDSAttemptFailuresValidityIntervalFailed(434)) - Initializing DS Client 2015-11-22 19:23:49,310 INFO [Thread-919] impl.TimelineClientImpl (TimelineClientImpl.java:serviceInit(296)) - Timeline service address: http://ip-172-31-20-42.ap-northeast-1.compute.internal:54125/ws/v1/timeline/ 2015-11-22 19:23:49,311 INFO [Thread-919] distributedshell.TestDistributedShell (TestDistributedShell.java:testDSAttemptFailuresValidityIntervalFailed(441)) - Running DS Client 2015-11-22 19:23:49,311 INFO [Thread-919] distributedshell.Client (Client.java:run(443)) - Running Client 2015-11-22 19:23:49,326 INFO [Thread-919] client.RMProxy (RMProxy.java:createRMProxy(98)) - Connecting to ResourceManager at ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:35934 2015-11-22 19:23:49,331 INFO [Thread-919] distributedshell.Client (Client.java:run(447)) - Got Cluster metric info from ASM, numNodeManagers=1 2015-11-22 19:23:49,333 INFO [Thread-919] distributedshell.Client (Client.java:run(452)) - Got Cluster node info from ASM 2015-11-22 19:23:49,333 INFO [Thread-919] distributedshell.Client (Client.java:run(454)) - Got node report from ASM for, nodeId=ip-172-31-20-42.ap-northeast-1.compute.internal:39971, nodeAddress=ip-172-31-20-42.ap-northeast-1.compute.internal:41774, nodeRackName=/default-rack, nodeNumContainers=0 2015-11-22 19:23:49,335 INFO [Thread-919] distributedshell.Client (Client.java:run(462)) - Queue info, queueName=default, queueCurrentCapacity=0.0, queueMaxCapacity=1.0, queueApplicationCount=0, queueChildQueueCount=0 2015-11-22 19:23:49,335 INFO [Thread-919] distributedshell.Client (Client.java:run(472)) - User ACL Info for Queue, queueName=root, userAcl=SUBMIT_APPLICATIONS 2015-11-22 19:23:49,336 INFO [Thread-919] distributedshell.Client (Client.java:run(472)) - User ACL Info for Queue, queueName=root, userAcl=ADMINISTER_QUEUE 2015-11-22 19:23:49,336 INFO [Thread-919] distributedshell.Client (Client.java:run(472)) - User ACL Info for Queue, queueName=default, userAcl=SUBMIT_APPLICATIONS 2015-11-22 19:23:49,336 INFO [Thread-919] distributedshell.Client (Client.java:run(472)) - User ACL Info for Queue, queueName=default, userAcl=ADMINISTER_QUEUE 2015-11-22 19:23:49,336 INFO [IPC Server handler 4 on 35934] resourcemanager.ClientRMService (ClientRMService.java:getNewApplicationId(305)) - Allocated new applicationId: 1 2015-11-22 19:23:49,337 INFO [Thread-919] distributedshell.Client (Client.java:run(491)) - Max mem capability of resources in this cluster 4096 2015-11-22 19:23:49,337 INFO [Thread-919] distributedshell.Client (Client.java:run(502)) - Max virtual cores capability of resources in this cluster 4 2015-11-22 19:23:49,337 INFO [Thread-919] distributedshell.Client (Client.java:run(528)) - Copy App Master jar from local filesystem and add to local environment 2015-11-22 19:23:49,346 INFO [Thread-919] distributedshell.Client (Client.java:run(577)) - Set the environment for the application master 2015-11-22 19:23:49,346 INFO [Thread-919] distributedshell.Client (Client.java:run(619)) - Setting up app master command 2015-11-22 19:23:49,346 INFO [Thread-919] distributedshell.Client (Client.java:run(650)) - Completed setting up app master command {{JAVA_HOME}}/bin/java -Xmx512m org.apache.hadoop.yarn.applications.distributedshell.TestDSSleepingAppMaster --container_memory 128 --container_vcores 1 --num_containers 1 --priority 0 1>/AppMaster.stdout 2>/AppMaster.stderr 2015-11-22 19:23:49,346 INFO [Thread-919] distributedshell.Client (Client.java:run(706)) - Submitting application to ASM 2015-11-22 19:23:49,348 INFO [IPC Server handler 5 on 35934] capacity.CapacityScheduler (CapacityScheduler.java:checkAndGetApplicationPriority(1941)) - Priority '0' is acceptable in queue : default for application: application_1448220223188_0001 for the user: ubuntu 2015-11-22 19:23:49,348 WARN [IPC Server handler 5 on 35934] rmapp.RMAppImpl (RMAppImpl.java:(420)) - The specific max attempts: 0 for application: 1 is invalid, because it is out of the range [1, 2]. Use the global max attempts instead. 2015-11-22 19:23:49,348 INFO [IPC Server handler 5 on 35934] rmapp.RMAppImpl (RMAppImpl.java:(431)) - The attemptFailuresValidityInterval for the application: application_1448220223188_0001 is 15000. 2015-11-22 19:23:49,348 INFO [IPC Server handler 5 on 35934] resourcemanager.ClientRMService (ClientRMService.java:submitApplication(602)) - Application with id 1 submitted by user ubuntu 2015-11-22 19:23:49,348 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:transition(1079)) - Storing application with id application_1448220223188_0001 2015-11-22 19:23:49,348 INFO [IPC Server handler 5 on 35934] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(154)) - USER=ubuntu IP=172.31.20.42 OPERATION=Submit Application Request TARGET=ClientRMService RESULT=SUCCESS APPID=application_1448220223188_0001 2015-11-22 19:23:49,348 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220223188_0001 State change from NEW to NEW_SAVING 2015-11-22 19:23:49,349 INFO [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:transition(208)) - Storing info for app: application_1448220223188_0001 2015-11-22 19:23:49,349 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220223188_0001 State change from NEW_SAVING to SUBMITTED 2015-11-22 19:23:49,350 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:addApplication(349)) - Application added - appId: application_1448220223188_0001 user: ubuntu leaf-queue of parent: root #applications: 1 2015-11-22 19:23:49,350 INFO [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:addApplication(765)) - Accepted application application_1448220223188_0001 from user: ubuntu, in queue: default 2015-11-22 19:23:49,350 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220223188_0001 State change from SUBMITTED to ACCEPTED 2015-11-22 19:23:49,351 INFO [AsyncDispatcher event handler] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:registerAppAttempt(661)) - Registering app attempt : appattempt_1448220223188_0001_000001 2015-11-22 19:23:49,351 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220223188_0001_000001 State change from NEW to SUBMITTED 2015-11-22 19:23:49,351 INFO [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:activateApplications(705)) - Application application_1448220223188_0001 from user: ubuntu activated in queue: default 2015-11-22 19:23:49,351 INFO [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:addApplicationAttempt(720)) - Application added - appId: application_1448220223188_0001 user: ubuntu, leaf-queue: default #user-pending-applications: 0 #user-active-applications: 1 #queue-pending-applications: 0 #queue-active-applications: 1 2015-11-22 19:23:49,351 INFO [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:addApplicationAttempt(805)) - Added Application Attempt appattempt_1448220223188_0001_000001 to scheduler from user ubuntu in queue default 2015-11-22 19:23:49,353 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220223188_0001_000001 State change from SUBMITTED to SCHEDULED 2015-11-22 19:23:49,554 INFO [Thread-919] impl.YarnClientImpl (YarnClientImpl.java:submitApplication(279)) - Submitted application application_1448220223188_0001 2015-11-22 19:23:49,880 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220223188_0001_01_000001 Container Transitioned from NEW to ALLOCATED 2015-11-22 19:23:49,880 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220223188_0001 CONTAINERID=container_1448220223188_0001_01_000001 2015-11-22 19:23:49,880 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(154)) - Assigned container container_1448220223188_0001_01_000001 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:39971, which has 1 containers, used and available after allocation 2015-11-22 19:23:49,881 INFO [ResourceManager Event Processor] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(89)) - assignedContainer application attempt=appattempt_1448220223188_0001_000001 container=container_1448220223188_0001_01_000001 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@1cf3c9e8 clusterResource= type=OFF_SWITCH 2015-11-22 19:23:49,881 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(589)) - Re-sorting assigned queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.125, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 2015-11-22 19:23:49,881 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(461)) - assignedContainer queue=root usedCapacity=0.125 absoluteUsedCapacity=0.125 used= cluster= 2015-11-22 19:23:49,883 INFO [AsyncDispatcher event handler] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:createAndGetNMToken(200)) - Sending NMToken for nodeId : ip-172-31-20-42.ap-northeast-1.compute.internal:39971 for container : container_1448220223188_0001_01_000001 2015-11-22 19:23:49,885 INFO [AsyncDispatcher event handler] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220223188_0001_01_000001 Container Transitioned from ALLOCATED to ACQUIRED 2015-11-22 19:23:49,885 INFO [AsyncDispatcher event handler] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:clearNodeSetForAttempt(146)) - Clear node set for appattempt_1448220223188_0001_000001 2015-11-22 19:23:49,885 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:storeAttempt(2026)) - Storing attempt: AppId: application_1448220223188_0001 AttemptId: appattempt_1448220223188_0001_000001 MasterContainer: Container: [ContainerId: container_1448220223188_0001_01_000001, NodeId: ip-172-31-20-42.ap-northeast-1.compute.internal:39971, NodeHttpAddress: ip-172-31-20-42.ap-northeast-1.compute.internal:41774, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.31.20.42:39971 }, ] 2015-11-22 19:23:49,885 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220223188_0001_000001 State change from SCHEDULED to ALLOCATED_SAVING 2015-11-22 19:23:49,886 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220223188_0001_000001 State change from ALLOCATED_SAVING to ALLOCATED 2015-11-22 19:23:49,886 INFO [ApplicationMasterLauncher #0] amlauncher.AMLauncher (AMLauncher.java:run(254)) - Launching masterappattempt_1448220223188_0001_000001 2015-11-22 19:23:49,891 INFO [ApplicationMasterLauncher #0] amlauncher.AMLauncher (AMLauncher.java:launch(106)) - Setting up container Container: [ContainerId: container_1448220223188_0001_01_000001, NodeId: ip-172-31-20-42.ap-northeast-1.compute.internal:39971, NodeHttpAddress: ip-172-31-20-42.ap-northeast-1.compute.internal:41774, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.31.20.42:39971 }, ] for AM appattempt_1448220223188_0001_000001 2015-11-22 19:23:49,891 INFO [ApplicationMasterLauncher #0] amlauncher.AMLauncher (AMLauncher.java:createAMContainerLaunchContext(188)) - Command to launch container container_1448220223188_0001_01_000001 : {{JAVA_HOME}}/bin/java -Xmx512m org.apache.hadoop.yarn.applications.distributedshell.TestDSSleepingAppMaster --container_memory 128 --container_vcores 1 --num_containers 1 --priority 0 1>/AppMaster.stdout 2>/AppMaster.stderr 2015-11-22 19:23:49,892 INFO [ApplicationMasterLauncher #0] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:createAndGetAMRMToken(195)) - Create AMRMToken for ApplicationAttempt: appattempt_1448220223188_0001_000001 2015-11-22 19:23:49,892 INFO [ApplicationMasterLauncher #0] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:createPassword(307)) - Creating password for appattempt_1448220223188_0001_000001 2015-11-22 19:23:49,897 INFO [Socket Reader #1 for port 39971] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220223188_0001_000001 (auth:SIMPLE) 2015-11-22 19:23:49,904 INFO [IPC Server handler 0 on 39971] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:startContainerInternal(889)) - Start request for container_1448220223188_0001_01_000001 by user ubuntu 2015-11-22 19:23:49,905 INFO [IPC Server handler 0 on 39971] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:startContainerInternal(930)) - Creating a new application reference for app application_1448220223188_0001 2015-11-22 19:23:49,905 INFO [IPC Server handler 0 on 39971] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu IP=172.31.20.42 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1448220223188_0001 CONTAINERID=container_1448220223188_0001_01_000001 2015-11-22 19:23:49,905 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(463)) - Application application_1448220223188_0001 transitioned from NEW to INITING 2015-11-22 19:23:49,906 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(303)) - Adding container_1448220223188_0001_01_000001 to application application_1448220223188_0001 2015-11-22 19:23:49,907 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(463)) - Application application_1448220223188_0001 transitioned from INITING to RUNNING 2015-11-22 19:23:49,907 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220223188_0001_01_000001 transitioned from NEW to LOCALIZING 2015-11-22 19:23:49,907 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_INIT for appId application_1448220223188_0001 2015-11-22 19:23:49,908 INFO [AsyncDispatcher event handler] localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource file:/home/ubuntu/DistributedShell/application_1448220223188_0001/AppMaster.jar transitioned from INIT to DOWNLOADING 2015-11-22 19:23:49,909 INFO [AsyncDispatcher event handler] localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource file:/home/ubuntu/DistributedShell/application_1448220223188_0001/shellCommands transitioned from INIT to DOWNLOADING 2015-11-22 19:23:49,909 INFO [AsyncDispatcher event handler] localizer.ResourceLocalizationService (ResourceLocalizationService.java:handle(742)) - Created localizer for container_1448220223188_0001_01_000001 2015-11-22 19:23:49,917 INFO [ApplicationMasterLauncher #0] amlauncher.AMLauncher (AMLauncher.java:launch(127)) - Done launching container Container: [ContainerId: container_1448220223188_0001_01_000001, NodeId: ip-172-31-20-42.ap-northeast-1.compute.internal:39971, NodeHttpAddress: ip-172-31-20-42.ap-northeast-1.compute.internal:41774, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.31.20.42:39971 }, ] for AM appattempt_1448220223188_0001_000001 2015-11-22 19:23:49,918 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220223188_0001_000001 State change from ALLOCATED to LAUNCHED 2015-11-22 19:23:49,921 INFO [LocalizerRunner for container_1448220223188_0001_01_000001] localizer.ResourceLocalizationService (ResourceLocalizationService.java:writeCredentials(1221)) - Writing credentials to the nmPrivate file /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/container_1448220223188_0001_01_000001.tokens. Credentials list: 2015-11-22 19:23:49,923 INFO [LocalizerRunner for container_1448220223188_0001_01_000001] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:createUserCacheDirs(632)) - Initializing user ubuntu 2015-11-22 19:23:49,930 INFO [LocalizerRunner for container_1448220223188_0001_01_000001] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:startLocalizer(126)) - Copying from /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/container_1448220223188_0001_01_000001.tokens to /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220223188_0001/container_1448220223188_0001_01_000001.tokens 2015-11-22 19:23:49,931 INFO [LocalizerRunner for container_1448220223188_0001_01_000001] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:startLocalizer(133)) - Localizer CWD set to /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220223188_0001 = file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220223188_0001 2015-11-22 19:23:49,970 INFO [IPC Server handler 1 on 48479] localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource file:/home/ubuntu/DistributedShell/application_1448220223188_0001/AppMaster.jar(->/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220223188_0001/filecache/10/AppMaster.jar) transitioned from DOWNLOADING to LOCALIZED 2015-11-22 19:23:49,987 INFO [IPC Server handler 2 on 48479] localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource file:/home/ubuntu/DistributedShell/application_1448220223188_0001/shellCommands(->/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220223188_0001/filecache/11/shellCommands) transitioned from DOWNLOADING to LOCALIZED 2015-11-22 19:23:49,988 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220223188_0001_01_000001 transitioned from LOCALIZING to LOCALIZED 2015-11-22 19:23:50,008 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220223188_0001_01_000001 transitioned from LOCALIZED to RUNNING 2015-11-22 19:23:50,026 INFO [ContainersLauncher #0] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:buildCommandExecutor(281)) - launchContainer: [nice, -n, 0, bash, /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220223188_0001/container_1448220223188_0001_01_000001/default_container_executor.sh] 2015-11-22 19:23:50,556 INFO [Thread-919] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=N/A, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220229348, yarnAppState=ACCEPTED, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:52325/proxy/application_1448220223188_0001/, appUser=ubuntu 2015-11-22 19:23:50,880 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220223188_0001_01_000001 Container Transitioned from ACQUIRED to RUNNING 2015-11-22 19:23:51,561 INFO [Thread-919] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=N/A, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220229348, yarnAppState=ACCEPTED, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:52325/proxy/application_1448220223188_0001/, appUser=ubuntu 2015-11-22 19:23:52,563 INFO [Thread-919] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=N/A, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220229348, yarnAppState=ACCEPTED, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:52325/proxy/application_1448220223188_0001/, appUser=ubuntu 2015-11-22 19:23:52,565 INFO [Socket Reader #1 for port 50491] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220223188_0001_000001 (auth:SIMPLE) 2015-11-22 19:23:52,573 INFO [IPC Server handler 1 on 50491] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:registerApplicationMaster(220)) - AM registration appattempt_1448220223188_0001_000001 2015-11-22 19:23:52,574 INFO [IPC Server handler 1 on 50491] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(133)) - USER=ubuntu IP=172.31.20.42 OPERATION=Register App Master TARGET=ApplicationMasterService RESULT=SUCCESS APPID=application_1448220223188_0001 APPATTEMPTID=appattempt_1448220223188_0001_000001 2015-11-22 19:23:52,574 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220223188_0001_000001 State change from LAUNCHED to RUNNING 2015-11-22 19:23:52,574 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220223188_0001 State change from ACCEPTED to RUNNING 2015-11-22 19:23:53,566 INFO [Thread-919] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220229348, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:52325/proxy/application_1448220223188_0001/, appUser=ubuntu 2015-11-22 19:23:53,882 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220223188_0001_01_000002 Container Transitioned from NEW to ALLOCATED 2015-11-22 19:23:53,882 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220223188_0001 CONTAINERID=container_1448220223188_0001_01_000002 2015-11-22 19:23:53,882 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(154)) - Assigned container container_1448220223188_0001_01_000002 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:39971, which has 2 containers, used and available after allocation 2015-11-22 19:23:53,882 INFO [ResourceManager Event Processor] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(89)) - assignedContainer application attempt=appattempt_1448220223188_0001_000001 container=container_1448220223188_0001_01_000002 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@1cf3c9e8 clusterResource= type=OFF_SWITCH 2015-11-22 19:23:53,883 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(589)) - Re-sorting assigned queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.15625, absoluteUsedCapacity=0.15625, numApps=1, numContainers=2 2015-11-22 19:23:53,883 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(461)) - assignedContainer queue=root usedCapacity=0.15625 absoluteUsedCapacity=0.15625 used= cluster= 2015-11-22 19:23:54,568 INFO [Thread-919] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220229348, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:52325/proxy/application_1448220223188_0001/, appUser=ubuntu 2015-11-22 19:23:54,625 INFO [IPC Server handler 2 on 50491] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:createAndGetNMToken(200)) - Sending NMToken for nodeId : ip-172-31-20-42.ap-northeast-1.compute.internal:39971 for container : container_1448220223188_0001_01_000002 2015-11-22 19:23:54,626 INFO [IPC Server handler 2 on 50491] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220223188_0001_01_000002 Container Transitioned from ALLOCATED to ACQUIRED 2015-11-22 19:23:54,696 INFO [Socket Reader #1 for port 39971] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220223188_0001_000001 (auth:SIMPLE) 2015-11-22 19:23:54,701 INFO [IPC Server handler 1 on 39971] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:startContainerInternal(889)) - Start request for container_1448220223188_0001_01_000002 by user ubuntu 2015-11-22 19:23:54,702 INFO [IPC Server handler 1 on 39971] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu IP=172.31.20.42 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1448220223188_0001 CONTAINERID=container_1448220223188_0001_01_000002 2015-11-22 19:23:54,702 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(303)) - Adding container_1448220223188_0001_01_000002 to application application_1448220223188_0001 2015-11-22 19:23:54,702 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220223188_0001_01_000002 transitioned from NEW to LOCALIZED 2015-11-22 19:23:54,702 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_INIT for appId application_1448220223188_0001 2015-11-22 19:23:54,739 INFO [Socket Reader #1 for port 39971] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220223188_0001_000001 (auth:SIMPLE) 2015-11-22 19:23:54,749 INFO [IPC Server handler 0 on 39971] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:getContainerStatusInternal(1221)) - Getting container-status for container_1448220223188_0001_01_000002 2015-11-22 19:23:54,750 INFO [IPC Server handler 0 on 39971] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:getContainerStatusInternal(1235)) - Returning ContainerStatus: [ContainerId: container_1448220223188_0001_01_000002, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ] 2015-11-22 19:23:54,765 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220223188_0001_01_000002 transitioned from LOCALIZED to RUNNING 2015-11-22 19:23:54,843 INFO [ContainersLauncher #1] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:buildCommandExecutor(281)) - launchContainer: [nice, -n, 0, bash, /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220223188_0001/container_1448220223188_0001_01_000002/default_container_executor.sh] 2015-11-22 19:23:54,882 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220223188_0001_01_000002 Container Transitioned from ACQUIRED to RUNNING 2015-11-22 19:23:55,570 INFO [Thread-919] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220229348, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:52325/proxy/application_1448220223188_0001/, appUser=ubuntu 2015-11-22 19:23:56,571 INFO [Thread-919] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220229348, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:52325/proxy/application_1448220223188_0001/, appUser=ubuntu 2015-11-22 19:23:57,573 INFO [Thread-919] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220229348, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:52325/proxy/application_1448220223188_0001/, appUser=ubuntu 2015-11-22 19:23:57,614 WARN [ContainersLauncher #0] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(237)) - Exit code from container container_1448220223188_0001_01_000001 is : 100 2015-11-22 19:23:57,615 WARN [ContainersLauncher #0] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(243)) - Exception from container-launch with container ID: container_1448220223188_0001_01_000001 and exit code: 100 ExitCodeException exitCode=100: at org.apache.hadoop.util.Shell.runCommand(Shell.java:909) at org.apache.hadoop.util.Shell.run(Shell.java:820) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1099) at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:225) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:304) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:84) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 2015-11-22 19:23:57,617 INFO [ContainersLauncher #0] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - Exception from container-launch. 2015-11-22 19:23:57,617 INFO [ContainersLauncher #0] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - Container id: container_1448220223188_0001_01_000001 2015-11-22 19:23:57,617 INFO [ContainersLauncher #0] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - Exit code: 100 2015-11-22 19:23:57,617 INFO [ContainersLauncher #0] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - Stack trace: ExitCodeException exitCode=100: 2015-11-22 19:23:57,617 INFO [ContainersLauncher #0] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - at org.apache.hadoop.util.Shell.runCommand(Shell.java:909) 2015-11-22 19:23:57,618 INFO [ContainersLauncher #0] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - at org.apache.hadoop.util.Shell.run(Shell.java:820) 2015-11-22 19:23:57,618 INFO [ContainersLauncher #0] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1099) 2015-11-22 19:23:57,618 INFO [ContainersLauncher #0] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:225) 2015-11-22 19:23:57,618 INFO [ContainersLauncher #0] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:304) 2015-11-22 19:23:57,618 INFO [ContainersLauncher #0] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:84) 2015-11-22 19:23:57,618 INFO [ContainersLauncher #0] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - at java.util.concurrent.FutureTask.run(FutureTask.java:266) 2015-11-22 19:23:57,618 INFO [ContainersLauncher #0] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 2015-11-22 19:23:57,618 INFO [ContainersLauncher #0] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 2015-11-22 19:23:57,618 INFO [ContainersLauncher #0] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - at java.lang.Thread.run(Thread.java:745) 2015-11-22 19:23:57,619 WARN [ContainersLauncher #0] launcher.ContainerLaunch (ContainerLaunch.java:call(348)) - Container exited with a non-zero exit code 100 2015-11-22 19:23:57,619 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220223188_0001_01_000001 transitioned from RUNNING to EXITED_WITH_FAILURE 2015-11-22 19:23:57,619 INFO [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(380)) - Cleaning up container container_1448220223188_0001_01_000001 2015-11-22 19:23:57,636 INFO [DeletionService #0] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(480)) - Deleting absolute path : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220223188_0001/container_1448220223188_0001_01_000001 2015-11-22 19:23:57,636 WARN [AsyncDispatcher event handler] nodemanager.NMAuditLogger (NMAuditLogger.java:logFailure(150)) - USER=ubuntu OPERATION=Container Finished - Failed TARGET=ContainerImpl RESULT=FAILURE DESCRIPTION=Container failed with state: EXITED_WITH_FAILURE APPID=application_1448220223188_0001 CONTAINERID=container_1448220223188_0001_01_000001 2015-11-22 19:23:57,637 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220223188_0001_01_000001 transitioned from EXITED_WITH_FAILURE to DONE 2015-11-22 19:23:57,637 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(346)) - Removing container_1448220223188_0001_01_000001 from application application_1448220223188_0001 2015-11-22 19:23:57,637 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_STOP for appId application_1448220223188_0001 2015-11-22 19:23:57,883 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220223188_0001_01_000001 Container Transitioned from RUNNING to COMPLETED 2015-11-22 19:23:57,884 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220223188_0001 CONTAINERID=container_1448220223188_0001_01_000001 2015-11-22 19:23:57,884 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:releaseContainer(248)) - Released container container_1448220223188_0001_01_000001 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:39971, which currently has 1 containers, used and available, release resources=true 2015-11-22 19:23:57,884 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:rememberTargetTransitionsAndStoreState(1258)) - Updating application attempt appattempt_1448220223188_0001_000001 with final state: FAILED, and exit status: 100 2015-11-22 19:23:57,884 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220223188_0001_000001 State change from RUNNING to FINAL_SAVING 2015-11-22 19:23:57,884 INFO [AsyncDispatcher event handler] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:unregisterAttempt(667)) - Unregistering app attempt : appattempt_1448220223188_0001_000001 2015-11-22 19:23:57,886 INFO [AsyncDispatcher event handler] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:applicationMasterFinished(124)) - Application finished, removing password for appattempt_1448220223188_0001_000001 2015-11-22 19:23:57,886 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220223188_0001_000001 State change from FINAL_SAVING to FAILED 2015-11-22 19:23:57,886 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:transition(1302)) - The number of failed attempts in previous 15000 milliseconds is 1. The max attempts is 2 2015-11-22 19:23:57,887 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220223188_0001 State change from RUNNING to ACCEPTED 2015-11-22 19:23:57,887 INFO [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:doneApplicationAttempt(844)) - Application Attempt appattempt_1448220223188_0001_000001 is done. finalState=FAILED 2015-11-22 19:23:57,887 INFO [AsyncDispatcher event handler] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:registerAppAttempt(661)) - Registering app attempt : appattempt_1448220223188_0001_000002 2015-11-22 19:23:57,887 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220223188_0001_000002 State change from NEW to SUBMITTED 2015-11-22 19:23:57,888 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220223188_0001_01_000002 Container Transitioned from RUNNING to KILLED 2015-11-22 19:23:57,888 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220223188_0001 CONTAINERID=container_1448220223188_0001_01_000002 2015-11-22 19:23:57,888 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:releaseContainer(248)) - Released container container_1448220223188_0001_01_000002 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:39971, which currently has 0 containers, used and available, release resources=true 2015-11-22 19:23:57,888 INFO [ApplicationMasterLauncher #1] amlauncher.AMLauncher (AMLauncher.java:run(268)) - Cleaning master appattempt_1448220223188_0001_000001 2015-11-22 19:23:57,888 INFO [ResourceManager Event Processor] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:clearRequests(125)) - Application application_1448220223188_0001 requests cleared 2015-11-22 19:23:57,889 INFO [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:removeApplicationAttempt(772)) - Application removed - appId: application_1448220223188_0001 user: ubuntu queue: default #user-pending-applications: 0 #user-active-applications: 0 #queue-pending-applications: 0 #queue-active-applications: 0 2015-11-22 19:23:57,889 INFO [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:activateApplications(705)) - Application application_1448220223188_0001 from user: ubuntu activated in queue: default 2015-11-22 19:23:57,889 INFO [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:addApplicationAttempt(720)) - Application added - appId: application_1448220223188_0001 user: ubuntu, leaf-queue: default #user-pending-applications: 0 #user-active-applications: 1 #queue-pending-applications: 0 #queue-active-applications: 1 2015-11-22 19:23:57,889 INFO [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:addApplicationAttempt(805)) - Added Application Attempt appattempt_1448220223188_0001_000002 to scheduler from user ubuntu in queue default 2015-11-22 19:23:57,890 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220223188_0001_000002 State change from SUBMITTED to SCHEDULED 2015-11-22 19:23:57,893 INFO [Socket Reader #1 for port 39971] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220223188_0001_000001 (auth:SIMPLE) 2015-11-22 19:23:57,897 INFO [IPC Server handler 2 on 39971] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:stopContainerInternal(1163)) - Stopping container with container Id: container_1448220223188_0001_01_000001 2015-11-22 19:23:57,897 INFO [IPC Server handler 2 on 39971] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu IP=172.31.20.42 OPERATION=Stop Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1448220223188_0001 CONTAINERID=container_1448220223188_0001_01_000001 2015-11-22 19:23:57,897 INFO [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:removeOrTrackCompletedContainersFromContext(605)) - Removed completed containers from NM context: [container_1448220223188_0001_01_000001] 2015-11-22 19:23:57,898 WARN [AsyncDispatcher event handler] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:handle(1283)) - Event EventType: KILL_CONTAINER sent to absent container container_1448220223188_0001_01_000001 2015-11-22 19:23:57,899 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220223188_0001_01_000002 transitioned from RUNNING to KILLING 2015-11-22 19:23:57,899 INFO [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(380)) - Cleaning up container container_1448220223188_0001_01_000002 2015-11-22 19:23:57,900 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220223188_0001_02_000001 Container Transitioned from NEW to ALLOCATED 2015-11-22 19:23:57,901 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220223188_0001 CONTAINERID=container_1448220223188_0001_02_000001 2015-11-22 19:23:57,901 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(154)) - Assigned container container_1448220223188_0001_02_000001 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:39971, which has 1 containers, used and available after allocation 2015-11-22 19:23:57,901 INFO [ResourceManager Event Processor] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(89)) - assignedContainer application attempt=appattempt_1448220223188_0001_000002 container=container_1448220223188_0001_02_000001 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@12343406 clusterResource= type=OFF_SWITCH 2015-11-22 19:23:57,910 INFO [AsyncDispatcher event handler] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:createAndGetNMToken(200)) - Sending NMToken for nodeId : ip-172-31-20-42.ap-northeast-1.compute.internal:39971 for container : container_1448220223188_0001_02_000001 2015-11-22 19:23:57,911 INFO [AsyncDispatcher event handler] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220223188_0001_02_000001 Container Transitioned from ALLOCATED to ACQUIRED 2015-11-22 19:23:57,912 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(589)) - Re-sorting assigned queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.125, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 2015-11-22 19:23:57,912 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(461)) - assignedContainer queue=root usedCapacity=0.125 absoluteUsedCapacity=0.125 used= cluster= 2015-11-22 19:23:57,912 INFO [AsyncDispatcher event handler] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:clearNodeSetForAttempt(146)) - Clear node set for appattempt_1448220223188_0001_000002 2015-11-22 19:23:57,913 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:storeAttempt(2026)) - Storing attempt: AppId: application_1448220223188_0001 AttemptId: appattempt_1448220223188_0001_000002 MasterContainer: Container: [ContainerId: container_1448220223188_0001_02_000001, NodeId: ip-172-31-20-42.ap-northeast-1.compute.internal:39971, NodeHttpAddress: ip-172-31-20-42.ap-northeast-1.compute.internal:41774, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.31.20.42:39971 }, ] 2015-11-22 19:23:57,914 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220223188_0001_000002 State change from SCHEDULED to ALLOCATED_SAVING 2015-11-22 19:23:57,918 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220223188_0001_000002 State change from ALLOCATED_SAVING to ALLOCATED 2015-11-22 19:23:57,918 INFO [ApplicationMasterLauncher #2] amlauncher.AMLauncher (AMLauncher.java:run(254)) - Launching masterappattempt_1448220223188_0001_000002 2015-11-22 19:23:57,923 INFO [ApplicationMasterLauncher #2] amlauncher.AMLauncher (AMLauncher.java:launch(106)) - Setting up container Container: [ContainerId: container_1448220223188_0001_02_000001, NodeId: ip-172-31-20-42.ap-northeast-1.compute.internal:39971, NodeHttpAddress: ip-172-31-20-42.ap-northeast-1.compute.internal:41774, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.31.20.42:39971 }, ] for AM appattempt_1448220223188_0001_000002 2015-11-22 19:23:57,923 WARN [ContainersLauncher #1] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(237)) - Exit code from container container_1448220223188_0001_01_000002 is : 143 2015-11-22 19:23:57,923 INFO [ApplicationMasterLauncher #2] amlauncher.AMLauncher (AMLauncher.java:createAMContainerLaunchContext(188)) - Command to launch container container_1448220223188_0001_02_000001 : {{JAVA_HOME}}/bin/java -Xmx512m org.apache.hadoop.yarn.applications.distributedshell.TestDSSleepingAppMaster --container_memory 128 --container_vcores 1 --num_containers 1 --priority 0 1>/AppMaster.stdout 2>/AppMaster.stderr 2015-11-22 19:23:57,924 INFO [ApplicationMasterLauncher #2] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:createAndGetAMRMToken(195)) - Create AMRMToken for ApplicationAttempt: appattempt_1448220223188_0001_000002 2015-11-22 19:23:57,924 INFO [ApplicationMasterLauncher #2] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:createPassword(307)) - Creating password for appattempt_1448220223188_0001_000002 2015-11-22 19:23:57,928 INFO [Socket Reader #1 for port 39971] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220223188_0001_000002 (auth:SIMPLE) 2015-11-22 19:23:57,938 INFO [IPC Server handler 3 on 39971] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:startContainerInternal(889)) - Start request for container_1448220223188_0001_02_000001 by user ubuntu 2015-11-22 19:23:57,938 INFO [IPC Server handler 3 on 39971] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu IP=172.31.20.42 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1448220223188_0001 CONTAINERID=container_1448220223188_0001_02_000001 2015-11-22 19:23:57,940 INFO [ApplicationMasterLauncher #2] amlauncher.AMLauncher (AMLauncher.java:launch(127)) - Done launching container Container: [ContainerId: container_1448220223188_0001_02_000001, NodeId: ip-172-31-20-42.ap-northeast-1.compute.internal:39971, NodeHttpAddress: ip-172-31-20-42.ap-northeast-1.compute.internal:41774, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.31.20.42:39971 }, ] for AM appattempt_1448220223188_0001_000002 2015-11-22 19:23:57,940 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220223188_0001_000002 State change from ALLOCATED to LAUNCHED 2015-11-22 19:23:57,941 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220223188_0001_01_000002 transitioned from KILLING to CONTAINER_CLEANEDUP_AFTER_KILL 2015-11-22 19:23:57,941 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(303)) - Adding container_1448220223188_0001_02_000001 to application application_1448220223188_0001 2015-11-22 19:23:57,942 INFO [DeletionService #2] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(480)) - Deleting absolute path : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220223188_0001/container_1448220223188_0001_01_000002 2015-11-22 19:23:57,943 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220223188_0001_02_000001 transitioned from NEW to LOCALIZING 2015-11-22 19:23:57,943 INFO [AsyncDispatcher event handler] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu OPERATION=Container Finished - Killed TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1448220223188_0001 CONTAINERID=container_1448220223188_0001_01_000002 2015-11-22 19:23:57,943 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220223188_0001_01_000002 transitioned from CONTAINER_CLEANEDUP_AFTER_KILL to DONE 2015-11-22 19:23:57,943 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_INIT for appId application_1448220223188_0001 2015-11-22 19:23:57,944 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(346)) - Removing container_1448220223188_0001_01_000002 from application application_1448220223188_0001 2015-11-22 19:23:57,944 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_STOP for appId application_1448220223188_0001 2015-11-22 19:23:57,944 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220223188_0001_02_000001 transitioned from LOCALIZING to LOCALIZED 2015-11-22 19:23:57,962 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220223188_0001_02_000001 transitioned from LOCALIZED to RUNNING 2015-11-22 19:23:57,984 INFO [ContainersLauncher #1] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:buildCommandExecutor(281)) - launchContainer: [nice, -n, 0, bash, /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220223188_0001/container_1448220223188_0001_02_000001/default_container_executor.sh] 2015-11-22 19:23:58,577 INFO [Thread-919] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=N/A, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220229348, yarnAppState=ACCEPTED, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:52325/proxy/application_1448220223188_0001/, appUser=ubuntu 2015-11-22 19:23:58,900 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220223188_0001_02_000001 Container Transitioned from ACQUIRED to RUNNING 2015-11-22 19:23:58,900 INFO [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:completedContainer(1486)) - Container container_1448220223188_0001_01_000002 completed with event FINISHED 2015-11-22 19:23:59,580 INFO [Thread-919] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=N/A, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220229348, yarnAppState=ACCEPTED, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:52325/proxy/application_1448220223188_0001/, appUser=ubuntu 2015-11-22 19:24:00,409 INFO [Socket Reader #1 for port 50491] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220223188_0001_000002 (auth:SIMPLE) 2015-11-22 19:24:00,419 INFO [IPC Server handler 1 on 50491] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:registerApplicationMaster(220)) - AM registration appattempt_1448220223188_0001_000002 2015-11-22 19:24:00,419 INFO [IPC Server handler 1 on 50491] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(133)) - USER=ubuntu IP=172.31.20.42 OPERATION=Register App Master TARGET=ApplicationMasterService RESULT=SUCCESS APPID=application_1448220223188_0001 APPATTEMPTID=appattempt_1448220223188_0001_000002 2015-11-22 19:24:00,419 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220223188_0001_000002 State change from LAUNCHED to RUNNING 2015-11-22 19:24:00,419 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220223188_0001 State change from ACCEPTED to RUNNING 2015-11-22 19:24:00,582 INFO [Thread-919] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220229348, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:52325/proxy/application_1448220223188_0001/, appUser=ubuntu 2015-11-22 19:24:01,584 INFO [Thread-919] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220229348, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:52325/proxy/application_1448220223188_0001/, appUser=ubuntu 2015-11-22 19:24:01,900 INFO [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:removeOrTrackCompletedContainersFromContext(605)) - Removed completed containers from NM context: [container_1448220223188_0001_01_000002] 2015-11-22 19:24:01,902 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220223188_0001_02_000002 Container Transitioned from NEW to ALLOCATED 2015-11-22 19:24:01,902 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220223188_0001 CONTAINERID=container_1448220223188_0001_02_000002 2015-11-22 19:24:01,902 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(154)) - Assigned container container_1448220223188_0001_02_000002 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:39971, which has 2 containers, used and available after allocation 2015-11-22 19:24:01,902 INFO [ResourceManager Event Processor] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(89)) - assignedContainer application attempt=appattempt_1448220223188_0001_000002 container=container_1448220223188_0001_02_000002 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@12343406 clusterResource= type=OFF_SWITCH 2015-11-22 19:24:01,902 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(589)) - Re-sorting assigned queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.15625, absoluteUsedCapacity=0.15625, numApps=1, numContainers=2 2015-11-22 19:24:01,902 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(461)) - assignedContainer queue=root usedCapacity=0.15625 absoluteUsedCapacity=0.15625 used= cluster= 2015-11-22 19:24:02,464 INFO [IPC Server handler 2 on 50491] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:createAndGetNMToken(200)) - Sending NMToken for nodeId : ip-172-31-20-42.ap-northeast-1.compute.internal:39971 for container : container_1448220223188_0001_02_000002 2015-11-22 19:24:02,465 INFO [IPC Server handler 2 on 50491] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220223188_0001_02_000002 Container Transitioned from ALLOCATED to ACQUIRED 2015-11-22 19:24:02,539 INFO [Socket Reader #1 for port 39971] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220223188_0001_000002 (auth:SIMPLE) 2015-11-22 19:24:02,546 INFO [IPC Server handler 4 on 39971] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:startContainerInternal(889)) - Start request for container_1448220223188_0001_02_000002 by user ubuntu 2015-11-22 19:24:02,546 INFO [IPC Server handler 4 on 39971] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu IP=172.31.20.42 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1448220223188_0001 CONTAINERID=container_1448220223188_0001_02_000002 2015-11-22 19:24:02,546 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(303)) - Adding container_1448220223188_0001_02_000002 to application application_1448220223188_0001 2015-11-22 19:24:02,546 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220223188_0001_02_000002 transitioned from NEW to LOCALIZED 2015-11-22 19:24:02,547 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_INIT for appId application_1448220223188_0001 2015-11-22 19:24:02,581 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220223188_0001_02_000002 transitioned from LOCALIZED to RUNNING 2015-11-22 19:24:02,588 INFO [Socket Reader #1 for port 39971] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220223188_0001_000002 (auth:SIMPLE) 2015-11-22 19:24:02,590 INFO [Thread-919] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220229348, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:52325/proxy/application_1448220223188_0001/, appUser=ubuntu 2015-11-22 19:24:02,602 INFO [IPC Server handler 8 on 39971] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:getContainerStatusInternal(1221)) - Getting container-status for container_1448220223188_0001_02_000002 2015-11-22 19:24:02,602 INFO [IPC Server handler 8 on 39971] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:getContainerStatusInternal(1235)) - Returning ContainerStatus: [ContainerId: container_1448220223188_0001_02_000002, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ] 2015-11-22 19:24:02,633 INFO [ContainersLauncher #0] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:buildCommandExecutor(281)) - launchContainer: [nice, -n, 0, bash, /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220223188_0001/container_1448220223188_0001_02_000002/default_container_executor.sh] 2015-11-22 19:24:02,902 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220223188_0001_02_000002 Container Transitioned from ACQUIRED to RUNNING 2015-11-22 19:24:03,592 INFO [Thread-919] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220229348, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:52325/proxy/application_1448220223188_0001/, appUser=ubuntu 2015-11-22 19:24:04,594 INFO [Thread-919] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220229348, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:52325/proxy/application_1448220223188_0001/, appUser=ubuntu 2015-11-22 19:24:05,458 WARN [ContainersLauncher #1] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(237)) - Exit code from container container_1448220223188_0001_02_000001 is : 100 2015-11-22 19:24:05,458 WARN [ContainersLauncher #1] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(243)) - Exception from container-launch with container ID: container_1448220223188_0001_02_000001 and exit code: 100 ExitCodeException exitCode=100: at org.apache.hadoop.util.Shell.runCommand(Shell.java:909) at org.apache.hadoop.util.Shell.run(Shell.java:820) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1099) at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:225) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:304) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:84) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 2015-11-22 19:24:05,458 INFO [ContainersLauncher #1] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - Exception from container-launch. 2015-11-22 19:24:05,459 INFO [ContainersLauncher #1] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - Container id: container_1448220223188_0001_02_000001 2015-11-22 19:24:05,459 INFO [ContainersLauncher #1] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - Exit code: 100 2015-11-22 19:24:05,459 INFO [ContainersLauncher #1] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - Stack trace: ExitCodeException exitCode=100: 2015-11-22 19:24:05,459 INFO [ContainersLauncher #1] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - at org.apache.hadoop.util.Shell.runCommand(Shell.java:909) 2015-11-22 19:24:05,459 INFO [ContainersLauncher #1] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - at org.apache.hadoop.util.Shell.run(Shell.java:820) 2015-11-22 19:24:05,459 INFO [ContainersLauncher #1] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1099) 2015-11-22 19:24:05,459 INFO [ContainersLauncher #1] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:225) 2015-11-22 19:24:05,459 INFO [ContainersLauncher #1] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:304) 2015-11-22 19:24:05,459 INFO [ContainersLauncher #1] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:84) 2015-11-22 19:24:05,459 INFO [ContainersLauncher #1] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - at java.util.concurrent.FutureTask.run(FutureTask.java:266) 2015-11-22 19:24:05,459 INFO [ContainersLauncher #1] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 2015-11-22 19:24:05,459 INFO [ContainersLauncher #1] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 2015-11-22 19:24:05,459 INFO [ContainersLauncher #1] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - at java.lang.Thread.run(Thread.java:745) 2015-11-22 19:24:05,460 WARN [ContainersLauncher #1] launcher.ContainerLaunch (ContainerLaunch.java:call(348)) - Container exited with a non-zero exit code 100 2015-11-22 19:24:05,460 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220223188_0001_02_000001 transitioned from RUNNING to EXITED_WITH_FAILURE 2015-11-22 19:24:05,460 INFO [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(380)) - Cleaning up container container_1448220223188_0001_02_000001 2015-11-22 19:24:05,473 INFO [DeletionService #1] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(480)) - Deleting absolute path : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220223188_0001/container_1448220223188_0001_02_000001 2015-11-22 19:24:05,473 WARN [AsyncDispatcher event handler] nodemanager.NMAuditLogger (NMAuditLogger.java:logFailure(150)) - USER=ubuntu OPERATION=Container Finished - Failed TARGET=ContainerImpl RESULT=FAILURE DESCRIPTION=Container failed with state: EXITED_WITH_FAILURE APPID=application_1448220223188_0001 CONTAINERID=container_1448220223188_0001_02_000001 2015-11-22 19:24:05,473 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220223188_0001_02_000001 transitioned from EXITED_WITH_FAILURE to DONE 2015-11-22 19:24:05,473 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(346)) - Removing container_1448220223188_0001_02_000001 from application application_1448220223188_0001 2015-11-22 19:24:05,473 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_STOP for appId application_1448220223188_0001 2015-11-22 19:24:05,596 INFO [Thread-919] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220229348, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:52325/proxy/application_1448220223188_0001/, appUser=ubuntu 2015-11-22 19:24:05,904 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220223188_0001_02_000001 Container Transitioned from RUNNING to COMPLETED 2015-11-22 19:24:05,904 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220223188_0001 CONTAINERID=container_1448220223188_0001_02_000001 2015-11-22 19:24:05,904 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:releaseContainer(248)) - Released container container_1448220223188_0001_02_000001 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:39971, which currently has 1 containers, used and available, release resources=true 2015-11-22 19:24:05,904 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:rememberTargetTransitionsAndStoreState(1258)) - Updating application attempt appattempt_1448220223188_0001_000002 with final state: FAILED, and exit status: 100 2015-11-22 19:24:05,904 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220223188_0001_000002 State change from RUNNING to FINAL_SAVING 2015-11-22 19:24:05,904 INFO [AsyncDispatcher event handler] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:unregisterAttempt(667)) - Unregistering app attempt : appattempt_1448220223188_0001_000002 2015-11-22 19:24:05,905 INFO [AsyncDispatcher event handler] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:applicationMasterFinished(124)) - Application finished, removing password for appattempt_1448220223188_0001_000002 2015-11-22 19:24:05,905 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220223188_0001_000002 State change from FINAL_SAVING to FAILED 2015-11-22 19:24:05,905 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:transition(1302)) - The number of failed attempts in previous 15000 milliseconds is 2. The max attempts is 2 2015-11-22 19:24:05,906 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:rememberTargetTransitionsAndStoreState(1098)) - Updating application application_1448220223188_0001 with final state: FAILED 2015-11-22 19:24:05,906 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220223188_0001 State change from RUNNING to FINAL_SAVING 2015-11-22 19:24:05,906 INFO [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:transition(237)) - Updating info for app: application_1448220223188_0001 2015-11-22 19:24:05,906 INFO [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:doneApplicationAttempt(844)) - Application Attempt appattempt_1448220223188_0001_000002 is done. finalState=FAILED 2015-11-22 19:24:05,906 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220223188_0001_02_000002 Container Transitioned from RUNNING to KILLED 2015-11-22 19:24:05,906 INFO [ApplicationMasterLauncher #3] amlauncher.AMLauncher (AMLauncher.java:run(268)) - Cleaning master appattempt_1448220223188_0001_000002 2015-11-22 19:24:05,906 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:transition(1038)) - Application application_1448220223188_0001 failed 2 times in previous 15000 milliseconds due to AM Container for appattempt_1448220223188_0001_000002 exited with exitCode: 100 Failing this attempt.Diagnostics: Exception from container-launch. Container id: container_1448220223188_0001_02_000001 Exit code: 100 Stack trace: ExitCodeException exitCode=100: at org.apache.hadoop.util.Shell.runCommand(Shell.java:909) at org.apache.hadoop.util.Shell.run(Shell.java:820) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1099) at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:225) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:304) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:84) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Container exited with a non-zero exit code 100 For more detailed output, check the application tracking page: http://ip-172-31-20-42.ap-northeast-1.compute.internal:52325/cluster/app/application_1448220223188_0001 Then click on links to logs of each attempt. . Failing the application. 2015-11-22 19:24:05,906 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220223188_0001 CONTAINERID=container_1448220223188_0001_02_000002 2015-11-22 19:24:05,907 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:releaseContainer(248)) - Released container container_1448220223188_0001_02_000002 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:39971, which currently has 0 containers, used and available, release resources=true 2015-11-22 19:24:05,907 INFO [ResourceManager Event Processor] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:clearRequests(125)) - Application application_1448220223188_0001 requests cleared 2015-11-22 19:24:05,907 INFO [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:removeApplicationAttempt(772)) - Application removed - appId: application_1448220223188_0001 user: ubuntu queue: default #user-pending-applications: 0 #user-active-applications: 0 #queue-pending-applications: 0 #queue-active-applications: 0 2015-11-22 19:24:05,911 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220223188_0001 State change from FINAL_SAVING to FAILED 2015-11-22 19:24:05,911 INFO [Socket Reader #1 for port 39971] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220223188_0001_000002 (auth:SIMPLE) 2015-11-22 19:24:05,911 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:removeApplication(374)) - Application removed - appId: application_1448220223188_0001 user: ubuntu leaf-queue of parent: root #applications: 0 2015-11-22 19:24:05,917 WARN [AsyncDispatcher event handler] resourcemanager.RMAuditLogger (RMAuditLogger.java:logFailure(269)) - USER=ubuntu OPERATION=Application Finished - Failed TARGET=RMAppManager RESULT=FAILURE DESCRIPTION=App failed with state: FAILED PERMISSIONS=Application application_1448220223188_0001 failed 2 times in previous 15000 milliseconds due to AM Container for appattempt_1448220223188_0001_000002 exited with exitCode: 100 Failing this attempt.Diagnostics: Exception from container-launch. Container id: container_1448220223188_0001_02_000001 Exit code: 100 Stack trace: ExitCodeException exitCode=100: at org.apache.hadoop.util.Shell.runCommand(Shell.java:909) at org.apache.hadoop.util.Shell.run(Shell.java:820) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1099) at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:225) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:304) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:84) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Container exited with a non-zero exit code 100 For more detailed output, check the application tracking page: http://ip-172-31-20-42.ap-northeast-1.compute.internal:52325/cluster/app/application_1448220223188_0001 Then click on links to logs of each attempt. . Failing the application. APPID=application_1448220223188_0001 2015-11-22 19:24:05,918 INFO [IPC Server handler 3 on 39971] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:stopContainerInternal(1163)) - Stopping container with container Id: container_1448220223188_0001_02_000001 2015-11-22 19:24:05,919 INFO [IPC Server handler 3 on 39971] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu IP=172.31.20.42 OPERATION=Stop Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1448220223188_0001 CONTAINERID=container_1448220223188_0001_02_000001 2015-11-22 19:24:05,919 INFO [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:removeOrTrackCompletedContainersFromContext(605)) - Removed completed containers from NM context: [container_1448220223188_0001_02_000001] 2015-11-22 19:24:05,920 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220223188_0001_02_000002 transitioned from RUNNING to KILLING 2015-11-22 19:24:05,920 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(463)) - Application application_1448220223188_0001 transitioned from RUNNING to FINISHING_CONTAINERS_WAIT 2015-11-22 19:24:05,920 INFO [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(380)) - Cleaning up container container_1448220223188_0001_02_000002 2015-11-22 19:24:05,921 INFO [AsyncDispatcher event handler] resourcemanager.RMAppManager$ApplicationSummary (RMAppManager.java:logAppSummary(187)) - appId=application_1448220223188_0001,name=DistributedShell,user=ubuntu,queue=default,state=FAILED,trackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:52325/cluster/app/application_1448220223188_0001,appMasterHost=N/A,startTime=1448220229348,finishTime=1448220245905,finalStatus=FAILED,memorySeconds=9220,vcoreSeconds=24,preemptedAMContainers=0,preemptedNonAMContainers=0,preemptedResources=,applicationType=YARN 2015-11-22 19:24:05,927 WARN [ContainersLauncher #0] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(237)) - Exit code from container container_1448220223188_0001_02_000002 is : 143 2015-11-22 19:24:05,936 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220223188_0001_02_000002 transitioned from KILLING to CONTAINER_CLEANEDUP_AFTER_KILL 2015-11-22 19:24:05,936 INFO [DeletionService #3] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(480)) - Deleting absolute path : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220223188_0001/container_1448220223188_0001_02_000002 2015-11-22 19:24:05,937 INFO [AsyncDispatcher event handler] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu OPERATION=Container Finished - Killed TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1448220223188_0001 CONTAINERID=container_1448220223188_0001_02_000002 2015-11-22 19:24:05,937 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220223188_0001_02_000002 transitioned from CONTAINER_CLEANEDUP_AFTER_KILL to DONE 2015-11-22 19:24:05,937 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(401)) - Removing container_1448220223188_0001_02_000002 from application application_1448220223188_0001 2015-11-22 19:24:05,937 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(463)) - Application application_1448220223188_0001 transitioned from FINISHING_CONTAINERS_WAIT to APPLICATION_RESOURCES_CLEANINGUP 2015-11-22 19:24:05,937 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_STOP for appId application_1448220223188_0001 2015-11-22 19:24:05,938 INFO [DeletionService #3] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(480)) - Deleting absolute path : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220223188_0001 2015-11-22 19:24:05,938 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event APPLICATION_STOP for appId application_1448220223188_0001 2015-11-22 19:24:05,938 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(463)) - Application application_1448220223188_0001 transitioned from APPLICATION_RESOURCES_CLEANINGUP to FINISHED 2015-11-22 19:24:05,938 INFO [AsyncDispatcher event handler] loghandler.NonAggregatingLogHandler (NonAggregatingLogHandler.java:handle(167)) - Scheduling Log Deletion for application: application_1448220223188_0001, with delay of 10800 seconds 2015-11-22 19:24:06,598 INFO [Thread-919] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=Application application_1448220223188_0001 failed 2 times in previous 15000 milliseconds due to AM Container for appattempt_1448220223188_0001_000002 exited with exitCode: 100 Failing this attempt.Diagnostics: Exception from container-launch. Container id: container_1448220223188_0001_02_000001 Exit code: 100 Stack trace: ExitCodeException exitCode=100: at org.apache.hadoop.util.Shell.runCommand(Shell.java:909) at org.apache.hadoop.util.Shell.run(Shell.java:820) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1099) at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:225) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:304) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:84) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Container exited with a non-zero exit code 100 For more detailed output, check the application tracking page: http://ip-172-31-20-42.ap-northeast-1.compute.internal:52325/cluster/app/application_1448220223188_0001 Then click on links to logs of each attempt. . Failing the application., appMasterHost=N/A, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220229348, yarnAppState=FAILED, distributedFinalState=FAILED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:52325/cluster/app/application_1448220223188_0001, appUser=ubuntu 2015-11-22 19:24:06,598 INFO [Thread-919] distributedshell.Client (Client.java:monitorApplication(771)) - Application did not finish. YarnState=FAILED, DSFinalStatus=FAILED. Breaking monitoring loop 2015-11-22 19:24:06,598 INFO [Thread-919] distributedshell.TestDistributedShell (TestDistributedShell.java:testDSAttemptFailuresValidityIntervalFailed(444)) - Client run completed. Result=false 2015-11-22 19:24:06,601 ERROR [Thread[Thread-915,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(670)) - ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted 2015-11-22 19:24:06,603 INFO [main] mortbay.log (Slf4jLog.java:info(67)) - Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:0 2015-11-22 19:24:06,706 INFO [main] applicationhistoryservice.ApplicationHistoryManagerImpl (ApplicationHistoryManagerImpl.java:serviceStop(78)) - Stopping ApplicationHistory 2015-11-22 19:24:06,708 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 47761 2015-11-22 19:24:06,710 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:24:06,710 INFO [IPC Server listener on 47761] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 47761 2015-11-22 19:24:06,713 ERROR [Thread[Thread-897,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(670)) - ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted 2015-11-22 19:24:06,715 INFO [main] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:unRegisterNM(268)) - Successfully Unregistered the Node ip-172-31-20-42.ap-northeast-1.compute.internal:39971 with ResourceManager. 2015-11-22 19:24:06,718 ERROR [Thread[Thread-890,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(670)) - ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted 2015-11-22 19:24:06,720 INFO [main] mortbay.log (Slf4jLog.java:info(67)) - Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:0 2015-11-22 19:24:06,822 INFO [main] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:cleanUpApplicationsOnNMShutDown(556)) - Applications still running : [application_1448220223188_0001] 2015-11-22 19:24:06,822 INFO [main] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:cleanUpApplicationsOnNMShutDown(572)) - Waiting for Applications to be Finished 2015-11-22 19:24:10,823 INFO [main] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:cleanUpApplicationsOnNMShutDown(589)) - Done waiting for Applications to be Finished. Still alive: [application_1448220223188_0001] 2015-11-22 19:24:10,824 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 39971 2015-11-22 19:24:10,828 INFO [IPC Server listener on 39971] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 39971 2015-11-22 19:24:10,830 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:24:20,834 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 48479 2015-11-22 19:24:20,835 INFO [IPC Server listener on 48479] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 48479 2015-11-22 19:24:20,835 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:24:20,836 INFO [Public Localizer] localizer.ResourceLocalizationService (ResourceLocalizationService.java:run(908)) - Public cache exiting 2015-11-22 19:24:20,838 WARN [Node Resource Monitor] nodemanager.NodeResourceMonitorImpl (NodeResourceMonitorImpl.java:run(154)) - org.apache.hadoop.yarn.server.nodemanager.NodeResourceMonitorImpl is interrupted. Exiting. 2015-11-22 19:24:25,842 WARN [main] server.MiniYARNCluster (MiniYARNCluster.java:waitForAppMastersToFinish(474)) - Stopping RM while some app masters are still alive 2015-11-22 19:24:25,844 ERROR [Thread[Thread-840,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(670)) - ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted 2015-11-22 19:24:25,846 INFO [main] mortbay.log (Slf4jLog.java:info(67)) - Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:0 2015-11-22 19:24:25,949 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 35934 2015-11-22 19:24:25,952 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:24:25,952 INFO [IPC Server listener on 35934] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 35934 2015-11-22 19:24:25,955 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 40969 2015-11-22 19:24:25,955 INFO [IPC Server listener on 40969] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 40969 2015-11-22 19:24:25,955 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:24:25,957 INFO [main] resourcemanager.ResourceManager (ResourceManager.java:transitionToStandby(1051)) - Transitioning to standby state 2015-11-22 19:24:25,957 WARN [ApplicationMaster Launcher] amlauncher.ApplicationMasterLauncher (ApplicationMasterLauncher.java:run(122)) - org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher$LauncherThread interrupted. Returning. 2015-11-22 19:24:25,959 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 50491 2015-11-22 19:24:25,961 INFO [IPC Server listener on 50491] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 50491 2015-11-22 19:24:25,961 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:24:25,962 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 49532 2015-11-22 19:24:25,964 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:24:25,964 INFO [IPC Server listener on 49532] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 49532 2015-11-22 19:24:25,964 ERROR [ResourceManager Event Processor] resourcemanager.ResourceManager (ResourceManager.java:run(691)) - Returning, interrupted : java.lang.InterruptedException 2015-11-22 19:24:25,964 INFO [Ping Checker] util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(135)) - NMLivelinessMonitor thread interrupted 2015-11-22 19:24:25,965 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:serviceStop(141)) - AsyncDispatcher is draining to stop, igonring any new events. 2015-11-22 19:24:25,967 INFO [Ping Checker] util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(135)) - AMLivelinessMonitor thread interrupted 2015-11-22 19:24:25,967 INFO [Ping Checker] util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(135)) - AMLivelinessMonitor thread interrupted 2015-11-22 19:24:25,967 ERROR [Thread[Thread-671,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(670)) - ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted 2015-11-22 19:24:25,968 INFO [Ping Checker] util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(135)) - org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.ContainerAllocationExpirer thread interrupted 2015-11-22 19:24:25,969 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stop(211)) - Stopping ApplicationHistoryServer metrics system... 2015-11-22 19:24:25,971 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stop(217)) - ApplicationHistoryServer metrics system stopped. 2015-11-22 19:24:25,971 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:shutdown(608)) - ApplicationHistoryServer metrics system shutdown complete. 2015-11-22 19:24:25,971 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:serviceStop(141)) - AsyncDispatcher is draining to stop, igonring any new events. 2015-11-22 19:24:25,971 INFO [main] resourcemanager.ResourceManager (ResourceManager.java:transitionToStandby(1058)) - Transitioned to standby state 2015-11-22 19:24:25,983 INFO [main] distributedshell.TestDistributedShell (TestDistributedShell.java:setupInternal(77)) - Starting up YARN cluster 2015-11-22 19:24:25,992 INFO [main] net.ServerSocketUtil (ServerSocketUtil.java:getPort(52)) - Using port 10021 2015-11-22 19:24:26,003 INFO [main] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2363)) - found resource core-site.xml at file:/home/ubuntu/hadoop-dev/hadoop-common-project/hadoop-common/target/test-classes/core-site.xml 2015-11-22 19:24:26,012 WARN [main] conf.Configuration (Configuration.java:loadProperty(2687)) - core-site.xml:an attempt to override final parameter: hadoop.tmp.dir; Ignoring. 2015-11-22 19:24:26,013 INFO [main] security.Groups (Groups.java:refresh(256)) - clearing userToGroupsMap cache 2015-11-22 19:24:26,014 INFO [main] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2363)) - found resource yarn-site.xml at file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/test-classes/yarn-site.xml 2015-11-22 19:24:26,029 WARN [main] conf.Configuration (Configuration.java:loadProperty(2687)) - yarn-site.xml:an attempt to override final parameter: hadoop.tmp.dir; Ignoring. 2015-11-22 19:24:26,029 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.RMFatalEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMFatalEventDispatcher 2015-11-22 19:24:26,030 INFO [main] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:(75)) - NMTokenKeyRollingInterval: 86400000ms and NMTokenKeyActivationDelay: 900000ms 2015-11-22 19:24:26,030 INFO [main] security.RMContainerTokenSecretManager (RMContainerTokenSecretManager.java:(78)) - ContainerTokenKeyRollingInterval: 86400000ms and ContainerTokenKeyActivationDelay: 900000ms 2015-11-22 19:24:26,030 INFO [main] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:(94)) - AMRMTokenKeyRollingInterval: 86400000ms and AMRMTokenKeyActivationDelay: 900000 ms 2015-11-22 19:24:26,031 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStoreEventType for class org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$ForwardingEventHandler 2015-11-22 19:24:26,031 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.NodesListManagerEventType for class org.apache.hadoop.yarn.server.resourcemanager.NodesListManager 2015-11-22 19:24:26,031 INFO [main] resourcemanager.ResourceManager (ResourceManager.java:createScheduler(295)) - Using Scheduler: org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler 2015-11-22 19:24:26,031 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.SchedulerEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher 2015-11-22 19:24:26,031 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationEventDispatcher 2015-11-22 19:24:26,031 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationAttemptEventDispatcher 2015-11-22 19:24:26,031 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$NodeEventDispatcher 2015-11-22 19:24:26,033 INFO [main] impl.MetricsConfig (MetricsConfig.java:loadFirst(115)) - loaded properties from hadoop-metrics2.properties 2015-11-22 19:24:26,034 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:startTimer(374)) - Scheduled snapshot period at 10 second(s). 2015-11-22 19:24:26,034 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:start(192)) - ResourceManager metrics system started 2015-11-22 19:24:26,035 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.RMAppManagerEventType for class org.apache.hadoop.yarn.server.resourcemanager.RMAppManager 2015-11-22 19:24:26,035 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncherEventType for class org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher 2015-11-22 19:24:26,035 WARN [main] util.MBeans (MBeans.java:register(73)) - Failed to register MBean "Hadoop:service=ResourceManager,name=RMNMInfo": Instance already exists. 2015-11-22 19:24:26,035 INFO [main] resourcemanager.RMNMInfo (RMNMInfo.java:(63)) - Registered RMNMInfo MBean 2015-11-22 19:24:26,035 INFO [main] util.HostsFileReader (HostsFileReader.java:refresh(132)) - Refreshing hosts (include/exclude) list 2015-11-22 19:24:26,036 INFO [main] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2363)) - found resource capacity-scheduler.xml at jar:file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/target/hadoop-yarn-server-tests-2.8.0-SNAPSHOT-tests.jar!/capacity-scheduler.xml 2015-11-22 19:24:26,049 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(647)) - max alloc mb per queue for root is undefined 2015-11-22 19:24:26,049 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(651)) - max alloc vcore per queue for root is undefined 2015-11-22 19:24:26,049 INFO [main] capacity.ParentQueue (ParentQueue.java:setupQueueConfigs(125)) - root, capacity=1.0, asboluteCapacity=1.0, maxCapacity=1.0, asboluteMaxCapacity=1.0, state=RUNNING, acls=ADMINISTER_QUEUE:*SUBMIT_APP:*, labels=*, , reservationsContinueLooking=true 2015-11-22 19:24:26,050 INFO [main] capacity.ParentQueue (ParentQueue.java:(104)) - Initialized parent-queue root name=root, fullname=root 2015-11-22 19:24:26,051 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(647)) - max alloc mb per queue for root.default is undefined 2015-11-22 19:24:26,051 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(651)) - max alloc vcore per queue for root.default is undefined 2015-11-22 19:24:26,052 INFO [main] capacity.LeafQueue (LeafQueue.java:setupQueueConfigs(217)) - Initializing default capacity = 1.0 [= (float) configuredCapacity / 100 ] asboluteCapacity = 1.0 [= parentAbsoluteCapacity * capacity ] maxCapacity = 1.0 [= configuredMaxCapacity ] absoluteMaxCapacity = 1.0 [= 1.0 maximumCapacity undefined, (parentAbsoluteMaxCapacity * maximumCapacity) / 100 otherwise ] userLimit = 100 [= configuredUserLimit ] userLimitFactor = 1.0 [= configuredUserLimitFactor ] maxApplications = 10000 [= configuredMaximumSystemApplicationsPerQueue or (int)(configuredMaximumSystemApplications * absoluteCapacity)] maxApplicationsPerUser = 10000 [= (int)(maxApplications * (userLimit / 100.0f) * userLimitFactor) ] usedCapacity = 0.0 [= usedResourcesMemory / (clusterResourceMemory * absoluteCapacity)] absoluteUsedCapacity = 0.0 [= usedResourcesMemory / clusterResourceMemory] maxAMResourcePerQueuePercent = 0.1 [= configuredMaximumAMResourcePercent ] minimumAllocationFactor = 0.984375 [= (float)(maximumAllocationMemory - minimumAllocationMemory) / maximumAllocationMemory ] maximumAllocation = [= configuredMaxAllocation ] numContainers = 0 [= currentNumContainers ] state = RUNNING [= configuredState ] acls = ADMINISTER_QUEUE:*SUBMIT_APP:* [= configuredAcls ] nodeLocalityDelay = -1 labels=*, nodeLocalityDelay = -1 reservationsContinueLooking = true preemptionDisabled = true defaultAppPriorityPerQueue = 0 2015-11-22 19:24:26,052 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:parseQueue(657)) - Initialized queue: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=0, numContainers=0 2015-11-22 19:24:26,052 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:parseQueue(657)) - Initialized queue: root: numChildQueue= 1, capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.0, numApps=0, numContainers=0 2015-11-22 19:24:26,052 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:initializeQueues(501)) - Initialized root queue root: numChildQueue= 1, capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.0, numApps=0, numContainers=0 2015-11-22 19:24:26,053 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:getUserGroupMappingPlacementRule(449)) - Initialized queue mappings, override: false 2015-11-22 19:24:26,053 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:initScheduler(314)) - Initialized CapacityScheduler with calculator=class org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator, minimumAllocation=<>, maximumAllocation=<>, asynchronousScheduling=false, asyncScheduleInterval=5ms 2015-11-22 19:24:26,053 INFO [main] metrics.SystemMetricsPublisher (SystemMetricsPublisher.java:serviceInit(95)) - YARN system metrics publishing service is not enabled 2015-11-22 19:24:26,053 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEventType for class org.apache.hadoop.yarn.server.MiniYARNCluster$1 2015-11-22 19:24:26,054 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:prepareDirs(558)) - Created localDir in /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0 2015-11-22 19:24:26,054 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:prepareDirs(558)) - Created logDir in /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-logDir-nm-0_0 2015-11-22 19:24:26,054 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceInit(540)) - Starting NM: 0 2015-11-22 19:24:26,063 INFO [main] nodemanager.NodeManager (NodeManager.java:getNodeHealthScriptRunner(255)) - Node Manager health check script is not available or doesn't have execute permission, so not starting the node health script runner. 2015-11-22 19:24:26,064 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ContainerEventDispatcher 2015-11-22 19:24:26,064 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ApplicationEventDispatcher 2015-11-22 19:24:26,064 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizationEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService 2015-11-22 19:24:26,064 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServicesEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices 2015-11-22 19:24:26,064 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl 2015-11-22 19:24:26,064 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncherEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncher 2015-11-22 19:24:26,064 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.ContainerManagerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl 2015-11-22 19:24:26,065 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.NodeManagerEventType for class org.apache.hadoop.yarn.server.MiniYARNCluster$ShortCircuitedNodeManager 2015-11-22 19:24:26,065 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:init(159)) - NodeManager metrics system started (again) 2015-11-22 19:24:26,066 INFO [main] nodemanager.NodeResourceMonitorImpl (NodeResourceMonitorImpl.java:serviceInit(76)) - Using ResourceCalculatorPlugin : org.apache.hadoop.yarn.util.ResourceCalculatorPlugin@6ad82709 2015-11-22 19:24:26,066 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.event.LogHandlerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.NonAggregatingLogHandler 2015-11-22 19:24:26,066 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.sharedcache.SharedCacheUploadEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.sharedcache.SharedCacheUploadService 2015-11-22 19:24:26,066 INFO [main] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:serviceInit(261)) - AMRMProxyService is disabled 2015-11-22 19:24:26,066 INFO [main] localizer.ResourceLocalizationService (ResourceLocalizationService.java:validateConf(224)) - per directory file limit = 8192 2015-11-22 19:24:26,078 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerTracker 2015-11-22 19:24:26,078 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(116)) - Using ResourceCalculatorPlugin : org.apache.hadoop.yarn.util.ResourceCalculatorPlugin@510f3d34 2015-11-22 19:24:26,078 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(121)) - Using ResourceCalculatorProcessTree : null 2015-11-22 19:24:26,078 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(158)) - Physical memory check enabled: false 2015-11-22 19:24:26,079 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(159)) - Virtual memory check enabled: false 2015-11-22 19:24:26,079 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:isEnabled(206)) - Neither virutal-memory nor physical-memory monitoring is needed. Not running the monitor-thread 2015-11-22 19:24:26,079 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(162)) - ContainersMonitor enabled: false 2015-11-22 19:24:26,079 INFO [main] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:serviceInit(175)) - Nodemanager resources: memory set to 4096MB. 2015-11-22 19:24:26,079 INFO [main] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:serviceInit(176)) - Nodemanager resources: vcores set to 8. 2015-11-22 19:24:26,080 INFO [main] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:serviceInit(208)) - Initialized nodemanager with : physical-memory=4096 virtual-memory=8602 virtual-cores=8 2015-11-22 19:24:26,080 WARN [main] applicationhistoryservice.ApplicationHistoryServer (ApplicationHistoryServer.java:createApplicationHistoryManager(218)) - The filesystem based application history store is deprecated. 2015-11-22 19:24:26,080 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:init(159)) - ApplicationHistoryServer metrics system started (again) 2015-11-22 19:24:26,081 INFO [main] applicationhistoryservice.ApplicationHistoryManagerImpl (ApplicationHistoryManagerImpl.java:serviceInit(61)) - ApplicationHistory Init 2015-11-22 19:24:26,082 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:startResourceManager(326)) - Waiting for RM to start... 2015-11-22 19:24:26,082 INFO [RM-0] resourcemanager.ResourceManager (ResourceManager.java:transitionToActive(1024)) - Transitioning to active state 2015-11-22 19:24:26,084 INFO [RM-0] recovery.RMStateStore (RMStateStore.java:transition(482)) - Updating AMRMToken 2015-11-22 19:24:26,084 INFO [RM-0] security.RMContainerTokenSecretManager (RMContainerTokenSecretManager.java:rollMasterKey(106)) - Rolling master-key for container-tokens 2015-11-22 19:24:26,085 INFO [RM-0] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:rollMasterKey(95)) - Rolling master-key for nm-tokens 2015-11-22 19:24:26,085 INFO [RM-0] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:24:26,085 INFO [RM-0] security.RMDelegationTokenSecretManager (RMDelegationTokenSecretManager.java:storeNewMasterKey(87)) - storing master key with keyID 1 2015-11-22 19:24:26,085 INFO [RM-0] recovery.RMStateStore (RMStateStore.java:transition(433)) - Storing RMDTMasterKey. 2015-11-22 19:24:26,086 INFO [Thread[Thread-1071,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2015-11-22 19:24:26,088 INFO [Thread[Thread-1071,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:24:26,092 INFO [Thread[Thread-1071,5,main]] security.RMDelegationTokenSecretManager (RMDelegationTokenSecretManager.java:storeNewMasterKey(87)) - storing master key with keyID 2 2015-11-22 19:24:26,096 INFO [Thread[Thread-1071,5,main]] recovery.RMStateStore (RMStateStore.java:transition(433)) - Storing RMDTMasterKey. 2015-11-22 19:24:26,105 INFO [RM-0] nodelabels.CommonNodeLabelsManager (CommonNodeLabelsManager.java:addToCluserNodeLabels(309)) - Add labels: [] 2015-11-22 19:24:26,105 INFO [RM-0] nodelabels.CommonNodeLabelsManager (CommonNodeLabelsManager.java:internalUpdateLabelsOnNodes(638)) - REPLACE labels on nodes: 2015-11-22 19:24:26,105 INFO [RM-0] nodelabels.CommonNodeLabelsManager (CommonNodeLabelsManager.java:internalUpdateLabelsOnNodes(640)) - NM=ip-172-31-20-42.ap-northeast-1.compute.internal:58694, labels=[x] 2015-11-22 19:24:26,106 INFO [RM-0] nodelabels.FileSystemNodeLabelsStore (FileSystemNodeLabelsStore.java:recover(279)) - Finished write mirror at:file:/tmp/hadoop-yarn-ubuntu/node-labels/nodelabel.mirror 2015-11-22 19:24:26,106 INFO [RM-0] nodelabels.FileSystemNodeLabelsStore (FileSystemNodeLabelsStore.java:recover(280)) - Finished create editlog file at:file:/tmp/hadoop-yarn-ubuntu/node-labels/nodelabel.editlog 2015-11-22 19:24:26,106 INFO [RM-0] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.nodelabels.event.NodeLabelsStoreEventType for class org.apache.hadoop.yarn.nodelabels.CommonNodeLabelsManager$ForwardingEventHandler 2015-11-22 19:24:26,109 INFO [RM-0] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:24:26,110 INFO [Socket Reader #1 for port 41635] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 41635 2015-11-22 19:24:26,112 INFO [RM-0] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.server.api.ResourceTrackerPB to the server 2015-11-22 19:24:26,112 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:24:26,112 INFO [IPC Server listener on 41635] ipc.Server (Server.java:run(794)) - IPC Server listener on 41635: starting 2015-11-22 19:24:26,121 INFO [RM-0] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:24:26,121 INFO [Socket Reader #1 for port 32866] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 32866 2015-11-22 19:24:26,123 INFO [RM-0] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB to the server 2015-11-22 19:24:26,123 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:24:26,128 INFO [IPC Server listener on 32866] ipc.Server (Server.java:run(794)) - IPC Server listener on 32866: starting 2015-11-22 19:24:26,135 INFO [RM-0] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:24:26,135 INFO [Socket Reader #1 for port 50548] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 50548 2015-11-22 19:24:26,137 INFO [RM-0] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.api.ApplicationClientProtocolPB to the server 2015-11-22 19:24:26,137 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:24:26,140 INFO [IPC Server listener on 50548] ipc.Server (Server.java:run(794)) - IPC Server listener on 50548: starting 2015-11-22 19:24:26,146 INFO [RM-0] resourcemanager.ResourceManager (ResourceManager.java:transitionToActive(1040)) - Transitioned to active state 2015-11-22 19:24:26,148 INFO [RM-0] server.AuthenticationFilter (AuthenticationFilter.java:constructSecretProvider(304)) - Unable to initialize FileSignerSecretProvider, falling back to use random secrets. 2015-11-22 19:24:26,148 INFO [RM-0] http.HttpRequestLog (HttpRequestLog.java:getRequestLog(80)) - Http request log for http.requests.resourcemanager is not defined 2015-11-22 19:24:26,149 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter) 2015-11-22 19:24:26,149 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context cluster 2015-11-22 19:24:26,149 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context logs 2015-11-22 19:24:26,149 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context static 2015-11-22 19:24:26,150 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context cluster 2015-11-22 19:24:26,150 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2015-11-22 19:24:26,150 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2015-11-22 19:24:26,150 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'Timeline Authentication Filter' (class=org.apache.hadoop.yarn.server.timeline.security.TimelineAuthenticationFilter) 2015-11-22 19:24:26,151 INFO [RM-0] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /cluster/* 2015-11-22 19:24:26,151 INFO [RM-0] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /ws/* 2015-11-22 19:24:26,157 INFO [RM-0] webapp.WebApps (WebApps.java:build(288)) - Registered webapp guice modules 2015-11-22 19:24:26,157 INFO [RM-0] http.HttpServer2 (HttpServer2.java:openListeners(917)) - Jetty bound to port 36077 2015-11-22 19:24:26,157 INFO [RM-0] mortbay.log (Slf4jLog.java:info(67)) - jetty-6.1.26 2015-11-22 19:24:26,169 INFO [RM-0] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:24:26,169 INFO [Thread[Thread-1240,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2015-11-22 19:24:26,170 INFO [Thread[Thread-1240,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:24:26,170 INFO [RM-0] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:24:26,171 INFO [Thread[Thread-1242,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) Nov 22, 2015 7:24:26 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.resourcemanager.webapp.JAXBContextResolver as a provider class Nov 22, 2015 7:24:26 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices as a root resource class Nov 22, 2015 7:24:26 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a provider class Nov 22, 2015 7:24:26 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate INFO: Initiating Jersey application, version 'Jersey: 1.9 09/02/2011 11:17 AM' Nov 22, 2015 7:24:26 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.resourcemanager.webapp.JAXBContextResolver to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:24:26 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.webapp.GenericExceptionHandler to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:24:26 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices to GuiceManagedComponentProvider with the scope "Singleton" 2015-11-22 19:24:26,333 INFO [RM-0] mortbay.log (Slf4jLog.java:info(67)) - Started HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:36077 2015-11-22 19:24:26,334 INFO [RM-0] webapp.WebApps (WebApps.java:start(307)) - Web app cluster started at 36077 2015-11-22 19:24:26,336 INFO [RM-0] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:24:26,337 INFO [Socket Reader #1 for port 39659] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 39659 2015-11-22 19:24:26,342 INFO [RM-0] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.server.api.ResourceManagerAdministrationProtocolPB to the server 2015-11-22 19:24:26,342 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:24:26,345 INFO [IPC Server listener on 39659] ipc.Server (Server.java:run(794)) - IPC Server listener on 39659: starting 2015-11-22 19:24:27,583 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:startResourceManager(339)) - MiniYARN ResourceManager address: ip-172-31-20-42.ap-northeast-1.compute.internal:50548 2015-11-22 19:24:27,583 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:startResourceManager(341)) - MiniYARN ResourceManager web address: ip-172-31-20-42.ap-northeast-1.compute.internal:36077 2015-11-22 19:24:27,583 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(459)) - MiniYARN ResourceManager address: ip-172-31-20-42.ap-northeast-1.compute.internal:50548 2015-11-22 19:24:27,583 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(461)) - MiniYARN ResourceManager web address: ip-172-31-20-42.ap-northeast-1.compute.internal:36077 2015-11-22 19:24:27,583 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(575)) - Waiting for NM 0 to start... 2015-11-22 19:24:27,589 INFO [Thread-1249] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:24:27,590 INFO [Socket Reader #1 for port 34720] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 34720 2015-11-22 19:24:27,591 INFO [Thread-1249] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.api.ContainerManagementProtocolPB to the server 2015-11-22 19:24:27,591 INFO [Thread-1249] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:serviceStart(464)) - Blocking new container-requests as container manager rpc server is still starting. 2015-11-22 19:24:27,592 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:24:27,592 INFO [IPC Server listener on 34720] ipc.Server (Server.java:run(794)) - IPC Server listener on 34720: starting 2015-11-22 19:24:27,598 INFO [Thread-1249] security.NMContainerTokenSecretManager (NMContainerTokenSecretManager.java:setNodeId(260)) - Updating node address : ip-172-31-20-42.ap-northeast-1.compute.internal:34720 2015-11-22 19:24:27,598 INFO [Thread-1249] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:24:27,599 INFO [Socket Reader #1 for port 43758] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 43758 2015-11-22 19:24:27,600 INFO [Thread-1249] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB to the server 2015-11-22 19:24:27,600 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:24:27,600 INFO [IPC Server listener on 43758] ipc.Server (Server.java:run(794)) - IPC Server listener on 43758: starting 2015-11-22 19:24:27,604 INFO [Thread-1249] localizer.ResourceLocalizationService (ResourceLocalizationService.java:serviceStart(363)) - Localizer started on port 43758 2015-11-22 19:24:27,611 INFO [Thread-1249] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:serviceStart(509)) - ContainerManager started at /172.31.20.42:34720 2015-11-22 19:24:27,611 INFO [Thread-1249] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:serviceStart(510)) - ContainerManager bound to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:0 2015-11-22 19:24:27,611 INFO [Thread-1249] webapp.WebServer (WebServer.java:serviceStart(71)) - Instantiating NMWebApp at ip-172-31-20-42.ap-northeast-1.compute.internal:0 2015-11-22 19:24:27,612 INFO [Thread-1249] server.AuthenticationFilter (AuthenticationFilter.java:constructSecretProvider(304)) - Unable to initialize FileSignerSecretProvider, falling back to use random secrets. 2015-11-22 19:24:27,612 INFO [Thread-1249] http.HttpRequestLog (HttpRequestLog.java:getRequestLog(80)) - Http request log for http.requests.nodemanager is not defined 2015-11-22 19:24:27,612 INFO [Thread-1249] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter) 2015-11-22 19:24:27,613 INFO [Thread-1249] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context node 2015-11-22 19:24:27,613 INFO [Thread-1249] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context logs 2015-11-22 19:24:27,613 INFO [Thread-1249] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context static 2015-11-22 19:24:27,613 INFO [Thread-1249] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context node 2015-11-22 19:24:27,613 INFO [Thread-1249] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2015-11-22 19:24:27,613 INFO [Thread-1249] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2015-11-22 19:24:27,614 INFO [Thread-1249] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'Timeline Authentication Filter' (class=org.apache.hadoop.yarn.server.timeline.security.TimelineAuthenticationFilter) 2015-11-22 19:24:27,615 INFO [Thread-1249] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /node/* 2015-11-22 19:24:27,615 INFO [Thread-1249] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /ws/* Nov 22, 2015 7:24:27 PM com.google.inject.servlet.GuiceFilter setPipeline WARNING: Multiple Servlet injectors detected. This is a warning indicating that you have more than one GuiceFilter running in your web application. If this is deliberate, you may safely ignore this message. If this is NOT deliberate however, your application may not work as expected. 2015-11-22 19:24:27,620 INFO [Thread-1249] webapp.WebApps (WebApps.java:build(288)) - Registered webapp guice modules 2015-11-22 19:24:27,620 INFO [Thread-1249] http.HttpServer2 (HttpServer2.java:openListeners(917)) - Jetty bound to port 50640 2015-11-22 19:24:27,620 INFO [Thread-1249] mortbay.log (Slf4jLog.java:info(67)) - jetty-6.1.26 2015-11-22 19:24:27,633 INFO [Thread-1249] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:24:27,634 INFO [Thread[Thread-1290,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2015-11-22 19:24:27,634 INFO [Thread[Thread-1290,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:24:27,635 INFO [Thread-1249] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:24:27,636 INFO [Thread[Thread-1292,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) Nov 22, 2015 7:24:27 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.nodemanager.webapp.NMWebServices as a root resource class Nov 22, 2015 7:24:27 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a provider class Nov 22, 2015 7:24:27 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.nodemanager.webapp.JAXBContextResolver as a provider class Nov 22, 2015 7:24:27 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate INFO: Initiating Jersey application, version 'Jersey: 1.9 09/02/2011 11:17 AM' Nov 22, 2015 7:24:27 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.nodemanager.webapp.JAXBContextResolver to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:24:27 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.webapp.GenericExceptionHandler to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:24:27 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.nodemanager.webapp.NMWebServices to GuiceManagedComponentProvider with the scope "Singleton" 2015-11-22 19:24:27,731 INFO [Thread-1249] mortbay.log (Slf4jLog.java:info(67)) - Started HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:50640 2015-11-22 19:24:27,731 INFO [Thread-1249] webapp.WebApps (WebApps.java:start(307)) - Web app node started at 50640 2015-11-22 19:24:27,731 INFO [Thread-1249] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:serviceStart(222)) - Node ID assigned is : ip-172-31-20-42.ap-northeast-1.compute.internal:34720 2015-11-22 19:24:27,732 INFO [Thread-1249] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNMContainerStatuses(550)) - Sending out 0 NM container statuses: [] 2015-11-22 19:24:27,732 INFO [Thread-1249] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:registerWithRM(330)) - Registering with RM using containers :[] 2015-11-22 19:24:27,733 INFO [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(566)) - ip-172-31-20-42.ap-northeast-1.compute.internal:34720 Node Transitioned from NEW to RUNNING 2015-11-22 19:24:27,733 INFO [Thread-1249] resourcemanager.ResourceTrackerService (ResourceTrackerService.java:registerNodeManager(375)) - NodeManager from node ip-172-31-20-42.ap-northeast-1.compute.internal(cmPort: 34720 httpPort: 50640) registered with capability: , assigned nodeId ip-172-31-20-42.ap-northeast-1.compute.internal:34720 2015-11-22 19:24:27,734 INFO [Thread-1249] security.NMContainerTokenSecretManager (NMContainerTokenSecretManager.java:setMasterKey(138)) - Rolling master-key for container-tokens, got key with id 1251599360 2015-11-22 19:24:27,735 INFO [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:addNode(1427)) - Added node ip-172-31-20-42.ap-northeast-1.compute.internal:34720 clusterResource: 2015-11-22 19:24:27,735 INFO [Thread-1249] security.NMTokenSecretManagerInNM (NMTokenSecretManagerInNM.java:setMasterKey(135)) - Rolling master-key for container-tokens, got key with id -1650614532 2015-11-22 19:24:27,735 INFO [Thread-1249] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:registerWithRM(391)) - Registered with ResourceManager as ip-172-31-20-42.ap-northeast-1.compute.internal:34720 with total resource of 2015-11-22 19:24:27,736 INFO [Thread-1249] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:registerWithRM(392)) - Notifying ContainerManager to unblock new container-requests 2015-11-22 19:24:28,584 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(728)) - Waiting for Timeline Server to start... 2015-11-22 19:24:28,585 INFO [Thread-1295] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:24:28,585 INFO [Thread[Thread-1297,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2015-11-22 19:24:28,586 INFO [Thread[Thread-1297,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:24:28,586 INFO [Thread-1295] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:24:28,586 INFO [Socket Reader #1 for port 48317] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 48317 2015-11-22 19:24:28,588 INFO [Thread-1295] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.api.ApplicationHistoryProtocolPB to the server 2015-11-22 19:24:28,589 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:24:28,589 INFO [IPC Server listener on 48317] ipc.Server (Server.java:run(794)) - IPC Server listener on 48317: starting 2015-11-22 19:24:28,591 INFO [Thread-1295] applicationhistoryservice.ApplicationHistoryClientService (ApplicationHistoryClientService.java:serviceStart(111)) - Instantiated ApplicationHistoryClientService at ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:48317 2015-11-22 19:24:28,592 INFO [Thread-1295] applicationhistoryservice.ApplicationHistoryManagerImpl (ApplicationHistoryManagerImpl.java:serviceStart(71)) - Starting ApplicationHistory 2015-11-22 19:24:28,593 INFO [Thread-1295] server.AuthenticationFilter (AuthenticationFilter.java:constructSecretProvider(304)) - Unable to initialize FileSignerSecretProvider, falling back to use random secrets. 2015-11-22 19:24:28,594 INFO [Thread-1295] http.HttpRequestLog (HttpRequestLog.java:getRequestLog(80)) - Http request log for http.requests.applicationhistory is not defined 2015-11-22 19:24:28,594 INFO [Thread-1295] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter) 2015-11-22 19:24:28,595 INFO [Thread-1295] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context applicationhistory 2015-11-22 19:24:28,595 INFO [Thread-1295] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context logs 2015-11-22 19:24:28,595 INFO [Thread-1295] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context static 2015-11-22 19:24:28,595 INFO [Thread-1295] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context applicationhistory 2015-11-22 19:24:28,595 INFO [Thread-1295] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2015-11-22 19:24:28,595 INFO [Thread-1295] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2015-11-22 19:24:28,596 INFO [Thread-1295] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'Timeline Authentication Filter' (class=org.apache.hadoop.yarn.server.timeline.security.TimelineAuthenticationFilter) 2015-11-22 19:24:28,597 INFO [Thread-1295] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /applicationhistory/* 2015-11-22 19:24:28,597 INFO [Thread-1295] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /ws/* Nov 22, 2015 7:24:28 PM com.google.inject.servlet.GuiceFilter setPipeline WARNING: Multiple Servlet injectors detected. This is a warning indicating that you have more than one GuiceFilter running in your web application. If this is deliberate, you may safely ignore this message. If this is NOT deliberate however, your application may not work as expected. 2015-11-22 19:24:28,603 INFO [Thread-1295] webapp.WebApps (WebApps.java:build(288)) - Registered webapp guice modules 2015-11-22 19:24:28,603 INFO [Thread-1295] http.HttpServer2 (HttpServer2.java:openListeners(917)) - Jetty bound to port 43454 2015-11-22 19:24:28,603 INFO [Thread-1295] mortbay.log (Slf4jLog.java:info(67)) - jetty-6.1.26 2015-11-22 19:24:28,640 INFO [Thread-1295] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:24:28,640 INFO [Thread[Thread-1315,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2015-11-22 19:24:28,641 INFO [Thread[Thread-1315,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:24:28,641 INFO [Thread-1295] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:24:28,642 INFO [Thread[Thread-1317,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) Nov 22, 2015 7:24:28 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider as a provider class Nov 22, 2015 7:24:28 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.AHSWebServices as a root resource class Nov 22, 2015 7:24:28 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.timeline.webapp.TimelineWebServices as a root resource class Nov 22, 2015 7:24:28 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a provider class Nov 22, 2015 7:24:28 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate INFO: Initiating Jersey application, version 'Jersey: 1.9 09/02/2011 11:17 AM' Nov 22, 2015 7:24:28 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.webapp.GenericExceptionHandler to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:24:28 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:24:28 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.AHSWebServices to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:24:28 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.timeline.webapp.TimelineWebServices to GuiceManagedComponentProvider with the scope "Singleton" 2015-11-22 19:24:28,745 INFO [Thread-1295] mortbay.log (Slf4jLog.java:info(67)) - Started HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:43454 2015-11-22 19:24:28,746 INFO [Thread-1295] applicationhistoryservice.ApplicationHistoryServer (ApplicationHistoryServer.java:startWebApp(342)) - Instantiating AHSWebApp at 43454 2015-11-22 19:24:30,084 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(739)) - MiniYARN ApplicationHistoryServer address: ip-172-31-20-42.ap-northeast-1.compute.internal:48317 2015-11-22 19:24:30,084 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(741)) - MiniYARN ApplicationHistoryServer web address: ip-172-31-20-42.ap-northeast-1.compute.internal:43454 2015-11-22 19:24:32,103 INFO [Thread-1319] distributedshell.TestDistributedShell (TestDistributedShell.java:testDSShellWithMultipleArgs(565)) - Initializing DS Client 2015-11-22 19:24:32,153 INFO [Thread-1319] impl.TimelineClientImpl (TimelineClientImpl.java:serviceInit(296)) - Timeline service address: http://ip-172-31-20-42.ap-northeast-1.compute.internal:43454/ws/v1/timeline/ 2015-11-22 19:24:32,154 INFO [Thread-1319] distributedshell.TestDistributedShell (TestDistributedShell.java:testDSShellWithMultipleArgs(570)) - Running DS Client 2015-11-22 19:24:32,154 INFO [Thread-1319] distributedshell.Client (Client.java:run(443)) - Running Client 2015-11-22 19:24:32,164 INFO [Thread-1319] client.RMProxy (RMProxy.java:createRMProxy(98)) - Connecting to ResourceManager at ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:50548 2015-11-22 19:24:32,168 INFO [Thread-1319] distributedshell.Client (Client.java:run(447)) - Got Cluster metric info from ASM, numNodeManagers=1 2015-11-22 19:24:32,170 INFO [Thread-1319] distributedshell.Client (Client.java:run(452)) - Got Cluster node info from ASM 2015-11-22 19:24:32,170 INFO [Thread-1319] distributedshell.Client (Client.java:run(454)) - Got node report from ASM for, nodeId=ip-172-31-20-42.ap-northeast-1.compute.internal:34720, nodeAddress=ip-172-31-20-42.ap-northeast-1.compute.internal:50640, nodeRackName=/default-rack, nodeNumContainers=0 2015-11-22 19:24:32,172 INFO [Thread-1319] distributedshell.Client (Client.java:run(462)) - Queue info, queueName=default, queueCurrentCapacity=0.0, queueMaxCapacity=1.0, queueApplicationCount=0, queueChildQueueCount=0 2015-11-22 19:24:32,173 INFO [Thread-1319] distributedshell.Client (Client.java:run(472)) - User ACL Info for Queue, queueName=root, userAcl=SUBMIT_APPLICATIONS 2015-11-22 19:24:32,173 INFO [Thread-1319] distributedshell.Client (Client.java:run(472)) - User ACL Info for Queue, queueName=root, userAcl=ADMINISTER_QUEUE 2015-11-22 19:24:32,173 INFO [Thread-1319] distributedshell.Client (Client.java:run(472)) - User ACL Info for Queue, queueName=default, userAcl=SUBMIT_APPLICATIONS 2015-11-22 19:24:32,173 INFO [Thread-1319] distributedshell.Client (Client.java:run(472)) - User ACL Info for Queue, queueName=default, userAcl=ADMINISTER_QUEUE 2015-11-22 19:24:32,174 INFO [IPC Server handler 4 on 50548] resourcemanager.ClientRMService (ClientRMService.java:getNewApplicationId(305)) - Allocated new applicationId: 1 2015-11-22 19:24:32,174 INFO [Thread-1319] distributedshell.Client (Client.java:run(491)) - Max mem capability of resources in this cluster 4096 2015-11-22 19:24:32,174 INFO [Thread-1319] distributedshell.Client (Client.java:run(502)) - Max virtual cores capability of resources in this cluster 4 2015-11-22 19:24:32,174 INFO [Thread-1319] distributedshell.Client (Client.java:run(528)) - Copy App Master jar from local filesystem and add to local environment 2015-11-22 19:24:32,185 INFO [Thread-1319] distributedshell.Client (Client.java:run(577)) - Set the environment for the application master 2015-11-22 19:24:32,186 INFO [Thread-1319] distributedshell.Client (Client.java:run(619)) - Setting up app master command 2015-11-22 19:24:32,186 INFO [Thread-1319] distributedshell.Client (Client.java:run(650)) - Completed setting up app master command {{JAVA_HOME}}/bin/java -Xmx512m org.apache.hadoop.yarn.applications.distributedshell.ApplicationMaster --container_memory 128 --container_vcores 1 --num_containers 4 --priority 0 1>/AppMaster.stdout 2>/AppMaster.stderr 2015-11-22 19:24:32,186 INFO [Thread-1319] distributedshell.Client (Client.java:run(706)) - Submitting application to ASM 2015-11-22 19:24:32,187 INFO [IPC Server handler 5 on 50548] capacity.CapacityScheduler (CapacityScheduler.java:checkAndGetApplicationPriority(1941)) - Priority '0' is acceptable in queue : default for application: application_1448220266082_0001 for the user: ubuntu 2015-11-22 19:24:32,187 WARN [IPC Server handler 5 on 50548] rmapp.RMAppImpl (RMAppImpl.java:(420)) - The specific max attempts: 0 for application: 1 is invalid, because it is out of the range [1, 2]. Use the global max attempts instead. 2015-11-22 19:24:32,187 INFO [IPC Server handler 5 on 50548] resourcemanager.ClientRMService (ClientRMService.java:submitApplication(602)) - Application with id 1 submitted by user ubuntu 2015-11-22 19:24:32,187 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:transition(1079)) - Storing application with id application_1448220266082_0001 2015-11-22 19:24:32,187 INFO [IPC Server handler 5 on 50548] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(154)) - USER=ubuntu IP=172.31.20.42 OPERATION=Submit Application Request TARGET=ClientRMService RESULT=SUCCESS APPID=application_1448220266082_0001 2015-11-22 19:24:32,187 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220266082_0001 State change from NEW to NEW_SAVING 2015-11-22 19:24:32,187 INFO [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:transition(208)) - Storing info for app: application_1448220266082_0001 2015-11-22 19:24:32,188 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220266082_0001 State change from NEW_SAVING to SUBMITTED 2015-11-22 19:24:32,188 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:addApplication(349)) - Application added - appId: application_1448220266082_0001 user: ubuntu leaf-queue of parent: root #applications: 1 2015-11-22 19:24:32,188 INFO [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:addApplication(765)) - Accepted application application_1448220266082_0001 from user: ubuntu, in queue: default 2015-11-22 19:24:32,188 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220266082_0001 State change from SUBMITTED to ACCEPTED 2015-11-22 19:24:32,189 INFO [AsyncDispatcher event handler] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:registerAppAttempt(661)) - Registering app attempt : appattempt_1448220266082_0001_000001 2015-11-22 19:24:32,189 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220266082_0001_000001 State change from NEW to SUBMITTED 2015-11-22 19:24:32,189 INFO [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:activateApplications(705)) - Application application_1448220266082_0001 from user: ubuntu activated in queue: default 2015-11-22 19:24:32,190 INFO [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:addApplicationAttempt(720)) - Application added - appId: application_1448220266082_0001 user: ubuntu, leaf-queue: default #user-pending-applications: 0 #user-active-applications: 1 #queue-pending-applications: 0 #queue-active-applications: 1 2015-11-22 19:24:32,190 INFO [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:addApplicationAttempt(805)) - Added Application Attempt appattempt_1448220266082_0001_000001 to scheduler from user ubuntu in queue default 2015-11-22 19:24:32,190 INFO [Thread-1319] impl.YarnClientImpl (YarnClientImpl.java:submitApplication(279)) - Submitted application application_1448220266082_0001 2015-11-22 19:24:32,191 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220266082_0001_000001 State change from SUBMITTED to SCHEDULED 2015-11-22 19:24:32,745 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220266082_0001_01_000001 Container Transitioned from NEW to ALLOCATED 2015-11-22 19:24:32,745 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220266082_0001 CONTAINERID=container_1448220266082_0001_01_000001 2015-11-22 19:24:32,745 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(154)) - Assigned container container_1448220266082_0001_01_000001 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:34720, which has 1 containers, used and available after allocation 2015-11-22 19:24:32,745 INFO [ResourceManager Event Processor] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(89)) - assignedContainer application attempt=appattempt_1448220266082_0001_000001 container=container_1448220266082_0001_01_000001 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@dccea8 clusterResource= type=OFF_SWITCH 2015-11-22 19:24:32,746 INFO [AsyncDispatcher event handler] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:createAndGetNMToken(200)) - Sending NMToken for nodeId : ip-172-31-20-42.ap-northeast-1.compute.internal:34720 for container : container_1448220266082_0001_01_000001 2015-11-22 19:24:32,747 INFO [AsyncDispatcher event handler] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220266082_0001_01_000001 Container Transitioned from ALLOCATED to ACQUIRED 2015-11-22 19:24:32,747 INFO [AsyncDispatcher event handler] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:clearNodeSetForAttempt(146)) - Clear node set for appattempt_1448220266082_0001_000001 2015-11-22 19:24:32,748 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(589)) - Re-sorting assigned queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.125, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 2015-11-22 19:24:32,748 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:storeAttempt(2026)) - Storing attempt: AppId: application_1448220266082_0001 AttemptId: appattempt_1448220266082_0001_000001 MasterContainer: Container: [ContainerId: container_1448220266082_0001_01_000001, NodeId: ip-172-31-20-42.ap-northeast-1.compute.internal:34720, NodeHttpAddress: ip-172-31-20-42.ap-northeast-1.compute.internal:50640, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.31.20.42:34720 }, ] 2015-11-22 19:24:32,748 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(461)) - assignedContainer queue=root usedCapacity=0.125 absoluteUsedCapacity=0.125 used= cluster= 2015-11-22 19:24:32,748 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220266082_0001_000001 State change from SCHEDULED to ALLOCATED_SAVING 2015-11-22 19:24:32,748 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220266082_0001_000001 State change from ALLOCATED_SAVING to ALLOCATED 2015-11-22 19:24:32,749 INFO [ApplicationMasterLauncher #0] amlauncher.AMLauncher (AMLauncher.java:run(254)) - Launching masterappattempt_1448220266082_0001_000001 2015-11-22 19:24:32,751 INFO [ApplicationMasterLauncher #0] amlauncher.AMLauncher (AMLauncher.java:launch(106)) - Setting up container Container: [ContainerId: container_1448220266082_0001_01_000001, NodeId: ip-172-31-20-42.ap-northeast-1.compute.internal:34720, NodeHttpAddress: ip-172-31-20-42.ap-northeast-1.compute.internal:50640, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.31.20.42:34720 }, ] for AM appattempt_1448220266082_0001_000001 2015-11-22 19:24:32,751 INFO [ApplicationMasterLauncher #0] amlauncher.AMLauncher (AMLauncher.java:createAMContainerLaunchContext(188)) - Command to launch container container_1448220266082_0001_01_000001 : {{JAVA_HOME}}/bin/java -Xmx512m org.apache.hadoop.yarn.applications.distributedshell.ApplicationMaster --container_memory 128 --container_vcores 1 --num_containers 4 --priority 0 1>/AppMaster.stdout 2>/AppMaster.stderr 2015-11-22 19:24:32,751 INFO [ApplicationMasterLauncher #0] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:createAndGetAMRMToken(195)) - Create AMRMToken for ApplicationAttempt: appattempt_1448220266082_0001_000001 2015-11-22 19:24:32,751 INFO [ApplicationMasterLauncher #0] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:createPassword(307)) - Creating password for appattempt_1448220266082_0001_000001 2015-11-22 19:24:32,754 INFO [Socket Reader #1 for port 34720] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220266082_0001_000001 (auth:SIMPLE) 2015-11-22 19:24:32,759 INFO [IPC Server handler 0 on 34720] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:startContainerInternal(889)) - Start request for container_1448220266082_0001_01_000001 by user ubuntu 2015-11-22 19:24:32,760 INFO [IPC Server handler 0 on 34720] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:startContainerInternal(930)) - Creating a new application reference for app application_1448220266082_0001 2015-11-22 19:24:32,760 INFO [IPC Server handler 0 on 34720] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu IP=172.31.20.42 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1448220266082_0001 CONTAINERID=container_1448220266082_0001_01_000001 2015-11-22 19:24:32,762 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(463)) - Application application_1448220266082_0001 transitioned from NEW to INITING 2015-11-22 19:24:32,762 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(303)) - Adding container_1448220266082_0001_01_000001 to application application_1448220266082_0001 2015-11-22 19:24:32,763 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(463)) - Application application_1448220266082_0001 transitioned from INITING to RUNNING 2015-11-22 19:24:32,763 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220266082_0001_01_000001 transitioned from NEW to LOCALIZING 2015-11-22 19:24:32,763 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_INIT for appId application_1448220266082_0001 2015-11-22 19:24:32,763 INFO [AsyncDispatcher event handler] localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource file:/home/ubuntu/DistributedShell/application_1448220266082_0001/AppMaster.jar transitioned from INIT to DOWNLOADING 2015-11-22 19:24:32,763 INFO [AsyncDispatcher event handler] localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource file:/home/ubuntu/DistributedShell/application_1448220266082_0001/shellCommands transitioned from INIT to DOWNLOADING 2015-11-22 19:24:32,764 INFO [AsyncDispatcher event handler] localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource file:/home/ubuntu/DistributedShell/application_1448220266082_0001/shellArgs transitioned from INIT to DOWNLOADING 2015-11-22 19:24:32,764 INFO [AsyncDispatcher event handler] localizer.ResourceLocalizationService (ResourceLocalizationService.java:handle(742)) - Created localizer for container_1448220266082_0001_01_000001 2015-11-22 19:24:32,766 INFO [ApplicationMasterLauncher #0] amlauncher.AMLauncher (AMLauncher.java:launch(127)) - Done launching container Container: [ContainerId: container_1448220266082_0001_01_000001, NodeId: ip-172-31-20-42.ap-northeast-1.compute.internal:34720, NodeHttpAddress: ip-172-31-20-42.ap-northeast-1.compute.internal:50640, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.31.20.42:34720 }, ] for AM appattempt_1448220266082_0001_000001 2015-11-22 19:24:32,766 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220266082_0001_000001 State change from ALLOCATED to LAUNCHED 2015-11-22 19:24:32,768 INFO [LocalizerRunner for container_1448220266082_0001_01_000001] localizer.ResourceLocalizationService (ResourceLocalizationService.java:writeCredentials(1221)) - Writing credentials to the nmPrivate file /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/container_1448220266082_0001_01_000001.tokens. Credentials list: 2015-11-22 19:24:32,769 INFO [LocalizerRunner for container_1448220266082_0001_01_000001] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:createUserCacheDirs(632)) - Initializing user ubuntu 2015-11-22 19:24:32,777 INFO [LocalizerRunner for container_1448220266082_0001_01_000001] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:startLocalizer(126)) - Copying from /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/container_1448220266082_0001_01_000001.tokens to /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220266082_0001/container_1448220266082_0001_01_000001.tokens 2015-11-22 19:24:32,777 INFO [LocalizerRunner for container_1448220266082_0001_01_000001] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:startLocalizer(133)) - Localizer CWD set to /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220266082_0001 = file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220266082_0001 2015-11-22 19:24:32,812 INFO [IPC Server handler 1 on 43758] localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource file:/home/ubuntu/DistributedShell/application_1448220266082_0001/AppMaster.jar(->/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220266082_0001/filecache/10/AppMaster.jar) transitioned from DOWNLOADING to LOCALIZED 2015-11-22 19:24:32,832 INFO [IPC Server handler 4 on 43758] localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource file:/home/ubuntu/DistributedShell/application_1448220266082_0001/shellCommands(->/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220266082_0001/filecache/11/shellCommands) transitioned from DOWNLOADING to LOCALIZED 2015-11-22 19:24:32,849 INFO [IPC Server handler 3 on 43758] localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource file:/home/ubuntu/DistributedShell/application_1448220266082_0001/shellArgs(->/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220266082_0001/filecache/12/shellArgs) transitioned from DOWNLOADING to LOCALIZED 2015-11-22 19:24:32,849 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220266082_0001_01_000001 transitioned from LOCALIZING to LOCALIZED 2015-11-22 19:24:32,868 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220266082_0001_01_000001 transitioned from LOCALIZED to RUNNING 2015-11-22 19:24:32,885 INFO [ContainersLauncher #0] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:buildCommandExecutor(281)) - launchContainer: [nice, -n, 0, bash, /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220266082_0001/container_1448220266082_0001_01_000001/default_container_executor.sh] 2015-11-22 19:24:33,193 INFO [Thread-1319] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=N/A, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220272187, yarnAppState=ACCEPTED, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:36077/proxy/application_1448220266082_0001/, appUser=ubuntu 2015-11-22 19:24:33,744 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220266082_0001_01_000001 Container Transitioned from ACQUIRED to RUNNING 2015-11-22 19:24:34,195 INFO [Thread-1319] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=N/A, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220272187, yarnAppState=ACCEPTED, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:36077/proxy/application_1448220266082_0001/, appUser=ubuntu 2015-11-22 19:24:35,197 INFO [Thread-1319] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=N/A, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220272187, yarnAppState=ACCEPTED, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:36077/proxy/application_1448220266082_0001/, appUser=ubuntu 2015-11-22 19:24:35,348 INFO [Socket Reader #1 for port 32866] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220266082_0001_000001 (auth:SIMPLE) 2015-11-22 19:24:35,356 INFO [IPC Server handler 0 on 32866] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:registerApplicationMaster(220)) - AM registration appattempt_1448220266082_0001_000001 2015-11-22 19:24:35,357 INFO [IPC Server handler 0 on 32866] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(133)) - USER=ubuntu IP=172.31.20.42 OPERATION=Register App Master TARGET=ApplicationMasterService RESULT=SUCCESS APPID=application_1448220266082_0001 APPATTEMPTID=appattempt_1448220266082_0001_000001 2015-11-22 19:24:35,357 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220266082_0001_000001 State change from LAUNCHED to RUNNING 2015-11-22 19:24:35,357 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220266082_0001 State change from ACCEPTED to RUNNING 2015-11-22 19:24:36,198 INFO [Thread-1319] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220272187, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:36077/proxy/application_1448220266082_0001/, appUser=ubuntu 2015-11-22 19:24:36,746 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220266082_0001_01_000002 Container Transitioned from NEW to ALLOCATED 2015-11-22 19:24:36,746 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220266082_0001 CONTAINERID=container_1448220266082_0001_01_000002 2015-11-22 19:24:36,746 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(154)) - Assigned container container_1448220266082_0001_01_000002 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:34720, which has 2 containers, used and available after allocation 2015-11-22 19:24:36,747 INFO [ResourceManager Event Processor] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(89)) - assignedContainer application attempt=appattempt_1448220266082_0001_000001 container=container_1448220266082_0001_01_000002 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@dccea8 clusterResource= type=OFF_SWITCH 2015-11-22 19:24:36,747 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(589)) - Re-sorting assigned queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.15625, absoluteUsedCapacity=0.15625, numApps=1, numContainers=2 2015-11-22 19:24:36,747 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(461)) - assignedContainer queue=root usedCapacity=0.15625 absoluteUsedCapacity=0.15625 used= cluster= 2015-11-22 19:24:37,200 INFO [Thread-1319] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220272187, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:36077/proxy/application_1448220266082_0001/, appUser=ubuntu 2015-11-22 19:24:37,400 INFO [IPC Server handler 2 on 32866] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:createAndGetNMToken(200)) - Sending NMToken for nodeId : ip-172-31-20-42.ap-northeast-1.compute.internal:34720 for container : container_1448220266082_0001_01_000002 2015-11-22 19:24:37,400 INFO [IPC Server handler 2 on 32866] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220266082_0001_01_000002 Container Transitioned from ALLOCATED to ACQUIRED 2015-11-22 19:24:37,468 INFO [Socket Reader #1 for port 34720] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220266082_0001_000001 (auth:SIMPLE) 2015-11-22 19:24:37,473 INFO [IPC Server handler 1 on 34720] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:startContainerInternal(889)) - Start request for container_1448220266082_0001_01_000002 by user ubuntu 2015-11-22 19:24:37,473 INFO [IPC Server handler 1 on 34720] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu IP=172.31.20.42 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1448220266082_0001 CONTAINERID=container_1448220266082_0001_01_000002 2015-11-22 19:24:37,473 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(303)) - Adding container_1448220266082_0001_01_000002 to application application_1448220266082_0001 2015-11-22 19:24:37,474 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220266082_0001_01_000002 transitioned from NEW to LOCALIZED 2015-11-22 19:24:37,474 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_INIT for appId application_1448220266082_0001 2015-11-22 19:24:37,505 INFO [Socket Reader #1 for port 34720] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220266082_0001_000001 (auth:SIMPLE) 2015-11-22 19:24:37,546 INFO [IPC Server handler 2 on 34720] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:getContainerStatusInternal(1221)) - Getting container-status for container_1448220266082_0001_01_000002 2015-11-22 19:24:37,547 INFO [IPC Server handler 2 on 34720] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:getContainerStatusInternal(1235)) - Returning ContainerStatus: [ContainerId: container_1448220266082_0001_01_000002, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ] 2015-11-22 19:24:37,552 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220266082_0001_01_000002 transitioned from LOCALIZED to RUNNING 2015-11-22 19:24:37,606 INFO [ContainersLauncher #1] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:buildCommandExecutor(281)) - launchContainer: [nice, -n, 0, bash, /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220266082_0001/container_1448220266082_0001_01_000002/default_container_executor.sh] 2015-11-22 19:24:37,614 INFO [ContainersLauncher #1] launcher.ContainerLaunch (ContainerLaunch.java:call(356)) - Container container_1448220266082_0001_01_000002 succeeded 2015-11-22 19:24:37,614 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220266082_0001_01_000002 transitioned from RUNNING to EXITED_WITH_SUCCESS 2015-11-22 19:24:37,614 INFO [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(380)) - Cleaning up container container_1448220266082_0001_01_000002 2015-11-22 19:24:37,627 INFO [DeletionService #0] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(480)) - Deleting absolute path : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220266082_0001/container_1448220266082_0001_01_000002 2015-11-22 19:24:37,627 INFO [AsyncDispatcher event handler] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu OPERATION=Container Finished - Succeeded TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1448220266082_0001 CONTAINERID=container_1448220266082_0001_01_000002 2015-11-22 19:24:37,628 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220266082_0001_01_000002 transitioned from EXITED_WITH_SUCCESS to DONE 2015-11-22 19:24:37,628 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(346)) - Removing container_1448220266082_0001_01_000002 from application application_1448220266082_0001 2015-11-22 19:24:37,628 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_STOP for appId application_1448220266082_0001 2015-11-22 19:24:37,746 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220266082_0001_01_000002 Container Transitioned from ACQUIRED to COMPLETED 2015-11-22 19:24:37,746 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220266082_0001 CONTAINERID=container_1448220266082_0001_01_000002 2015-11-22 19:24:37,747 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:releaseContainer(248)) - Released container container_1448220266082_0001_01_000002 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:34720, which currently has 1 containers, used and available, release resources=true 2015-11-22 19:24:37,747 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220266082_0001_01_000003 Container Transitioned from NEW to ALLOCATED 2015-11-22 19:24:37,747 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220266082_0001 CONTAINERID=container_1448220266082_0001_01_000003 2015-11-22 19:24:37,747 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(154)) - Assigned container container_1448220266082_0001_01_000003 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:34720, which has 2 containers, used and available after allocation 2015-11-22 19:24:37,747 INFO [ResourceManager Event Processor] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(89)) - assignedContainer application attempt=appattempt_1448220266082_0001_000001 container=container_1448220266082_0001_01_000003 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@dccea8 clusterResource= type=OFF_SWITCH 2015-11-22 19:24:37,748 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(589)) - Re-sorting assigned queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.15625, absoluteUsedCapacity=0.15625, numApps=1, numContainers=2 2015-11-22 19:24:37,748 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(461)) - assignedContainer queue=root usedCapacity=0.15625 absoluteUsedCapacity=0.15625 used= cluster= 2015-11-22 19:24:38,202 INFO [Thread-1319] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220272187, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:36077/proxy/application_1448220266082_0001/, appUser=ubuntu 2015-11-22 19:24:38,412 INFO [IPC Server handler 4 on 32866] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220266082_0001_01_000003 Container Transitioned from ALLOCATED to ACQUIRED 2015-11-22 19:24:38,460 INFO [Socket Reader #1 for port 34720] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220266082_0001_000001 (auth:SIMPLE) 2015-11-22 19:24:38,466 INFO [IPC Server handler 1 on 34720] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:startContainerInternal(889)) - Start request for container_1448220266082_0001_01_000003 by user ubuntu 2015-11-22 19:24:38,466 INFO [IPC Server handler 1 on 34720] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu IP=172.31.20.42 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1448220266082_0001 CONTAINERID=container_1448220266082_0001_01_000003 2015-11-22 19:24:38,466 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(303)) - Adding container_1448220266082_0001_01_000003 to application application_1448220266082_0001 2015-11-22 19:24:38,467 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220266082_0001_01_000003 transitioned from NEW to LOCALIZED 2015-11-22 19:24:38,467 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_INIT for appId application_1448220266082_0001 2015-11-22 19:24:38,496 INFO [Socket Reader #1 for port 34720] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220266082_0001_000001 (auth:SIMPLE) 2015-11-22 19:24:38,507 INFO [IPC Server handler 2 on 34720] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:getContainerStatusInternal(1221)) - Getting container-status for container_1448220266082_0001_01_000003 2015-11-22 19:24:38,507 INFO [IPC Server handler 2 on 34720] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:getContainerStatusInternal(1235)) - Returning ContainerStatus: [ContainerId: container_1448220266082_0001_01_000003, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ] 2015-11-22 19:24:38,524 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220266082_0001_01_000003 transitioned from LOCALIZED to RUNNING 2015-11-22 19:24:38,546 INFO [ContainersLauncher #1] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:buildCommandExecutor(281)) - launchContainer: [nice, -n, 0, bash, /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220266082_0001/container_1448220266082_0001_01_000003/default_container_executor.sh] 2015-11-22 19:24:38,554 INFO [ContainersLauncher #1] launcher.ContainerLaunch (ContainerLaunch.java:call(356)) - Container container_1448220266082_0001_01_000003 succeeded 2015-11-22 19:24:38,554 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220266082_0001_01_000003 transitioned from RUNNING to EXITED_WITH_SUCCESS 2015-11-22 19:24:38,554 INFO [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(380)) - Cleaning up container container_1448220266082_0001_01_000003 2015-11-22 19:24:38,567 INFO [DeletionService #2] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(480)) - Deleting absolute path : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220266082_0001/container_1448220266082_0001_01_000003 2015-11-22 19:24:38,567 INFO [AsyncDispatcher event handler] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu OPERATION=Container Finished - Succeeded TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1448220266082_0001 CONTAINERID=container_1448220266082_0001_01_000003 2015-11-22 19:24:38,567 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220266082_0001_01_000003 transitioned from EXITED_WITH_SUCCESS to DONE 2015-11-22 19:24:38,567 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(346)) - Removing container_1448220266082_0001_01_000003 from application application_1448220266082_0001 2015-11-22 19:24:38,567 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_STOP for appId application_1448220266082_0001 2015-11-22 19:24:38,747 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220266082_0001_01_000003 Container Transitioned from ACQUIRED to COMPLETED 2015-11-22 19:24:38,747 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220266082_0001 CONTAINERID=container_1448220266082_0001_01_000003 2015-11-22 19:24:38,747 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:releaseContainer(248)) - Released container container_1448220266082_0001_01_000003 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:34720, which currently has 1 containers, used and available, release resources=true 2015-11-22 19:24:38,748 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220266082_0001_01_000004 Container Transitioned from NEW to ALLOCATED 2015-11-22 19:24:38,748 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220266082_0001 CONTAINERID=container_1448220266082_0001_01_000004 2015-11-22 19:24:38,748 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(154)) - Assigned container container_1448220266082_0001_01_000004 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:34720, which has 2 containers, used and available after allocation 2015-11-22 19:24:38,748 INFO [ResourceManager Event Processor] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(89)) - assignedContainer application attempt=appattempt_1448220266082_0001_000001 container=container_1448220266082_0001_01_000004 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@dccea8 clusterResource= type=OFF_SWITCH 2015-11-22 19:24:38,749 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(589)) - Re-sorting assigned queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.15625, absoluteUsedCapacity=0.15625, numApps=1, numContainers=2 2015-11-22 19:24:38,749 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(461)) - assignedContainer queue=root usedCapacity=0.15625 absoluteUsedCapacity=0.15625 used= cluster= 2015-11-22 19:24:39,204 INFO [Thread-1319] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220272187, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:36077/proxy/application_1448220266082_0001/, appUser=ubuntu 2015-11-22 19:24:39,416 INFO [IPC Server handler 5 on 32866] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220266082_0001_01_000004 Container Transitioned from ALLOCATED to ACQUIRED 2015-11-22 19:24:39,462 INFO [Socket Reader #1 for port 34720] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220266082_0001_000001 (auth:SIMPLE) 2015-11-22 19:24:39,466 INFO [IPC Server handler 1 on 34720] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:startContainerInternal(889)) - Start request for container_1448220266082_0001_01_000004 by user ubuntu 2015-11-22 19:24:39,466 INFO [IPC Server handler 1 on 34720] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu IP=172.31.20.42 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1448220266082_0001 CONTAINERID=container_1448220266082_0001_01_000004 2015-11-22 19:24:39,467 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(303)) - Adding container_1448220266082_0001_01_000004 to application application_1448220266082_0001 2015-11-22 19:24:39,467 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220266082_0001_01_000004 transitioned from NEW to LOCALIZED 2015-11-22 19:24:39,467 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_INIT for appId application_1448220266082_0001 2015-11-22 19:24:39,489 INFO [Socket Reader #1 for port 34720] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220266082_0001_000001 (auth:SIMPLE) 2015-11-22 19:24:39,505 INFO [IPC Server handler 2 on 34720] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:getContainerStatusInternal(1221)) - Getting container-status for container_1448220266082_0001_01_000004 2015-11-22 19:24:39,505 INFO [IPC Server handler 2 on 34720] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:getContainerStatusInternal(1235)) - Returning ContainerStatus: [ContainerId: container_1448220266082_0001_01_000004, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ] 2015-11-22 19:24:39,506 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220266082_0001_01_000004 transitioned from LOCALIZED to RUNNING 2015-11-22 19:24:39,538 INFO [ContainersLauncher #1] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:buildCommandExecutor(281)) - launchContainer: [nice, -n, 0, bash, /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220266082_0001/container_1448220266082_0001_01_000004/default_container_executor.sh] 2015-11-22 19:24:39,557 INFO [ContainersLauncher #1] launcher.ContainerLaunch (ContainerLaunch.java:call(356)) - Container container_1448220266082_0001_01_000004 succeeded 2015-11-22 19:24:39,557 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220266082_0001_01_000004 transitioned from RUNNING to EXITED_WITH_SUCCESS 2015-11-22 19:24:39,557 INFO [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(380)) - Cleaning up container container_1448220266082_0001_01_000004 2015-11-22 19:24:39,580 INFO [DeletionService #0] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(480)) - Deleting absolute path : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220266082_0001/container_1448220266082_0001_01_000004 2015-11-22 19:24:39,581 INFO [AsyncDispatcher event handler] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu OPERATION=Container Finished - Succeeded TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1448220266082_0001 CONTAINERID=container_1448220266082_0001_01_000004 2015-11-22 19:24:39,581 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220266082_0001_01_000004 transitioned from EXITED_WITH_SUCCESS to DONE 2015-11-22 19:24:39,581 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(346)) - Removing container_1448220266082_0001_01_000004 from application application_1448220266082_0001 2015-11-22 19:24:39,582 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_STOP for appId application_1448220266082_0001 2015-11-22 19:24:39,747 INFO [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:removeOrTrackCompletedContainersFromContext(605)) - Removed completed containers from NM context: [container_1448220266082_0001_01_000002] 2015-11-22 19:24:39,748 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220266082_0001_01_000004 Container Transitioned from ACQUIRED to COMPLETED 2015-11-22 19:24:39,748 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220266082_0001 CONTAINERID=container_1448220266082_0001_01_000004 2015-11-22 19:24:39,748 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:releaseContainer(248)) - Released container container_1448220266082_0001_01_000004 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:34720, which currently has 1 containers, used and available, release resources=true 2015-11-22 19:24:39,748 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220266082_0001_01_000005 Container Transitioned from NEW to ALLOCATED 2015-11-22 19:24:39,749 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220266082_0001 CONTAINERID=container_1448220266082_0001_01_000005 2015-11-22 19:24:39,749 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(154)) - Assigned container container_1448220266082_0001_01_000005 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:34720, which has 2 containers, used and available after allocation 2015-11-22 19:24:39,749 INFO [ResourceManager Event Processor] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(89)) - assignedContainer application attempt=appattempt_1448220266082_0001_000001 container=container_1448220266082_0001_01_000005 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@dccea8 clusterResource= type=OFF_SWITCH 2015-11-22 19:24:39,749 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(589)) - Re-sorting assigned queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.15625, absoluteUsedCapacity=0.15625, numApps=1, numContainers=2 2015-11-22 19:24:39,749 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(461)) - assignedContainer queue=root usedCapacity=0.15625 absoluteUsedCapacity=0.15625 used= cluster= 2015-11-22 19:24:40,206 INFO [Thread-1319] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220272187, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:36077/proxy/application_1448220266082_0001/, appUser=ubuntu 2015-11-22 19:24:40,419 INFO [IPC Server handler 6 on 32866] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220266082_0001_01_000005 Container Transitioned from ALLOCATED to ACQUIRED 2015-11-22 19:24:40,480 INFO [Socket Reader #1 for port 34720] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220266082_0001_000001 (auth:SIMPLE) 2015-11-22 19:24:40,488 INFO [IPC Server handler 2 on 34720] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:startContainerInternal(889)) - Start request for container_1448220266082_0001_01_000005 by user ubuntu 2015-11-22 19:24:40,488 INFO [IPC Server handler 2 on 34720] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu IP=172.31.20.42 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1448220266082_0001 CONTAINERID=container_1448220266082_0001_01_000005 2015-11-22 19:24:40,488 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(303)) - Adding container_1448220266082_0001_01_000005 to application application_1448220266082_0001 2015-11-22 19:24:40,488 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220266082_0001_01_000005 transitioned from NEW to LOCALIZED 2015-11-22 19:24:40,488 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_INIT for appId application_1448220266082_0001 2015-11-22 19:24:40,502 INFO [Socket Reader #1 for port 34720] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220266082_0001_000001 (auth:SIMPLE) 2015-11-22 19:24:40,516 INFO [IPC Server handler 5 on 34720] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:getContainerStatusInternal(1221)) - Getting container-status for container_1448220266082_0001_01_000005 2015-11-22 19:24:40,516 INFO [IPC Server handler 5 on 34720] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:getContainerStatusInternal(1235)) - Returning ContainerStatus: [ContainerId: container_1448220266082_0001_01_000005, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ] 2015-11-22 19:24:40,526 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220266082_0001_01_000005 transitioned from LOCALIZED to RUNNING 2015-11-22 19:24:40,565 INFO [ContainersLauncher #1] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:buildCommandExecutor(281)) - launchContainer: [nice, -n, 0, bash, /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220266082_0001/container_1448220266082_0001_01_000005/default_container_executor.sh] 2015-11-22 19:24:40,574 INFO [ContainersLauncher #1] launcher.ContainerLaunch (ContainerLaunch.java:call(356)) - Container container_1448220266082_0001_01_000005 succeeded 2015-11-22 19:24:40,574 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220266082_0001_01_000005 transitioned from RUNNING to EXITED_WITH_SUCCESS 2015-11-22 19:24:40,574 INFO [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(380)) - Cleaning up container container_1448220266082_0001_01_000005 2015-11-22 19:24:40,588 INFO [DeletionService #2] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(480)) - Deleting absolute path : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220266082_0001/container_1448220266082_0001_01_000005 2015-11-22 19:24:40,588 INFO [AsyncDispatcher event handler] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu OPERATION=Container Finished - Succeeded TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1448220266082_0001 CONTAINERID=container_1448220266082_0001_01_000005 2015-11-22 19:24:40,588 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220266082_0001_01_000005 transitioned from EXITED_WITH_SUCCESS to DONE 2015-11-22 19:24:40,589 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(346)) - Removing container_1448220266082_0001_01_000005 from application application_1448220266082_0001 2015-11-22 19:24:40,589 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_STOP for appId application_1448220266082_0001 2015-11-22 19:24:40,748 INFO [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:removeOrTrackCompletedContainersFromContext(605)) - Removed completed containers from NM context: [container_1448220266082_0001_01_000003] 2015-11-22 19:24:40,749 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220266082_0001_01_000005 Container Transitioned from ACQUIRED to COMPLETED 2015-11-22 19:24:40,749 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220266082_0001 CONTAINERID=container_1448220266082_0001_01_000005 2015-11-22 19:24:40,749 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:releaseContainer(248)) - Released container container_1448220266082_0001_01_000005 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:34720, which currently has 1 containers, used and available, release resources=true 2015-11-22 19:24:41,208 INFO [Thread-1319] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220272187, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:36077/proxy/application_1448220266082_0001/, appUser=ubuntu 2015-11-22 19:24:41,641 INFO [Socket Reader #1 for port 34720] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220266082_0001_000001 (auth:SIMPLE) 2015-11-22 19:24:41,643 INFO [IPC Server handler 0 on 34720] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:stopContainerInternal(1163)) - Stopping container with container Id: container_1448220266082_0001_01_000003 2015-11-22 19:24:41,651 INFO [Socket Reader #1 for port 34720] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220266082_0001_000001 (auth:SIMPLE) 2015-11-22 19:24:41,654 INFO [IPC Server handler 1 on 34720] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:stopContainerInternal(1163)) - Stopping container with container Id: container_1448220266082_0001_01_000002 2015-11-22 19:24:41,659 INFO [Socket Reader #1 for port 34720] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220266082_0001_000001 (auth:SIMPLE) 2015-11-22 19:24:41,662 INFO [IPC Server handler 2 on 34720] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:stopContainerInternal(1163)) - Stopping container with container Id: container_1448220266082_0001_01_000005 2015-11-22 19:24:41,662 INFO [IPC Server handler 2 on 34720] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu IP=172.31.20.42 OPERATION=Stop Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1448220266082_0001 CONTAINERID=container_1448220266082_0001_01_000005 2015-11-22 19:24:41,663 INFO [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:removeOrTrackCompletedContainersFromContext(605)) - Removed completed containers from NM context: [container_1448220266082_0001_01_000004] 2015-11-22 19:24:41,668 INFO [Socket Reader #1 for port 34720] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220266082_0001_000001 (auth:SIMPLE) 2015-11-22 19:24:41,672 INFO [IPC Server handler 5 on 34720] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:stopContainerInternal(1163)) - Stopping container with container Id: container_1448220266082_0001_01_000004 2015-11-22 19:24:41,679 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:rememberTargetTransitionsAndStoreState(1258)) - Updating application attempt appattempt_1448220266082_0001_000001 with final state: FINISHING, and exit status: -1000 2015-11-22 19:24:41,679 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220266082_0001_000001 State change from RUNNING to FINAL_SAVING 2015-11-22 19:24:41,679 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:rememberTargetTransitionsAndStoreState(1098)) - Updating application application_1448220266082_0001 with final state: FINISHING 2015-11-22 19:24:41,679 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220266082_0001 State change from RUNNING to FINAL_SAVING 2015-11-22 19:24:41,679 INFO [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:transition(237)) - Updating info for app: application_1448220266082_0001 2015-11-22 19:24:41,679 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220266082_0001_000001 State change from FINAL_SAVING to FINISHING 2015-11-22 19:24:41,680 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220266082_0001 State change from FINAL_SAVING to FINISHING 2015-11-22 19:24:41,783 INFO [IPC Server handler 9 on 32866] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:finishApplicationMaster(300)) - application_1448220266082_0001 unregistered successfully. 2015-11-22 19:24:41,814 INFO [ContainersLauncher #0] launcher.ContainerLaunch (ContainerLaunch.java:call(356)) - Container container_1448220266082_0001_01_000001 succeeded 2015-11-22 19:24:41,815 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220266082_0001_01_000001 transitioned from RUNNING to EXITED_WITH_SUCCESS 2015-11-22 19:24:41,815 INFO [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(380)) - Cleaning up container container_1448220266082_0001_01_000001 2015-11-22 19:24:41,836 INFO [DeletionService #3] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(480)) - Deleting absolute path : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220266082_0001/container_1448220266082_0001_01_000001 2015-11-22 19:24:41,836 INFO [AsyncDispatcher event handler] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu OPERATION=Container Finished - Succeeded TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1448220266082_0001 CONTAINERID=container_1448220266082_0001_01_000001 2015-11-22 19:24:41,836 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220266082_0001_01_000001 transitioned from EXITED_WITH_SUCCESS to DONE 2015-11-22 19:24:41,837 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(346)) - Removing container_1448220266082_0001_01_000001 from application application_1448220266082_0001 2015-11-22 19:24:41,837 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_STOP for appId application_1448220266082_0001 2015-11-22 19:24:42,210 INFO [Thread-1319] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220272187, yarnAppState=FINISHED, distributedFinalState=SUCCEEDED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:36077/proxy/application_1448220266082_0001/, appUser=ubuntu 2015-11-22 19:24:42,210 INFO [Thread-1319] distributedshell.Client (Client.java:monitorApplication(759)) - Application has completed successfully. Breaking monitoring loop 2015-11-22 19:24:42,210 INFO [Thread-1319] distributedshell.TestDistributedShell (TestDistributedShell.java:testDSShellWithMultipleArgs(572)) - Client run completed. Result=true 2015-11-22 19:24:42,214 ERROR [Thread[Thread-1315,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(670)) - ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted 2015-11-22 19:24:42,216 INFO [main] mortbay.log (Slf4jLog.java:info(67)) - Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:0 2015-11-22 19:24:42,322 INFO [main] applicationhistoryservice.ApplicationHistoryManagerImpl (ApplicationHistoryManagerImpl.java:serviceStop(78)) - Stopping ApplicationHistory 2015-11-22 19:24:42,323 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 48317 2015-11-22 19:24:42,324 INFO [IPC Server listener on 48317] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 48317 2015-11-22 19:24:42,324 ERROR [Thread[Thread-1297,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(670)) - ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted 2015-11-22 19:24:42,325 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:24:42,325 INFO [main] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:unRegisterNM(268)) - Successfully Unregistered the Node ip-172-31-20-42.ap-northeast-1.compute.internal:34720 with ResourceManager. 2015-11-22 19:24:42,330 ERROR [Thread[Thread-1290,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(670)) - ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted 2015-11-22 19:24:42,331 INFO [main] mortbay.log (Slf4jLog.java:info(67)) - Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:0 2015-11-22 19:24:42,433 INFO [main] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:cleanUpApplicationsOnNMShutDown(556)) - Applications still running : [application_1448220266082_0001] 2015-11-22 19:24:42,433 INFO [main] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:cleanUpApplicationsOnNMShutDown(572)) - Waiting for Applications to be Finished 2015-11-22 19:24:42,433 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(463)) - Application application_1448220266082_0001 transitioned from RUNNING to APPLICATION_RESOURCES_CLEANINGUP 2015-11-22 19:24:42,434 INFO [DeletionService #1] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(480)) - Deleting absolute path : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220266082_0001 2015-11-22 19:24:42,434 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event APPLICATION_STOP for appId application_1448220266082_0001 2015-11-22 19:24:42,434 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(463)) - Application application_1448220266082_0001 transitioned from APPLICATION_RESOURCES_CLEANINGUP to FINISHED 2015-11-22 19:24:42,434 INFO [AsyncDispatcher event handler] loghandler.NonAggregatingLogHandler (NonAggregatingLogHandler.java:handle(167)) - Scheduling Log Deletion for application: application_1448220266082_0001, with delay of 10800 seconds 2015-11-22 19:24:46,434 INFO [main] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:cleanUpApplicationsOnNMShutDown(589)) - Done waiting for Applications to be Finished. Still alive: [application_1448220266082_0001] 2015-11-22 19:24:46,435 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 34720 2015-11-22 19:24:46,438 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:24:46,438 INFO [IPC Server listener on 34720] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 34720 2015-11-22 19:24:56,439 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 43758 2015-11-22 19:24:56,441 INFO [IPC Server listener on 43758] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 43758 2015-11-22 19:24:56,441 INFO [Public Localizer] localizer.ResourceLocalizationService (ResourceLocalizationService.java:run(908)) - Public cache exiting 2015-11-22 19:24:56,441 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:24:56,441 WARN [Node Resource Monitor] nodemanager.NodeResourceMonitorImpl (NodeResourceMonitorImpl.java:run(154)) - org.apache.hadoop.yarn.server.nodemanager.NodeResourceMonitorImpl is interrupted. Exiting. 2015-11-22 19:24:56,445 ERROR [Thread[Thread-1240,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(670)) - ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted 2015-11-22 19:24:56,446 INFO [main] mortbay.log (Slf4jLog.java:info(67)) - Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:0 2015-11-22 19:24:56,549 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 50548 2015-11-22 19:24:56,552 INFO [IPC Server listener on 50548] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 50548 2015-11-22 19:24:56,553 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:24:56,553 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 39659 2015-11-22 19:24:56,553 INFO [IPC Server listener on 39659] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 39659 2015-11-22 19:24:56,554 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:24:56,555 INFO [main] resourcemanager.ResourceManager (ResourceManager.java:transitionToStandby(1051)) - Transitioning to standby state 2015-11-22 19:24:56,555 WARN [ApplicationMaster Launcher] amlauncher.ApplicationMasterLauncher (ApplicationMasterLauncher.java:run(122)) - org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher$LauncherThread interrupted. Returning. 2015-11-22 19:24:56,555 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 32866 2015-11-22 19:24:56,558 INFO [IPC Server listener on 32866] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 32866 2015-11-22 19:24:56,558 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:24:56,558 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 41635 2015-11-22 19:24:56,561 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:24:56,561 INFO [Ping Checker] util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(135)) - NMLivelinessMonitor thread interrupted 2015-11-22 19:24:56,561 INFO [IPC Server listener on 41635] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 41635 2015-11-22 19:24:56,561 ERROR [ResourceManager Event Processor] resourcemanager.ResourceManager (ResourceManager.java:run(691)) - Returning, interrupted : java.lang.InterruptedException 2015-11-22 19:24:56,562 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:serviceStop(141)) - AsyncDispatcher is draining to stop, igonring any new events. 2015-11-22 19:24:56,563 INFO [Ping Checker] util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(135)) - AMLivelinessMonitor thread interrupted 2015-11-22 19:24:56,564 ERROR [Thread[Thread-1071,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(670)) - ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted 2015-11-22 19:24:56,564 INFO [Ping Checker] util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(135)) - AMLivelinessMonitor thread interrupted 2015-11-22 19:24:56,565 INFO [Ping Checker] util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(135)) - org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.ContainerAllocationExpirer thread interrupted 2015-11-22 19:24:56,566 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stop(211)) - Stopping ApplicationHistoryServer metrics system... 2015-11-22 19:24:56,570 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stop(217)) - ApplicationHistoryServer metrics system stopped. 2015-11-22 19:24:56,570 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:shutdown(608)) - ApplicationHistoryServer metrics system shutdown complete. 2015-11-22 19:24:56,571 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:serviceStop(141)) - AsyncDispatcher is draining to stop, igonring any new events. 2015-11-22 19:24:56,571 INFO [main] resourcemanager.ResourceManager (ResourceManager.java:transitionToStandby(1058)) - Transitioned to standby state 2015-11-22 19:24:56,582 INFO [main] distributedshell.TestDistributedShell (TestDistributedShell.java:setupInternal(77)) - Starting up YARN cluster 2015-11-22 19:24:56,595 INFO [main] net.ServerSocketUtil (ServerSocketUtil.java:getPort(52)) - Using port 10021 2015-11-22 19:24:56,608 INFO [main] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2363)) - found resource core-site.xml at file:/home/ubuntu/hadoop-dev/hadoop-common-project/hadoop-common/target/test-classes/core-site.xml 2015-11-22 19:24:56,618 WARN [main] conf.Configuration (Configuration.java:loadProperty(2687)) - core-site.xml:an attempt to override final parameter: hadoop.tmp.dir; Ignoring. 2015-11-22 19:24:56,618 INFO [main] security.Groups (Groups.java:refresh(256)) - clearing userToGroupsMap cache 2015-11-22 19:24:56,619 INFO [main] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2363)) - found resource yarn-site.xml at file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/test-classes/yarn-site.xml 2015-11-22 19:24:56,632 WARN [main] conf.Configuration (Configuration.java:loadProperty(2687)) - yarn-site.xml:an attempt to override final parameter: hadoop.tmp.dir; Ignoring. 2015-11-22 19:24:56,633 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.RMFatalEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMFatalEventDispatcher 2015-11-22 19:24:56,633 INFO [main] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:(75)) - NMTokenKeyRollingInterval: 86400000ms and NMTokenKeyActivationDelay: 900000ms 2015-11-22 19:24:56,634 INFO [main] security.RMContainerTokenSecretManager (RMContainerTokenSecretManager.java:(78)) - ContainerTokenKeyRollingInterval: 86400000ms and ContainerTokenKeyActivationDelay: 900000ms 2015-11-22 19:24:56,634 INFO [main] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:(94)) - AMRMTokenKeyRollingInterval: 86400000ms and AMRMTokenKeyActivationDelay: 900000 ms 2015-11-22 19:24:56,634 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStoreEventType for class org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$ForwardingEventHandler 2015-11-22 19:24:56,634 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.NodesListManagerEventType for class org.apache.hadoop.yarn.server.resourcemanager.NodesListManager 2015-11-22 19:24:56,634 INFO [main] resourcemanager.ResourceManager (ResourceManager.java:createScheduler(295)) - Using Scheduler: org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler 2015-11-22 19:24:56,635 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.SchedulerEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher 2015-11-22 19:24:56,635 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationEventDispatcher 2015-11-22 19:24:56,635 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationAttemptEventDispatcher 2015-11-22 19:24:56,635 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$NodeEventDispatcher 2015-11-22 19:24:56,636 INFO [main] impl.MetricsConfig (MetricsConfig.java:loadFirst(115)) - loaded properties from hadoop-metrics2.properties 2015-11-22 19:24:56,637 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:startTimer(374)) - Scheduled snapshot period at 10 second(s). 2015-11-22 19:24:56,638 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:start(192)) - ResourceManager metrics system started 2015-11-22 19:24:56,638 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.RMAppManagerEventType for class org.apache.hadoop.yarn.server.resourcemanager.RMAppManager 2015-11-22 19:24:56,639 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncherEventType for class org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher 2015-11-22 19:24:56,639 WARN [main] util.MBeans (MBeans.java:register(73)) - Failed to register MBean "Hadoop:service=ResourceManager,name=RMNMInfo": Instance already exists. 2015-11-22 19:24:56,639 INFO [main] resourcemanager.RMNMInfo (RMNMInfo.java:(63)) - Registered RMNMInfo MBean 2015-11-22 19:24:56,639 INFO [main] util.HostsFileReader (HostsFileReader.java:refresh(132)) - Refreshing hosts (include/exclude) list 2015-11-22 19:24:56,640 INFO [main] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2363)) - found resource capacity-scheduler.xml at jar:file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/target/hadoop-yarn-server-tests-2.8.0-SNAPSHOT-tests.jar!/capacity-scheduler.xml 2015-11-22 19:24:56,653 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(647)) - max alloc mb per queue for root is undefined 2015-11-22 19:24:56,653 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(651)) - max alloc vcore per queue for root is undefined 2015-11-22 19:24:56,653 INFO [main] capacity.ParentQueue (ParentQueue.java:setupQueueConfigs(125)) - root, capacity=1.0, asboluteCapacity=1.0, maxCapacity=1.0, asboluteMaxCapacity=1.0, state=RUNNING, acls=ADMINISTER_QUEUE:*SUBMIT_APP:*, labels=*, , reservationsContinueLooking=true 2015-11-22 19:24:56,654 INFO [main] capacity.ParentQueue (ParentQueue.java:(104)) - Initialized parent-queue root name=root, fullname=root 2015-11-22 19:24:56,655 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(647)) - max alloc mb per queue for root.default is undefined 2015-11-22 19:24:56,655 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(651)) - max alloc vcore per queue for root.default is undefined 2015-11-22 19:24:56,656 INFO [main] capacity.LeafQueue (LeafQueue.java:setupQueueConfigs(217)) - Initializing default capacity = 1.0 [= (float) configuredCapacity / 100 ] asboluteCapacity = 1.0 [= parentAbsoluteCapacity * capacity ] maxCapacity = 1.0 [= configuredMaxCapacity ] absoluteMaxCapacity = 1.0 [= 1.0 maximumCapacity undefined, (parentAbsoluteMaxCapacity * maximumCapacity) / 100 otherwise ] userLimit = 100 [= configuredUserLimit ] userLimitFactor = 1.0 [= configuredUserLimitFactor ] maxApplications = 10000 [= configuredMaximumSystemApplicationsPerQueue or (int)(configuredMaximumSystemApplications * absoluteCapacity)] maxApplicationsPerUser = 10000 [= (int)(maxApplications * (userLimit / 100.0f) * userLimitFactor) ] usedCapacity = 0.0 [= usedResourcesMemory / (clusterResourceMemory * absoluteCapacity)] absoluteUsedCapacity = 0.0 [= usedResourcesMemory / clusterResourceMemory] maxAMResourcePerQueuePercent = 0.1 [= configuredMaximumAMResourcePercent ] minimumAllocationFactor = 0.984375 [= (float)(maximumAllocationMemory - minimumAllocationMemory) / maximumAllocationMemory ] maximumAllocation = [= configuredMaxAllocation ] numContainers = 0 [= currentNumContainers ] state = RUNNING [= configuredState ] acls = ADMINISTER_QUEUE:*SUBMIT_APP:* [= configuredAcls ] nodeLocalityDelay = -1 labels=*, nodeLocalityDelay = -1 reservationsContinueLooking = true preemptionDisabled = true defaultAppPriorityPerQueue = 0 2015-11-22 19:24:56,656 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:parseQueue(657)) - Initialized queue: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=0, numContainers=0 2015-11-22 19:24:56,657 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:parseQueue(657)) - Initialized queue: root: numChildQueue= 1, capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.0, numApps=0, numContainers=0 2015-11-22 19:24:56,657 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:initializeQueues(501)) - Initialized root queue root: numChildQueue= 1, capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.0, numApps=0, numContainers=0 2015-11-22 19:24:56,657 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:getUserGroupMappingPlacementRule(449)) - Initialized queue mappings, override: false 2015-11-22 19:24:56,657 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:initScheduler(314)) - Initialized CapacityScheduler with calculator=class org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator, minimumAllocation=<>, maximumAllocation=<>, asynchronousScheduling=false, asyncScheduleInterval=5ms 2015-11-22 19:24:56,658 INFO [main] metrics.SystemMetricsPublisher (SystemMetricsPublisher.java:serviceInit(95)) - YARN system metrics publishing service is not enabled 2015-11-22 19:24:56,658 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEventType for class org.apache.hadoop.yarn.server.MiniYARNCluster$1 2015-11-22 19:24:56,658 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:prepareDirs(558)) - Created localDir in /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0 2015-11-22 19:24:56,658 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:prepareDirs(558)) - Created logDir in /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-logDir-nm-0_0 2015-11-22 19:24:56,658 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceInit(540)) - Starting NM: 0 2015-11-22 19:24:56,667 INFO [main] nodemanager.NodeManager (NodeManager.java:getNodeHealthScriptRunner(255)) - Node Manager health check script is not available or doesn't have execute permission, so not starting the node health script runner. 2015-11-22 19:24:56,668 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ContainerEventDispatcher 2015-11-22 19:24:56,668 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ApplicationEventDispatcher 2015-11-22 19:24:56,668 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizationEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService 2015-11-22 19:24:56,668 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServicesEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices 2015-11-22 19:24:56,668 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl 2015-11-22 19:24:56,668 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncherEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncher 2015-11-22 19:24:56,668 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.ContainerManagerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl 2015-11-22 19:24:56,669 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.NodeManagerEventType for class org.apache.hadoop.yarn.server.MiniYARNCluster$ShortCircuitedNodeManager 2015-11-22 19:24:56,669 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:init(159)) - NodeManager metrics system started (again) 2015-11-22 19:24:56,670 INFO [main] nodemanager.NodeResourceMonitorImpl (NodeResourceMonitorImpl.java:serviceInit(76)) - Using ResourceCalculatorPlugin : org.apache.hadoop.yarn.util.ResourceCalculatorPlugin@13b6aecc 2015-11-22 19:24:56,670 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.event.LogHandlerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.NonAggregatingLogHandler 2015-11-22 19:24:56,670 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.sharedcache.SharedCacheUploadEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.sharedcache.SharedCacheUploadService 2015-11-22 19:24:56,670 INFO [main] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:serviceInit(261)) - AMRMProxyService is disabled 2015-11-22 19:24:56,670 INFO [main] localizer.ResourceLocalizationService (ResourceLocalizationService.java:validateConf(224)) - per directory file limit = 8192 2015-11-22 19:24:56,682 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerTracker 2015-11-22 19:24:56,682 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(116)) - Using ResourceCalculatorPlugin : org.apache.hadoop.yarn.util.ResourceCalculatorPlugin@158a8276 2015-11-22 19:24:56,682 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(121)) - Using ResourceCalculatorProcessTree : null 2015-11-22 19:24:56,682 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(158)) - Physical memory check enabled: false 2015-11-22 19:24:56,682 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(159)) - Virtual memory check enabled: false 2015-11-22 19:24:56,682 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:isEnabled(206)) - Neither virutal-memory nor physical-memory monitoring is needed. Not running the monitor-thread 2015-11-22 19:24:56,682 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(162)) - ContainersMonitor enabled: false 2015-11-22 19:24:56,683 INFO [main] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:serviceInit(175)) - Nodemanager resources: memory set to 4096MB. 2015-11-22 19:24:56,683 INFO [main] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:serviceInit(176)) - Nodemanager resources: vcores set to 8. 2015-11-22 19:24:56,683 INFO [main] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:serviceInit(208)) - Initialized nodemanager with : physical-memory=4096 virtual-memory=8602 virtual-cores=8 2015-11-22 19:24:56,683 WARN [main] applicationhistoryservice.ApplicationHistoryServer (ApplicationHistoryServer.java:createApplicationHistoryManager(218)) - The filesystem based application history store is deprecated. 2015-11-22 19:24:56,683 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:init(159)) - ApplicationHistoryServer metrics system started (again) 2015-11-22 19:24:56,684 INFO [main] applicationhistoryservice.ApplicationHistoryManagerImpl (ApplicationHistoryManagerImpl.java:serviceInit(61)) - ApplicationHistory Init 2015-11-22 19:24:56,684 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:startResourceManager(326)) - Waiting for RM to start... 2015-11-22 19:24:56,684 INFO [RM-0] resourcemanager.ResourceManager (ResourceManager.java:transitionToActive(1024)) - Transitioning to active state 2015-11-22 19:24:56,685 INFO [RM-0] recovery.RMStateStore (RMStateStore.java:transition(482)) - Updating AMRMToken 2015-11-22 19:24:56,685 INFO [RM-0] security.RMContainerTokenSecretManager (RMContainerTokenSecretManager.java:rollMasterKey(106)) - Rolling master-key for container-tokens 2015-11-22 19:24:56,685 INFO [RM-0] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:rollMasterKey(95)) - Rolling master-key for nm-tokens 2015-11-22 19:24:56,685 INFO [RM-0] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:24:56,686 INFO [RM-0] security.RMDelegationTokenSecretManager (RMDelegationTokenSecretManager.java:storeNewMasterKey(87)) - storing master key with keyID 1 2015-11-22 19:24:56,686 INFO [RM-0] recovery.RMStateStore (RMStateStore.java:transition(433)) - Storing RMDTMasterKey. 2015-11-22 19:24:56,686 INFO [Thread[Thread-1508,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2015-11-22 19:24:56,686 INFO [Thread[Thread-1508,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:24:56,688 INFO [RM-0] nodelabels.CommonNodeLabelsManager (CommonNodeLabelsManager.java:addToCluserNodeLabels(309)) - Add labels: [] 2015-11-22 19:24:56,688 INFO [RM-0] nodelabels.CommonNodeLabelsManager (CommonNodeLabelsManager.java:internalUpdateLabelsOnNodes(638)) - REPLACE labels on nodes: 2015-11-22 19:24:56,688 INFO [RM-0] nodelabels.CommonNodeLabelsManager (CommonNodeLabelsManager.java:internalUpdateLabelsOnNodes(640)) - NM=ip-172-31-20-42.ap-northeast-1.compute.internal:58694, labels=[x] 2015-11-22 19:24:56,689 INFO [RM-0] nodelabels.FileSystemNodeLabelsStore (FileSystemNodeLabelsStore.java:recover(279)) - Finished write mirror at:file:/tmp/hadoop-yarn-ubuntu/node-labels/nodelabel.mirror 2015-11-22 19:24:56,689 INFO [RM-0] nodelabels.FileSystemNodeLabelsStore (FileSystemNodeLabelsStore.java:recover(280)) - Finished create editlog file at:file:/tmp/hadoop-yarn-ubuntu/node-labels/nodelabel.editlog 2015-11-22 19:24:56,689 INFO [RM-0] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.nodelabels.event.NodeLabelsStoreEventType for class org.apache.hadoop.yarn.nodelabels.CommonNodeLabelsManager$ForwardingEventHandler 2015-11-22 19:24:56,689 INFO [Thread[Thread-1508,5,main]] security.RMDelegationTokenSecretManager (RMDelegationTokenSecretManager.java:storeNewMasterKey(87)) - storing master key with keyID 2 2015-11-22 19:24:56,690 INFO [Thread[Thread-1508,5,main]] recovery.RMStateStore (RMStateStore.java:transition(433)) - Storing RMDTMasterKey. 2015-11-22 19:24:56,691 INFO [RM-0] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:24:56,691 INFO [Socket Reader #1 for port 56122] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 56122 2015-11-22 19:24:56,692 INFO [RM-0] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.server.api.ResourceTrackerPB to the server 2015-11-22 19:24:56,693 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:24:56,695 INFO [IPC Server listener on 56122] ipc.Server (Server.java:run(794)) - IPC Server listener on 56122: starting 2015-11-22 19:24:56,703 INFO [RM-0] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:24:56,703 INFO [Socket Reader #1 for port 52849] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 52849 2015-11-22 19:24:56,705 INFO [RM-0] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB to the server 2015-11-22 19:24:56,706 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:24:56,706 INFO [IPC Server listener on 52849] ipc.Server (Server.java:run(794)) - IPC Server listener on 52849: starting 2015-11-22 19:24:56,712 INFO [RM-0] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:24:56,713 INFO [Socket Reader #1 for port 51403] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 51403 2015-11-22 19:24:56,714 INFO [RM-0] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.api.ApplicationClientProtocolPB to the server 2015-11-22 19:24:56,714 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:24:56,715 INFO [IPC Server listener on 51403] ipc.Server (Server.java:run(794)) - IPC Server listener on 51403: starting 2015-11-22 19:24:56,719 INFO [RM-0] resourcemanager.ResourceManager (ResourceManager.java:transitionToActive(1040)) - Transitioned to active state 2015-11-22 19:24:56,720 INFO [RM-0] server.AuthenticationFilter (AuthenticationFilter.java:constructSecretProvider(304)) - Unable to initialize FileSignerSecretProvider, falling back to use random secrets. 2015-11-22 19:24:56,720 INFO [RM-0] http.HttpRequestLog (HttpRequestLog.java:getRequestLog(80)) - Http request log for http.requests.resourcemanager is not defined 2015-11-22 19:24:56,721 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter) 2015-11-22 19:24:56,722 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context cluster 2015-11-22 19:24:56,722 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context logs 2015-11-22 19:24:56,722 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context static 2015-11-22 19:24:56,722 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context cluster 2015-11-22 19:24:56,722 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2015-11-22 19:24:56,722 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2015-11-22 19:24:56,723 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'Timeline Authentication Filter' (class=org.apache.hadoop.yarn.server.timeline.security.TimelineAuthenticationFilter) 2015-11-22 19:24:56,723 INFO [RM-0] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /cluster/* 2015-11-22 19:24:56,723 INFO [RM-0] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /ws/* 2015-11-22 19:24:56,728 INFO [RM-0] webapp.WebApps (WebApps.java:build(288)) - Registered webapp guice modules 2015-11-22 19:24:56,728 INFO [RM-0] http.HttpServer2 (HttpServer2.java:openListeners(917)) - Jetty bound to port 47975 2015-11-22 19:24:56,728 INFO [RM-0] mortbay.log (Slf4jLog.java:info(67)) - jetty-6.1.26 2015-11-22 19:24:56,739 INFO [RM-0] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:24:56,739 INFO [Thread[Thread-1677,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2015-11-22 19:24:56,740 INFO [Thread[Thread-1677,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:24:56,740 INFO [RM-0] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:24:56,742 INFO [Thread[Thread-1679,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) Nov 22, 2015 7:24:56 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.resourcemanager.webapp.JAXBContextResolver as a provider class Nov 22, 2015 7:24:56 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices as a root resource class Nov 22, 2015 7:24:56 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a provider class Nov 22, 2015 7:24:56 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate INFO: Initiating Jersey application, version 'Jersey: 1.9 09/02/2011 11:17 AM' Nov 22, 2015 7:24:56 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.resourcemanager.webapp.JAXBContextResolver to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:24:56 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.webapp.GenericExceptionHandler to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:24:56 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices to GuiceManagedComponentProvider with the scope "Singleton" 2015-11-22 19:24:56,895 INFO [RM-0] mortbay.log (Slf4jLog.java:info(67)) - Started HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:47975 2015-11-22 19:24:56,896 INFO [RM-0] webapp.WebApps (WebApps.java:start(307)) - Web app cluster started at 47975 2015-11-22 19:24:56,897 INFO [RM-0] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:24:56,898 INFO [Socket Reader #1 for port 49153] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 49153 2015-11-22 19:24:56,901 INFO [RM-0] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.server.api.ResourceManagerAdministrationProtocolPB to the server 2015-11-22 19:24:56,901 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:24:56,901 INFO [IPC Server listener on 49153] ipc.Server (Server.java:run(794)) - IPC Server listener on 49153: starting 2015-11-22 19:24:58,185 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:startResourceManager(339)) - MiniYARN ResourceManager address: ip-172-31-20-42.ap-northeast-1.compute.internal:51403 2015-11-22 19:24:58,185 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:startResourceManager(341)) - MiniYARN ResourceManager web address: ip-172-31-20-42.ap-northeast-1.compute.internal:47975 2015-11-22 19:24:58,185 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(459)) - MiniYARN ResourceManager address: ip-172-31-20-42.ap-northeast-1.compute.internal:51403 2015-11-22 19:24:58,185 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(461)) - MiniYARN ResourceManager web address: ip-172-31-20-42.ap-northeast-1.compute.internal:47975 2015-11-22 19:24:58,185 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(575)) - Waiting for NM 0 to start... 2015-11-22 19:24:58,188 INFO [Thread-1686] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:24:58,189 INFO [Socket Reader #1 for port 48952] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 48952 2015-11-22 19:24:58,190 INFO [Thread-1686] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.api.ContainerManagementProtocolPB to the server 2015-11-22 19:24:58,190 INFO [Thread-1686] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:serviceStart(464)) - Blocking new container-requests as container manager rpc server is still starting. 2015-11-22 19:24:58,191 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:24:58,191 INFO [IPC Server listener on 48952] ipc.Server (Server.java:run(794)) - IPC Server listener on 48952: starting 2015-11-22 19:24:58,193 INFO [Thread-1686] security.NMContainerTokenSecretManager (NMContainerTokenSecretManager.java:setNodeId(260)) - Updating node address : ip-172-31-20-42.ap-northeast-1.compute.internal:48952 2015-11-22 19:24:58,193 INFO [Thread-1686] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:24:58,193 INFO [Socket Reader #1 for port 47193] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 47193 2015-11-22 19:24:58,194 INFO [Thread-1686] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB to the server 2015-11-22 19:24:58,194 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:24:58,194 INFO [IPC Server listener on 47193] ipc.Server (Server.java:run(794)) - IPC Server listener on 47193: starting 2015-11-22 19:24:58,196 INFO [Thread-1686] localizer.ResourceLocalizationService (ResourceLocalizationService.java:serviceStart(363)) - Localizer started on port 47193 2015-11-22 19:24:58,205 INFO [Thread-1686] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:serviceStart(509)) - ContainerManager started at /172.31.20.42:48952 2015-11-22 19:24:58,205 INFO [Thread-1686] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:serviceStart(510)) - ContainerManager bound to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:0 2015-11-22 19:24:58,205 INFO [Thread-1686] webapp.WebServer (WebServer.java:serviceStart(71)) - Instantiating NMWebApp at ip-172-31-20-42.ap-northeast-1.compute.internal:0 2015-11-22 19:24:58,206 INFO [Thread-1686] server.AuthenticationFilter (AuthenticationFilter.java:constructSecretProvider(304)) - Unable to initialize FileSignerSecretProvider, falling back to use random secrets. 2015-11-22 19:24:58,206 INFO [Thread-1686] http.HttpRequestLog (HttpRequestLog.java:getRequestLog(80)) - Http request log for http.requests.nodemanager is not defined 2015-11-22 19:24:58,207 INFO [Thread-1686] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter) 2015-11-22 19:24:58,207 INFO [Thread-1686] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context node 2015-11-22 19:24:58,207 INFO [Thread-1686] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context logs 2015-11-22 19:24:58,207 INFO [Thread-1686] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context static 2015-11-22 19:24:58,207 INFO [Thread-1686] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context node 2015-11-22 19:24:58,207 INFO [Thread-1686] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2015-11-22 19:24:58,208 INFO [Thread-1686] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2015-11-22 19:24:58,208 INFO [Thread-1686] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'Timeline Authentication Filter' (class=org.apache.hadoop.yarn.server.timeline.security.TimelineAuthenticationFilter) 2015-11-22 19:24:58,209 INFO [Thread-1686] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /node/* 2015-11-22 19:24:58,209 INFO [Thread-1686] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /ws/* Nov 22, 2015 7:24:58 PM com.google.inject.servlet.GuiceFilter setPipeline WARNING: Multiple Servlet injectors detected. This is a warning indicating that you have more than one GuiceFilter running in your web application. If this is deliberate, you may safely ignore this message. If this is NOT deliberate however, your application may not work as expected. 2015-11-22 19:24:58,213 INFO [Thread-1686] webapp.WebApps (WebApps.java:build(288)) - Registered webapp guice modules 2015-11-22 19:24:58,214 INFO [Thread-1686] http.HttpServer2 (HttpServer2.java:openListeners(917)) - Jetty bound to port 36078 2015-11-22 19:24:58,214 INFO [Thread-1686] mortbay.log (Slf4jLog.java:info(67)) - jetty-6.1.26 2015-11-22 19:24:58,228 INFO [Thread-1686] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:24:58,229 INFO [Thread[Thread-1727,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2015-11-22 19:24:58,229 INFO [Thread[Thread-1727,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:24:58,230 INFO [Thread-1686] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:24:58,230 INFO [Thread[Thread-1729,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) Nov 22, 2015 7:24:58 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.nodemanager.webapp.NMWebServices as a root resource class Nov 22, 2015 7:24:58 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a provider class Nov 22, 2015 7:24:58 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.nodemanager.webapp.JAXBContextResolver as a provider class Nov 22, 2015 7:24:58 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate INFO: Initiating Jersey application, version 'Jersey: 1.9 09/02/2011 11:17 AM' Nov 22, 2015 7:24:58 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.nodemanager.webapp.JAXBContextResolver to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:24:58 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.webapp.GenericExceptionHandler to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:24:58 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.nodemanager.webapp.NMWebServices to GuiceManagedComponentProvider with the scope "Singleton" 2015-11-22 19:24:58,316 INFO [Thread-1686] mortbay.log (Slf4jLog.java:info(67)) - Started HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:36078 2015-11-22 19:24:58,317 INFO [Thread-1686] webapp.WebApps (WebApps.java:start(307)) - Web app node started at 36078 2015-11-22 19:24:58,317 INFO [Thread-1686] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:serviceStart(222)) - Node ID assigned is : ip-172-31-20-42.ap-northeast-1.compute.internal:48952 2015-11-22 19:24:58,317 INFO [Thread-1686] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNMContainerStatuses(550)) - Sending out 0 NM container statuses: [] 2015-11-22 19:24:58,317 INFO [Thread-1686] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:registerWithRM(330)) - Registering with RM using containers :[] 2015-11-22 19:24:58,318 INFO [Thread-1686] resourcemanager.ResourceTrackerService (ResourceTrackerService.java:registerNodeManager(375)) - NodeManager from node ip-172-31-20-42.ap-northeast-1.compute.internal(cmPort: 48952 httpPort: 36078) registered with capability: , assigned nodeId ip-172-31-20-42.ap-northeast-1.compute.internal:48952 2015-11-22 19:24:58,318 INFO [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(566)) - ip-172-31-20-42.ap-northeast-1.compute.internal:48952 Node Transitioned from NEW to RUNNING 2015-11-22 19:24:58,318 INFO [Thread-1686] security.NMContainerTokenSecretManager (NMContainerTokenSecretManager.java:setMasterKey(138)) - Rolling master-key for container-tokens, got key with id 77037313 2015-11-22 19:24:58,319 INFO [Thread-1686] security.NMTokenSecretManagerInNM (NMTokenSecretManagerInNM.java:setMasterKey(135)) - Rolling master-key for container-tokens, got key with id 38564346 2015-11-22 19:24:58,319 INFO [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:addNode(1427)) - Added node ip-172-31-20-42.ap-northeast-1.compute.internal:48952 clusterResource: 2015-11-22 19:24:58,320 INFO [Thread-1686] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:registerWithRM(391)) - Registered with ResourceManager as ip-172-31-20-42.ap-northeast-1.compute.internal:48952 with total resource of 2015-11-22 19:24:58,320 INFO [Thread-1686] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:registerWithRM(392)) - Notifying ContainerManager to unblock new container-requests 2015-11-22 19:24:59,186 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(728)) - Waiting for Timeline Server to start... 2015-11-22 19:24:59,186 INFO [Thread-1732] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:24:59,186 INFO [Thread[Thread-1734,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2015-11-22 19:24:59,187 INFO [Thread[Thread-1734,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:24:59,187 INFO [Thread-1732] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:24:59,189 INFO [Socket Reader #1 for port 32776] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 32776 2015-11-22 19:24:59,190 INFO [Thread-1732] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.api.ApplicationHistoryProtocolPB to the server 2015-11-22 19:24:59,191 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:24:59,191 INFO [IPC Server listener on 32776] ipc.Server (Server.java:run(794)) - IPC Server listener on 32776: starting 2015-11-22 19:24:59,193 INFO [Thread-1732] applicationhistoryservice.ApplicationHistoryClientService (ApplicationHistoryClientService.java:serviceStart(111)) - Instantiated ApplicationHistoryClientService at ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:32776 2015-11-22 19:24:59,193 INFO [Thread-1732] applicationhistoryservice.ApplicationHistoryManagerImpl (ApplicationHistoryManagerImpl.java:serviceStart(71)) - Starting ApplicationHistory 2015-11-22 19:24:59,194 INFO [Thread-1732] server.AuthenticationFilter (AuthenticationFilter.java:constructSecretProvider(304)) - Unable to initialize FileSignerSecretProvider, falling back to use random secrets. 2015-11-22 19:24:59,194 INFO [Thread-1732] http.HttpRequestLog (HttpRequestLog.java:getRequestLog(80)) - Http request log for http.requests.applicationhistory is not defined 2015-11-22 19:24:59,195 INFO [Thread-1732] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter) 2015-11-22 19:24:59,195 INFO [Thread-1732] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context applicationhistory 2015-11-22 19:24:59,195 INFO [Thread-1732] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context static 2015-11-22 19:24:59,195 INFO [Thread-1732] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context logs 2015-11-22 19:24:59,196 INFO [Thread-1732] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context applicationhistory 2015-11-22 19:24:59,196 INFO [Thread-1732] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2015-11-22 19:24:59,196 INFO [Thread-1732] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2015-11-22 19:24:59,196 INFO [Thread-1732] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'Timeline Authentication Filter' (class=org.apache.hadoop.yarn.server.timeline.security.TimelineAuthenticationFilter) 2015-11-22 19:24:59,197 INFO [Thread-1732] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /applicationhistory/* 2015-11-22 19:24:59,197 INFO [Thread-1732] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /ws/* Nov 22, 2015 7:24:59 PM com.google.inject.servlet.GuiceFilter setPipeline WARNING: Multiple Servlet injectors detected. This is a warning indicating that you have more than one GuiceFilter running in your web application. If this is deliberate, you may safely ignore this message. If this is NOT deliberate however, your application may not work as expected. 2015-11-22 19:24:59,203 INFO [Thread-1732] webapp.WebApps (WebApps.java:build(288)) - Registered webapp guice modules 2015-11-22 19:24:59,204 INFO [Thread-1732] http.HttpServer2 (HttpServer2.java:openListeners(917)) - Jetty bound to port 56902 2015-11-22 19:24:59,205 INFO [Thread-1732] mortbay.log (Slf4jLog.java:info(67)) - jetty-6.1.26 2015-11-22 19:24:59,219 INFO [Thread-1732] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:24:59,220 INFO [Thread[Thread-1752,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2015-11-22 19:24:59,220 INFO [Thread[Thread-1752,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:24:59,221 INFO [Thread-1732] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:24:59,221 INFO [Thread[Thread-1754,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) Nov 22, 2015 7:24:59 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider as a provider class Nov 22, 2015 7:24:59 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.AHSWebServices as a root resource class Nov 22, 2015 7:24:59 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.timeline.webapp.TimelineWebServices as a root resource class Nov 22, 2015 7:24:59 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a provider class Nov 22, 2015 7:24:59 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate INFO: Initiating Jersey application, version 'Jersey: 1.9 09/02/2011 11:17 AM' Nov 22, 2015 7:24:59 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.webapp.GenericExceptionHandler to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:24:59 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:24:59 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.AHSWebServices to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:24:59 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.timeline.webapp.TimelineWebServices to GuiceManagedComponentProvider with the scope "Singleton" 2015-11-22 19:24:59,330 INFO [Thread-1732] mortbay.log (Slf4jLog.java:info(67)) - Started HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:56902 2015-11-22 19:24:59,331 INFO [Thread-1732] applicationhistoryservice.ApplicationHistoryServer (ApplicationHistoryServer.java:startWebApp(342)) - Instantiating AHSWebApp at 56902 2015-11-22 19:25:00,686 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(739)) - MiniYARN ApplicationHistoryServer address: ip-172-31-20-42.ap-northeast-1.compute.internal:32776 2015-11-22 19:25:00,686 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(741)) - MiniYARN ApplicationHistoryServer web address: ip-172-31-20-42.ap-northeast-1.compute.internal:56902 2015-11-22 19:25:02,724 INFO [Thread-1756] distributedshell.TestDistributedShell (TestDistributedShell.java:testDSAttemptFailuresValidityIntervalSucess(396)) - Initializing DS Client 2015-11-22 19:25:02,780 INFO [Thread-1756] impl.TimelineClientImpl (TimelineClientImpl.java:serviceInit(296)) - Timeline service address: http://ip-172-31-20-42.ap-northeast-1.compute.internal:56902/ws/v1/timeline/ 2015-11-22 19:25:02,780 INFO [Thread-1756] distributedshell.TestDistributedShell (TestDistributedShell.java:testDSAttemptFailuresValidityIntervalSucess(403)) - Running DS Client 2015-11-22 19:25:02,780 INFO [Thread-1756] distributedshell.Client (Client.java:run(443)) - Running Client 2015-11-22 19:25:02,793 INFO [Thread-1756] client.RMProxy (RMProxy.java:createRMProxy(98)) - Connecting to ResourceManager at ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:51403 2015-11-22 19:25:02,797 INFO [Thread-1756] distributedshell.Client (Client.java:run(447)) - Got Cluster metric info from ASM, numNodeManagers=1 2015-11-22 19:25:02,801 INFO [Thread-1756] distributedshell.Client (Client.java:run(452)) - Got Cluster node info from ASM 2015-11-22 19:25:02,801 INFO [Thread-1756] distributedshell.Client (Client.java:run(454)) - Got node report from ASM for, nodeId=ip-172-31-20-42.ap-northeast-1.compute.internal:48952, nodeAddress=ip-172-31-20-42.ap-northeast-1.compute.internal:36078, nodeRackName=/default-rack, nodeNumContainers=0 2015-11-22 19:25:02,802 INFO [Thread-1756] distributedshell.Client (Client.java:run(462)) - Queue info, queueName=default, queueCurrentCapacity=0.0, queueMaxCapacity=1.0, queueApplicationCount=0, queueChildQueueCount=0 2015-11-22 19:25:02,803 INFO [Thread-1756] distributedshell.Client (Client.java:run(472)) - User ACL Info for Queue, queueName=root, userAcl=SUBMIT_APPLICATIONS 2015-11-22 19:25:02,803 INFO [Thread-1756] distributedshell.Client (Client.java:run(472)) - User ACL Info for Queue, queueName=root, userAcl=ADMINISTER_QUEUE 2015-11-22 19:25:02,803 INFO [Thread-1756] distributedshell.Client (Client.java:run(472)) - User ACL Info for Queue, queueName=default, userAcl=SUBMIT_APPLICATIONS 2015-11-22 19:25:02,803 INFO [Thread-1756] distributedshell.Client (Client.java:run(472)) - User ACL Info for Queue, queueName=default, userAcl=ADMINISTER_QUEUE 2015-11-22 19:25:02,804 INFO [IPC Server handler 4 on 51403] resourcemanager.ClientRMService (ClientRMService.java:getNewApplicationId(305)) - Allocated new applicationId: 1 2015-11-22 19:25:02,804 INFO [Thread-1756] distributedshell.Client (Client.java:run(491)) - Max mem capability of resources in this cluster 4096 2015-11-22 19:25:02,804 INFO [Thread-1756] distributedshell.Client (Client.java:run(502)) - Max virtual cores capability of resources in this cluster 4 2015-11-22 19:25:02,804 INFO [Thread-1756] distributedshell.Client (Client.java:run(528)) - Copy App Master jar from local filesystem and add to local environment 2015-11-22 19:25:02,811 INFO [Thread-1756] distributedshell.Client (Client.java:run(577)) - Set the environment for the application master 2015-11-22 19:25:02,812 INFO [Thread-1756] distributedshell.Client (Client.java:run(619)) - Setting up app master command 2015-11-22 19:25:02,812 INFO [Thread-1756] distributedshell.Client (Client.java:run(650)) - Completed setting up app master command {{JAVA_HOME}}/bin/java -Xmx512m org.apache.hadoop.yarn.applications.distributedshell.TestDSSleepingAppMaster --container_memory 128 --container_vcores 1 --num_containers 1 --priority 0 1>/AppMaster.stdout 2>/AppMaster.stderr 2015-11-22 19:25:02,812 INFO [Thread-1756] distributedshell.Client (Client.java:run(706)) - Submitting application to ASM 2015-11-22 19:25:02,813 INFO [IPC Server handler 5 on 51403] capacity.CapacityScheduler (CapacityScheduler.java:checkAndGetApplicationPriority(1941)) - Priority '0' is acceptable in queue : default for application: application_1448220296684_0001 for the user: ubuntu 2015-11-22 19:25:02,813 WARN [IPC Server handler 5 on 51403] rmapp.RMAppImpl (RMAppImpl.java:(420)) - The specific max attempts: 0 for application: 1 is invalid, because it is out of the range [1, 2]. Use the global max attempts instead. 2015-11-22 19:25:02,813 INFO [IPC Server handler 5 on 51403] rmapp.RMAppImpl (RMAppImpl.java:(431)) - The attemptFailuresValidityInterval for the application: application_1448220296684_0001 is 2500. 2015-11-22 19:25:02,814 INFO [IPC Server handler 5 on 51403] resourcemanager.ClientRMService (ClientRMService.java:submitApplication(602)) - Application with id 1 submitted by user ubuntu 2015-11-22 19:25:02,814 INFO [IPC Server handler 5 on 51403] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(154)) - USER=ubuntu IP=172.31.20.42 OPERATION=Submit Application Request TARGET=ClientRMService RESULT=SUCCESS APPID=application_1448220296684_0001 2015-11-22 19:25:02,814 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:transition(1079)) - Storing application with id application_1448220296684_0001 2015-11-22 19:25:02,814 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220296684_0001 State change from NEW to NEW_SAVING 2015-11-22 19:25:02,814 INFO [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:transition(208)) - Storing info for app: application_1448220296684_0001 2015-11-22 19:25:02,814 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220296684_0001 State change from NEW_SAVING to SUBMITTED 2015-11-22 19:25:02,815 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:addApplication(349)) - Application added - appId: application_1448220296684_0001 user: ubuntu leaf-queue of parent: root #applications: 1 2015-11-22 19:25:02,815 INFO [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:addApplication(765)) - Accepted application application_1448220296684_0001 from user: ubuntu, in queue: default 2015-11-22 19:25:02,815 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220296684_0001 State change from SUBMITTED to ACCEPTED 2015-11-22 19:25:02,815 INFO [AsyncDispatcher event handler] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:registerAppAttempt(661)) - Registering app attempt : appattempt_1448220296684_0001_000001 2015-11-22 19:25:02,816 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220296684_0001_000001 State change from NEW to SUBMITTED 2015-11-22 19:25:02,816 INFO [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:activateApplications(705)) - Application application_1448220296684_0001 from user: ubuntu activated in queue: default 2015-11-22 19:25:02,816 INFO [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:addApplicationAttempt(720)) - Application added - appId: application_1448220296684_0001 user: ubuntu, leaf-queue: default #user-pending-applications: 0 #user-active-applications: 1 #queue-pending-applications: 0 #queue-active-applications: 1 2015-11-22 19:25:02,816 INFO [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:addApplicationAttempt(805)) - Added Application Attempt appattempt_1448220296684_0001_000001 to scheduler from user ubuntu in queue default 2015-11-22 19:25:02,818 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220296684_0001_000001 State change from SUBMITTED to SCHEDULED 2015-11-22 19:25:03,017 INFO [Thread-1756] impl.YarnClientImpl (YarnClientImpl.java:submitApplication(279)) - Submitted application application_1448220296684_0001 2015-11-22 19:25:03,324 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220296684_0001_01_000001 Container Transitioned from NEW to ALLOCATED 2015-11-22 19:25:03,324 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220296684_0001 CONTAINERID=container_1448220296684_0001_01_000001 2015-11-22 19:25:03,325 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(154)) - Assigned container container_1448220296684_0001_01_000001 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:48952, which has 1 containers, used and available after allocation 2015-11-22 19:25:03,325 INFO [ResourceManager Event Processor] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(89)) - assignedContainer application attempt=appattempt_1448220296684_0001_000001 container=container_1448220296684_0001_01_000001 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@6bd5797a clusterResource= type=OFF_SWITCH 2015-11-22 19:25:03,325 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(589)) - Re-sorting assigned queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.125, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 2015-11-22 19:25:03,325 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(461)) - assignedContainer queue=root usedCapacity=0.125 absoluteUsedCapacity=0.125 used= cluster= 2015-11-22 19:25:03,328 INFO [AsyncDispatcher event handler] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:createAndGetNMToken(200)) - Sending NMToken for nodeId : ip-172-31-20-42.ap-northeast-1.compute.internal:48952 for container : container_1448220296684_0001_01_000001 2015-11-22 19:25:03,329 INFO [AsyncDispatcher event handler] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220296684_0001_01_000001 Container Transitioned from ALLOCATED to ACQUIRED 2015-11-22 19:25:03,329 INFO [AsyncDispatcher event handler] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:clearNodeSetForAttempt(146)) - Clear node set for appattempt_1448220296684_0001_000001 2015-11-22 19:25:03,329 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:storeAttempt(2026)) - Storing attempt: AppId: application_1448220296684_0001 AttemptId: appattempt_1448220296684_0001_000001 MasterContainer: Container: [ContainerId: container_1448220296684_0001_01_000001, NodeId: ip-172-31-20-42.ap-northeast-1.compute.internal:48952, NodeHttpAddress: ip-172-31-20-42.ap-northeast-1.compute.internal:36078, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.31.20.42:48952 }, ] 2015-11-22 19:25:03,330 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220296684_0001_000001 State change from SCHEDULED to ALLOCATED_SAVING 2015-11-22 19:25:03,330 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220296684_0001_000001 State change from ALLOCATED_SAVING to ALLOCATED 2015-11-22 19:25:03,330 INFO [ApplicationMasterLauncher #0] amlauncher.AMLauncher (AMLauncher.java:run(254)) - Launching masterappattempt_1448220296684_0001_000001 2015-11-22 19:25:03,332 INFO [ApplicationMasterLauncher #0] amlauncher.AMLauncher (AMLauncher.java:launch(106)) - Setting up container Container: [ContainerId: container_1448220296684_0001_01_000001, NodeId: ip-172-31-20-42.ap-northeast-1.compute.internal:48952, NodeHttpAddress: ip-172-31-20-42.ap-northeast-1.compute.internal:36078, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.31.20.42:48952 }, ] for AM appattempt_1448220296684_0001_000001 2015-11-22 19:25:03,332 INFO [ApplicationMasterLauncher #0] amlauncher.AMLauncher (AMLauncher.java:createAMContainerLaunchContext(188)) - Command to launch container container_1448220296684_0001_01_000001 : {{JAVA_HOME}}/bin/java -Xmx512m org.apache.hadoop.yarn.applications.distributedshell.TestDSSleepingAppMaster --container_memory 128 --container_vcores 1 --num_containers 1 --priority 0 1>/AppMaster.stdout 2>/AppMaster.stderr 2015-11-22 19:25:03,333 INFO [ApplicationMasterLauncher #0] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:createAndGetAMRMToken(195)) - Create AMRMToken for ApplicationAttempt: appattempt_1448220296684_0001_000001 2015-11-22 19:25:03,333 INFO [ApplicationMasterLauncher #0] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:createPassword(307)) - Creating password for appattempt_1448220296684_0001_000001 2015-11-22 19:25:03,336 INFO [Socket Reader #1 for port 48952] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220296684_0001_000001 (auth:SIMPLE) 2015-11-22 19:25:03,340 INFO [IPC Server handler 0 on 48952] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:startContainerInternal(889)) - Start request for container_1448220296684_0001_01_000001 by user ubuntu 2015-11-22 19:25:03,340 INFO [IPC Server handler 0 on 48952] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:startContainerInternal(930)) - Creating a new application reference for app application_1448220296684_0001 2015-11-22 19:25:03,340 INFO [IPC Server handler 0 on 48952] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu IP=172.31.20.42 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1448220296684_0001 CONTAINERID=container_1448220296684_0001_01_000001 2015-11-22 19:25:03,341 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(463)) - Application application_1448220296684_0001 transitioned from NEW to INITING 2015-11-22 19:25:03,341 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(303)) - Adding container_1448220296684_0001_01_000001 to application application_1448220296684_0001 2015-11-22 19:25:03,343 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(463)) - Application application_1448220296684_0001 transitioned from INITING to RUNNING 2015-11-22 19:25:03,343 INFO [ApplicationMasterLauncher #0] amlauncher.AMLauncher (AMLauncher.java:launch(127)) - Done launching container Container: [ContainerId: container_1448220296684_0001_01_000001, NodeId: ip-172-31-20-42.ap-northeast-1.compute.internal:48952, NodeHttpAddress: ip-172-31-20-42.ap-northeast-1.compute.internal:36078, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.31.20.42:48952 }, ] for AM appattempt_1448220296684_0001_000001 2015-11-22 19:25:03,343 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220296684_0001_01_000001 transitioned from NEW to LOCALIZING 2015-11-22 19:25:03,343 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_INIT for appId application_1448220296684_0001 2015-11-22 19:25:03,343 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220296684_0001_000001 State change from ALLOCATED to LAUNCHED 2015-11-22 19:25:03,344 INFO [AsyncDispatcher event handler] localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource file:/home/ubuntu/DistributedShell/application_1448220296684_0001/AppMaster.jar transitioned from INIT to DOWNLOADING 2015-11-22 19:25:03,344 INFO [AsyncDispatcher event handler] localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource file:/home/ubuntu/DistributedShell/application_1448220296684_0001/shellCommands transitioned from INIT to DOWNLOADING 2015-11-22 19:25:03,344 INFO [AsyncDispatcher event handler] localizer.ResourceLocalizationService (ResourceLocalizationService.java:handle(742)) - Created localizer for container_1448220296684_0001_01_000001 2015-11-22 19:25:03,351 INFO [LocalizerRunner for container_1448220296684_0001_01_000001] localizer.ResourceLocalizationService (ResourceLocalizationService.java:writeCredentials(1221)) - Writing credentials to the nmPrivate file /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/container_1448220296684_0001_01_000001.tokens. Credentials list: 2015-11-22 19:25:03,352 INFO [LocalizerRunner for container_1448220296684_0001_01_000001] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:createUserCacheDirs(632)) - Initializing user ubuntu 2015-11-22 19:25:03,360 INFO [LocalizerRunner for container_1448220296684_0001_01_000001] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:startLocalizer(126)) - Copying from /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/container_1448220296684_0001_01_000001.tokens to /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220296684_0001/container_1448220296684_0001_01_000001.tokens 2015-11-22 19:25:03,360 INFO [LocalizerRunner for container_1448220296684_0001_01_000001] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:startLocalizer(133)) - Localizer CWD set to /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220296684_0001 = file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220296684_0001 2015-11-22 19:25:03,428 INFO [IPC Server handler 1 on 47193] localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource file:/home/ubuntu/DistributedShell/application_1448220296684_0001/AppMaster.jar(->/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220296684_0001/filecache/10/AppMaster.jar) transitioned from DOWNLOADING to LOCALIZED 2015-11-22 19:25:03,447 INFO [IPC Server handler 2 on 47193] localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource file:/home/ubuntu/DistributedShell/application_1448220296684_0001/shellCommands(->/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220296684_0001/filecache/11/shellCommands) transitioned from DOWNLOADING to LOCALIZED 2015-11-22 19:25:03,448 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220296684_0001_01_000001 transitioned from LOCALIZING to LOCALIZED 2015-11-22 19:25:03,479 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220296684_0001_01_000001 transitioned from LOCALIZED to RUNNING 2015-11-22 19:25:03,501 INFO [ContainersLauncher #0] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:buildCommandExecutor(281)) - launchContainer: [nice, -n, 0, bash, /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220296684_0001/container_1448220296684_0001_01_000001/default_container_executor.sh] 2015-11-22 19:25:04,020 INFO [Thread-1756] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=N/A, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220302813, yarnAppState=ACCEPTED, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:47975/proxy/application_1448220296684_0001/, appUser=ubuntu 2015-11-22 19:25:04,324 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220296684_0001_01_000001 Container Transitioned from ACQUIRED to RUNNING 2015-11-22 19:25:05,022 INFO [Thread-1756] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=N/A, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220302813, yarnAppState=ACCEPTED, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:47975/proxy/application_1448220296684_0001/, appUser=ubuntu 2015-11-22 19:25:06,024 INFO [Thread-1756] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=N/A, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220302813, yarnAppState=ACCEPTED, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:47975/proxy/application_1448220296684_0001/, appUser=ubuntu 2015-11-22 19:25:06,032 INFO [Socket Reader #1 for port 52849] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220296684_0001_000001 (auth:SIMPLE) 2015-11-22 19:25:06,042 INFO [IPC Server handler 4 on 52849] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:registerApplicationMaster(220)) - AM registration appattempt_1448220296684_0001_000001 2015-11-22 19:25:06,043 INFO [IPC Server handler 4 on 52849] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(133)) - USER=ubuntu IP=172.31.20.42 OPERATION=Register App Master TARGET=ApplicationMasterService RESULT=SUCCESS APPID=application_1448220296684_0001 APPATTEMPTID=appattempt_1448220296684_0001_000001 2015-11-22 19:25:06,043 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220296684_0001_000001 State change from LAUNCHED to RUNNING 2015-11-22 19:25:06,043 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220296684_0001 State change from ACCEPTED to RUNNING 2015-11-22 19:25:07,026 INFO [Thread-1756] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220302813, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:47975/proxy/application_1448220296684_0001/, appUser=ubuntu 2015-11-22 19:25:07,326 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220296684_0001_01_000002 Container Transitioned from NEW to ALLOCATED 2015-11-22 19:25:07,326 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220296684_0001 CONTAINERID=container_1448220296684_0001_01_000002 2015-11-22 19:25:07,326 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(154)) - Assigned container container_1448220296684_0001_01_000002 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:48952, which has 2 containers, used and available after allocation 2015-11-22 19:25:07,327 INFO [ResourceManager Event Processor] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(89)) - assignedContainer application attempt=appattempt_1448220296684_0001_000001 container=container_1448220296684_0001_01_000002 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@6bd5797a clusterResource= type=OFF_SWITCH 2015-11-22 19:25:07,327 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(589)) - Re-sorting assigned queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.15625, absoluteUsedCapacity=0.15625, numApps=1, numContainers=2 2015-11-22 19:25:07,327 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(461)) - assignedContainer queue=root usedCapacity=0.15625 absoluteUsedCapacity=0.15625 used= cluster= 2015-11-22 19:25:08,027 INFO [Thread-1756] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220302813, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:47975/proxy/application_1448220296684_0001/, appUser=ubuntu 2015-11-22 19:25:08,100 INFO [IPC Server handler 0 on 52849] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:createAndGetNMToken(200)) - Sending NMToken for nodeId : ip-172-31-20-42.ap-northeast-1.compute.internal:48952 for container : container_1448220296684_0001_01_000002 2015-11-22 19:25:08,101 INFO [IPC Server handler 0 on 52849] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220296684_0001_01_000002 Container Transitioned from ALLOCATED to ACQUIRED 2015-11-22 19:25:08,173 INFO [Socket Reader #1 for port 48952] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220296684_0001_000001 (auth:SIMPLE) 2015-11-22 19:25:08,177 INFO [IPC Server handler 1 on 48952] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:startContainerInternal(889)) - Start request for container_1448220296684_0001_01_000002 by user ubuntu 2015-11-22 19:25:08,177 INFO [IPC Server handler 1 on 48952] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu IP=172.31.20.42 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1448220296684_0001 CONTAINERID=container_1448220296684_0001_01_000002 2015-11-22 19:25:08,177 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(303)) - Adding container_1448220296684_0001_01_000002 to application application_1448220296684_0001 2015-11-22 19:25:08,178 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220296684_0001_01_000002 transitioned from NEW to LOCALIZED 2015-11-22 19:25:08,178 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_INIT for appId application_1448220296684_0001 2015-11-22 19:25:08,217 INFO [Socket Reader #1 for port 48952] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220296684_0001_000001 (auth:SIMPLE) 2015-11-22 19:25:08,228 INFO [IPC Server handler 0 on 48952] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:getContainerStatusInternal(1221)) - Getting container-status for container_1448220296684_0001_01_000002 2015-11-22 19:25:08,229 INFO [IPC Server handler 0 on 48952] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:getContainerStatusInternal(1235)) - Returning ContainerStatus: [ContainerId: container_1448220296684_0001_01_000002, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ] 2015-11-22 19:25:08,260 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220296684_0001_01_000002 transitioned from LOCALIZED to RUNNING 2015-11-22 19:25:08,296 INFO [ContainersLauncher #1] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:buildCommandExecutor(281)) - launchContainer: [nice, -n, 0, bash, /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220296684_0001/container_1448220296684_0001_01_000002/default_container_executor.sh] 2015-11-22 19:25:08,326 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220296684_0001_01_000002 Container Transitioned from ACQUIRED to RUNNING 2015-11-22 19:25:09,029 INFO [Thread-1756] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220302813, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:47975/proxy/application_1448220296684_0001/, appUser=ubuntu 2015-11-22 19:25:10,031 INFO [Thread-1756] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220302813, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:47975/proxy/application_1448220296684_0001/, appUser=ubuntu 2015-11-22 19:25:11,033 INFO [Thread-1756] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220302813, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:47975/proxy/application_1448220296684_0001/, appUser=ubuntu 2015-11-22 19:25:11,088 WARN [ContainersLauncher #0] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(237)) - Exit code from container container_1448220296684_0001_01_000001 is : 100 2015-11-22 19:25:11,088 WARN [ContainersLauncher #0] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(243)) - Exception from container-launch with container ID: container_1448220296684_0001_01_000001 and exit code: 100 ExitCodeException exitCode=100: at org.apache.hadoop.util.Shell.runCommand(Shell.java:909) at org.apache.hadoop.util.Shell.run(Shell.java:820) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1099) at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:225) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:304) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:84) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 2015-11-22 19:25:11,088 INFO [ContainersLauncher #0] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - Exception from container-launch. 2015-11-22 19:25:11,089 INFO [ContainersLauncher #0] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - Container id: container_1448220296684_0001_01_000001 2015-11-22 19:25:11,089 INFO [ContainersLauncher #0] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - Exit code: 100 2015-11-22 19:25:11,089 INFO [ContainersLauncher #0] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - Stack trace: ExitCodeException exitCode=100: 2015-11-22 19:25:11,089 INFO [ContainersLauncher #0] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - at org.apache.hadoop.util.Shell.runCommand(Shell.java:909) 2015-11-22 19:25:11,089 INFO [ContainersLauncher #0] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - at org.apache.hadoop.util.Shell.run(Shell.java:820) 2015-11-22 19:25:11,089 INFO [ContainersLauncher #0] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1099) 2015-11-22 19:25:11,089 INFO [ContainersLauncher #0] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:225) 2015-11-22 19:25:11,089 INFO [ContainersLauncher #0] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:304) 2015-11-22 19:25:11,089 INFO [ContainersLauncher #0] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:84) 2015-11-22 19:25:11,089 INFO [ContainersLauncher #0] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - at java.util.concurrent.FutureTask.run(FutureTask.java:266) 2015-11-22 19:25:11,089 INFO [ContainersLauncher #0] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 2015-11-22 19:25:11,089 INFO [ContainersLauncher #0] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 2015-11-22 19:25:11,090 INFO [ContainersLauncher #0] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - at java.lang.Thread.run(Thread.java:745) 2015-11-22 19:25:11,090 WARN [ContainersLauncher #0] launcher.ContainerLaunch (ContainerLaunch.java:call(348)) - Container exited with a non-zero exit code 100 2015-11-22 19:25:11,090 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220296684_0001_01_000001 transitioned from RUNNING to EXITED_WITH_FAILURE 2015-11-22 19:25:11,090 INFO [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(380)) - Cleaning up container container_1448220296684_0001_01_000001 2015-11-22 19:25:11,105 INFO [DeletionService #0] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(480)) - Deleting absolute path : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220296684_0001/container_1448220296684_0001_01_000001 2015-11-22 19:25:11,105 WARN [AsyncDispatcher event handler] nodemanager.NMAuditLogger (NMAuditLogger.java:logFailure(150)) - USER=ubuntu OPERATION=Container Finished - Failed TARGET=ContainerImpl RESULT=FAILURE DESCRIPTION=Container failed with state: EXITED_WITH_FAILURE APPID=application_1448220296684_0001 CONTAINERID=container_1448220296684_0001_01_000001 2015-11-22 19:25:11,105 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220296684_0001_01_000001 transitioned from EXITED_WITH_FAILURE to DONE 2015-11-22 19:25:11,106 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(346)) - Removing container_1448220296684_0001_01_000001 from application application_1448220296684_0001 2015-11-22 19:25:11,106 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_STOP for appId application_1448220296684_0001 2015-11-22 19:25:11,328 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220296684_0001_01_000001 Container Transitioned from RUNNING to COMPLETED 2015-11-22 19:25:11,328 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220296684_0001 CONTAINERID=container_1448220296684_0001_01_000001 2015-11-22 19:25:11,328 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:releaseContainer(248)) - Released container container_1448220296684_0001_01_000001 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:48952, which currently has 1 containers, used and available, release resources=true 2015-11-22 19:25:11,328 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:rememberTargetTransitionsAndStoreState(1258)) - Updating application attempt appattempt_1448220296684_0001_000001 with final state: FAILED, and exit status: 100 2015-11-22 19:25:11,328 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220296684_0001_000001 State change from RUNNING to FINAL_SAVING 2015-11-22 19:25:11,329 INFO [AsyncDispatcher event handler] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:unregisterAttempt(667)) - Unregistering app attempt : appattempt_1448220296684_0001_000001 2015-11-22 19:25:11,329 INFO [AsyncDispatcher event handler] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:applicationMasterFinished(124)) - Application finished, removing password for appattempt_1448220296684_0001_000001 2015-11-22 19:25:11,329 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220296684_0001_000001 State change from FINAL_SAVING to FAILED 2015-11-22 19:25:11,329 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:transition(1302)) - The number of failed attempts in previous 2500 milliseconds is 1. The max attempts is 2 2015-11-22 19:25:11,329 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220296684_0001 State change from RUNNING to ACCEPTED 2015-11-22 19:25:11,330 INFO [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:doneApplicationAttempt(844)) - Application Attempt appattempt_1448220296684_0001_000001 is done. finalState=FAILED 2015-11-22 19:25:11,330 INFO [AsyncDispatcher event handler] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:registerAppAttempt(661)) - Registering app attempt : appattempt_1448220296684_0001_000002 2015-11-22 19:25:11,330 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220296684_0001_01_000002 Container Transitioned from RUNNING to KILLED 2015-11-22 19:25:11,330 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220296684_0001_000002 State change from NEW to SUBMITTED 2015-11-22 19:25:11,330 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220296684_0001 CONTAINERID=container_1448220296684_0001_01_000002 2015-11-22 19:25:11,330 INFO [ApplicationMasterLauncher #1] amlauncher.AMLauncher (AMLauncher.java:run(268)) - Cleaning master appattempt_1448220296684_0001_000001 2015-11-22 19:25:11,330 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:releaseContainer(248)) - Released container container_1448220296684_0001_01_000002 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:48952, which currently has 0 containers, used and available, release resources=true 2015-11-22 19:25:11,331 INFO [ResourceManager Event Processor] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:clearRequests(125)) - Application application_1448220296684_0001 requests cleared 2015-11-22 19:25:11,331 INFO [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:removeApplicationAttempt(772)) - Application removed - appId: application_1448220296684_0001 user: ubuntu queue: default #user-pending-applications: 0 #user-active-applications: 0 #queue-pending-applications: 0 #queue-active-applications: 0 2015-11-22 19:25:11,331 INFO [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:activateApplications(705)) - Application application_1448220296684_0001 from user: ubuntu activated in queue: default 2015-11-22 19:25:11,331 INFO [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:addApplicationAttempt(720)) - Application added - appId: application_1448220296684_0001 user: ubuntu, leaf-queue: default #user-pending-applications: 0 #user-active-applications: 1 #queue-pending-applications: 0 #queue-active-applications: 1 2015-11-22 19:25:11,331 INFO [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:addApplicationAttempt(805)) - Added Application Attempt appattempt_1448220296684_0001_000002 to scheduler from user ubuntu in queue default 2015-11-22 19:25:11,332 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220296684_0001_000002 State change from SUBMITTED to SCHEDULED 2015-11-22 19:25:11,337 INFO [Socket Reader #1 for port 48952] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220296684_0001_000001 (auth:SIMPLE) 2015-11-22 19:25:11,339 INFO [IPC Server handler 1 on 48952] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:stopContainerInternal(1163)) - Stopping container with container Id: container_1448220296684_0001_01_000001 2015-11-22 19:25:11,339 INFO [IPC Server handler 1 on 48952] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu IP=172.31.20.42 OPERATION=Stop Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1448220296684_0001 CONTAINERID=container_1448220296684_0001_01_000001 2015-11-22 19:25:11,340 INFO [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:removeOrTrackCompletedContainersFromContext(605)) - Removed completed containers from NM context: [container_1448220296684_0001_01_000001] 2015-11-22 19:25:11,340 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220296684_0001_01_000002 transitioned from RUNNING to KILLING 2015-11-22 19:25:11,341 INFO [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(380)) - Cleaning up container container_1448220296684_0001_01_000002 2015-11-22 19:25:11,368 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220296684_0001_02_000001 Container Transitioned from NEW to ALLOCATED 2015-11-22 19:25:11,368 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220296684_0001 CONTAINERID=container_1448220296684_0001_02_000001 2015-11-22 19:25:11,368 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(154)) - Assigned container container_1448220296684_0001_02_000001 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:48952, which has 1 containers, used and available after allocation 2015-11-22 19:25:11,368 INFO [ResourceManager Event Processor] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(89)) - assignedContainer application attempt=appattempt_1448220296684_0001_000002 container=container_1448220296684_0001_02_000001 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@65129a70 clusterResource= type=OFF_SWITCH 2015-11-22 19:25:11,369 INFO [AsyncDispatcher event handler] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:createAndGetNMToken(200)) - Sending NMToken for nodeId : ip-172-31-20-42.ap-northeast-1.compute.internal:48952 for container : container_1448220296684_0001_02_000001 2015-11-22 19:25:11,372 INFO [AsyncDispatcher event handler] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220296684_0001_02_000001 Container Transitioned from ALLOCATED to ACQUIRED 2015-11-22 19:25:11,372 INFO [AsyncDispatcher event handler] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:clearNodeSetForAttempt(146)) - Clear node set for appattempt_1448220296684_0001_000002 2015-11-22 19:25:11,372 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(589)) - Re-sorting assigned queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.125, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 2015-11-22 19:25:11,372 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(461)) - assignedContainer queue=root usedCapacity=0.125 absoluteUsedCapacity=0.125 used= cluster= 2015-11-22 19:25:11,372 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:storeAttempt(2026)) - Storing attempt: AppId: application_1448220296684_0001 AttemptId: appattempt_1448220296684_0001_000002 MasterContainer: Container: [ContainerId: container_1448220296684_0001_02_000001, NodeId: ip-172-31-20-42.ap-northeast-1.compute.internal:48952, NodeHttpAddress: ip-172-31-20-42.ap-northeast-1.compute.internal:36078, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.31.20.42:48952 }, ] 2015-11-22 19:25:11,373 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220296684_0001_000002 State change from SCHEDULED to ALLOCATED_SAVING 2015-11-22 19:25:11,374 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220296684_0001_000002 State change from ALLOCATED_SAVING to ALLOCATED 2015-11-22 19:25:11,388 WARN [ContainersLauncher #1] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(237)) - Exit code from container container_1448220296684_0001_01_000002 is : 143 2015-11-22 19:25:11,389 INFO [ApplicationMasterLauncher #2] amlauncher.AMLauncher (AMLauncher.java:run(254)) - Launching masterappattempt_1448220296684_0001_000002 2015-11-22 19:25:11,390 INFO [ApplicationMasterLauncher #2] amlauncher.AMLauncher (AMLauncher.java:launch(106)) - Setting up container Container: [ContainerId: container_1448220296684_0001_02_000001, NodeId: ip-172-31-20-42.ap-northeast-1.compute.internal:48952, NodeHttpAddress: ip-172-31-20-42.ap-northeast-1.compute.internal:36078, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.31.20.42:48952 }, ] for AM appattempt_1448220296684_0001_000002 2015-11-22 19:25:11,390 INFO [ApplicationMasterLauncher #2] amlauncher.AMLauncher (AMLauncher.java:createAMContainerLaunchContext(188)) - Command to launch container container_1448220296684_0001_02_000001 : {{JAVA_HOME}}/bin/java -Xmx512m org.apache.hadoop.yarn.applications.distributedshell.TestDSSleepingAppMaster --container_memory 128 --container_vcores 1 --num_containers 1 --priority 0 1>/AppMaster.stdout 2>/AppMaster.stderr 2015-11-22 19:25:11,390 INFO [ApplicationMasterLauncher #2] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:createAndGetAMRMToken(195)) - Create AMRMToken for ApplicationAttempt: appattempt_1448220296684_0001_000002 2015-11-22 19:25:11,390 INFO [ApplicationMasterLauncher #2] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:createPassword(307)) - Creating password for appattempt_1448220296684_0001_000002 2015-11-22 19:25:11,393 INFO [Socket Reader #1 for port 48952] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220296684_0001_000002 (auth:SIMPLE) 2015-11-22 19:25:11,397 INFO [IPC Server handler 2 on 48952] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:startContainerInternal(889)) - Start request for container_1448220296684_0001_02_000001 by user ubuntu 2015-11-22 19:25:11,397 INFO [IPC Server handler 2 on 48952] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu IP=172.31.20.42 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1448220296684_0001 CONTAINERID=container_1448220296684_0001_02_000001 2015-11-22 19:25:11,400 INFO [ApplicationMasterLauncher #2] amlauncher.AMLauncher (AMLauncher.java:launch(127)) - Done launching container Container: [ContainerId: container_1448220296684_0001_02_000001, NodeId: ip-172-31-20-42.ap-northeast-1.compute.internal:48952, NodeHttpAddress: ip-172-31-20-42.ap-northeast-1.compute.internal:36078, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.31.20.42:48952 }, ] for AM appattempt_1448220296684_0001_000002 2015-11-22 19:25:11,400 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220296684_0001_000002 State change from ALLOCATED to LAUNCHED 2015-11-22 19:25:11,402 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220296684_0001_01_000002 transitioned from KILLING to CONTAINER_CLEANEDUP_AFTER_KILL 2015-11-22 19:25:11,402 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(303)) - Adding container_1448220296684_0001_02_000001 to application application_1448220296684_0001 2015-11-22 19:25:11,403 INFO [DeletionService #0] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(480)) - Deleting absolute path : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220296684_0001/container_1448220296684_0001_01_000002 2015-11-22 19:25:11,403 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220296684_0001_02_000001 transitioned from NEW to LOCALIZING 2015-11-22 19:25:11,403 INFO [AsyncDispatcher event handler] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu OPERATION=Container Finished - Killed TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1448220296684_0001 CONTAINERID=container_1448220296684_0001_01_000002 2015-11-22 19:25:11,404 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220296684_0001_01_000002 transitioned from CONTAINER_CLEANEDUP_AFTER_KILL to DONE 2015-11-22 19:25:11,404 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_INIT for appId application_1448220296684_0001 2015-11-22 19:25:11,404 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(346)) - Removing container_1448220296684_0001_01_000002 from application application_1448220296684_0001 2015-11-22 19:25:11,404 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_STOP for appId application_1448220296684_0001 2015-11-22 19:25:11,404 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220296684_0001_02_000001 transitioned from LOCALIZING to LOCALIZED 2015-11-22 19:25:11,420 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220296684_0001_02_000001 transitioned from LOCALIZED to RUNNING 2015-11-22 19:25:11,437 INFO [ContainersLauncher #1] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:buildCommandExecutor(281)) - launchContainer: [nice, -n, 0, bash, /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220296684_0001/container_1448220296684_0001_02_000001/default_container_executor.sh] 2015-11-22 19:25:12,036 INFO [Thread-1756] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=N/A, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220302813, yarnAppState=ACCEPTED, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:47975/proxy/application_1448220296684_0001/, appUser=ubuntu 2015-11-22 19:25:12,342 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220296684_0001_02_000001 Container Transitioned from ACQUIRED to RUNNING 2015-11-22 19:25:12,342 INFO [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:completedContainer(1486)) - Container container_1448220296684_0001_01_000002 completed with event FINISHED 2015-11-22 19:25:13,038 INFO [Thread-1756] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=N/A, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220302813, yarnAppState=ACCEPTED, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:47975/proxy/application_1448220296684_0001/, appUser=ubuntu 2015-11-22 19:25:13,835 INFO [Socket Reader #1 for port 52849] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220296684_0001_000002 (auth:SIMPLE) 2015-11-22 19:25:13,843 INFO [IPC Server handler 4 on 52849] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:registerApplicationMaster(220)) - AM registration appattempt_1448220296684_0001_000002 2015-11-22 19:25:13,843 INFO [IPC Server handler 4 on 52849] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(133)) - USER=ubuntu IP=172.31.20.42 OPERATION=Register App Master TARGET=ApplicationMasterService RESULT=SUCCESS APPID=application_1448220296684_0001 APPATTEMPTID=appattempt_1448220296684_0001_000002 2015-11-22 19:25:13,844 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220296684_0001_000002 State change from LAUNCHED to RUNNING 2015-11-22 19:25:13,844 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220296684_0001 State change from ACCEPTED to RUNNING 2015-11-22 19:25:14,039 INFO [Thread-1756] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220302813, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:47975/proxy/application_1448220296684_0001/, appUser=ubuntu 2015-11-22 19:25:15,041 INFO [Thread-1756] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220302813, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:47975/proxy/application_1448220296684_0001/, appUser=ubuntu 2015-11-22 19:25:15,342 INFO [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:removeOrTrackCompletedContainersFromContext(605)) - Removed completed containers from NM context: [container_1448220296684_0001_01_000002] 2015-11-22 19:25:15,344 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220296684_0001_02_000002 Container Transitioned from NEW to ALLOCATED 2015-11-22 19:25:15,344 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220296684_0001 CONTAINERID=container_1448220296684_0001_02_000002 2015-11-22 19:25:15,344 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(154)) - Assigned container container_1448220296684_0001_02_000002 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:48952, which has 2 containers, used and available after allocation 2015-11-22 19:25:15,344 INFO [ResourceManager Event Processor] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(89)) - assignedContainer application attempt=appattempt_1448220296684_0001_000002 container=container_1448220296684_0001_02_000002 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@65129a70 clusterResource= type=OFF_SWITCH 2015-11-22 19:25:15,345 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(589)) - Re-sorting assigned queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.15625, absoluteUsedCapacity=0.15625, numApps=1, numContainers=2 2015-11-22 19:25:15,345 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(461)) - assignedContainer queue=root usedCapacity=0.15625 absoluteUsedCapacity=0.15625 used= cluster= 2015-11-22 19:25:15,890 INFO [IPC Server handler 0 on 52849] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:createAndGetNMToken(200)) - Sending NMToken for nodeId : ip-172-31-20-42.ap-northeast-1.compute.internal:48952 for container : container_1448220296684_0001_02_000002 2015-11-22 19:25:15,890 INFO [IPC Server handler 0 on 52849] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220296684_0001_02_000002 Container Transitioned from ALLOCATED to ACQUIRED 2015-11-22 19:25:15,965 INFO [Socket Reader #1 for port 48952] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220296684_0001_000002 (auth:SIMPLE) 2015-11-22 19:25:15,976 INFO [IPC Server handler 3 on 48952] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:startContainerInternal(889)) - Start request for container_1448220296684_0001_02_000002 by user ubuntu 2015-11-22 19:25:15,976 INFO [IPC Server handler 3 on 48952] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu IP=172.31.20.42 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1448220296684_0001 CONTAINERID=container_1448220296684_0001_02_000002 2015-11-22 19:25:15,977 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(303)) - Adding container_1448220296684_0001_02_000002 to application application_1448220296684_0001 2015-11-22 19:25:15,977 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220296684_0001_02_000002 transitioned from NEW to LOCALIZED 2015-11-22 19:25:15,977 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_INIT for appId application_1448220296684_0001 2015-11-22 19:25:16,012 INFO [Socket Reader #1 for port 48952] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220296684_0001_000002 (auth:SIMPLE) 2015-11-22 19:25:16,024 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220296684_0001_02_000002 transitioned from LOCALIZED to RUNNING 2015-11-22 19:25:16,026 INFO [IPC Server handler 6 on 48952] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:getContainerStatusInternal(1221)) - Getting container-status for container_1448220296684_0001_02_000002 2015-11-22 19:25:16,026 INFO [IPC Server handler 6 on 48952] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:getContainerStatusInternal(1235)) - Returning ContainerStatus: [ContainerId: container_1448220296684_0001_02_000002, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ] 2015-11-22 19:25:16,045 INFO [Thread-1756] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220302813, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:47975/proxy/application_1448220296684_0001/, appUser=ubuntu 2015-11-22 19:25:16,061 INFO [ContainersLauncher #0] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:buildCommandExecutor(281)) - launchContainer: [nice, -n, 0, bash, /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220296684_0001/container_1448220296684_0001_02_000002/default_container_executor.sh] 2015-11-22 19:25:16,344 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220296684_0001_02_000002 Container Transitioned from ACQUIRED to RUNNING 2015-11-22 19:25:17,046 INFO [Thread-1756] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220302813, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:47975/proxy/application_1448220296684_0001/, appUser=ubuntu 2015-11-22 19:25:18,048 INFO [Thread-1756] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220302813, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:47975/proxy/application_1448220296684_0001/, appUser=ubuntu 2015-11-22 19:25:18,884 WARN [ContainersLauncher #1] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(237)) - Exit code from container container_1448220296684_0001_02_000001 is : 100 2015-11-22 19:25:18,884 WARN [ContainersLauncher #1] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(243)) - Exception from container-launch with container ID: container_1448220296684_0001_02_000001 and exit code: 100 ExitCodeException exitCode=100: at org.apache.hadoop.util.Shell.runCommand(Shell.java:909) at org.apache.hadoop.util.Shell.run(Shell.java:820) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1099) at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:225) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:304) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:84) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 2015-11-22 19:25:18,884 INFO [ContainersLauncher #1] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - Exception from container-launch. 2015-11-22 19:25:18,884 INFO [ContainersLauncher #1] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - Container id: container_1448220296684_0001_02_000001 2015-11-22 19:25:18,884 INFO [ContainersLauncher #1] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - Exit code: 100 2015-11-22 19:25:18,884 INFO [ContainersLauncher #1] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - Stack trace: ExitCodeException exitCode=100: 2015-11-22 19:25:18,885 INFO [ContainersLauncher #1] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - at org.apache.hadoop.util.Shell.runCommand(Shell.java:909) 2015-11-22 19:25:18,885 INFO [ContainersLauncher #1] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - at org.apache.hadoop.util.Shell.run(Shell.java:820) 2015-11-22 19:25:18,885 INFO [ContainersLauncher #1] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1099) 2015-11-22 19:25:18,885 INFO [ContainersLauncher #1] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:225) 2015-11-22 19:25:18,885 INFO [ContainersLauncher #1] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:304) 2015-11-22 19:25:18,885 INFO [ContainersLauncher #1] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:84) 2015-11-22 19:25:18,885 INFO [ContainersLauncher #1] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - at java.util.concurrent.FutureTask.run(FutureTask.java:266) 2015-11-22 19:25:18,885 INFO [ContainersLauncher #1] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 2015-11-22 19:25:18,885 INFO [ContainersLauncher #1] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 2015-11-22 19:25:18,885 INFO [ContainersLauncher #1] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - at java.lang.Thread.run(Thread.java:745) 2015-11-22 19:25:18,885 WARN [ContainersLauncher #1] launcher.ContainerLaunch (ContainerLaunch.java:call(348)) - Container exited with a non-zero exit code 100 2015-11-22 19:25:18,885 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220296684_0001_02_000001 transitioned from RUNNING to EXITED_WITH_FAILURE 2015-11-22 19:25:18,885 INFO [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(380)) - Cleaning up container container_1448220296684_0001_02_000001 2015-11-22 19:25:18,898 INFO [DeletionService #2] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(480)) - Deleting absolute path : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220296684_0001/container_1448220296684_0001_02_000001 2015-11-22 19:25:18,899 WARN [AsyncDispatcher event handler] nodemanager.NMAuditLogger (NMAuditLogger.java:logFailure(150)) - USER=ubuntu OPERATION=Container Finished - Failed TARGET=ContainerImpl RESULT=FAILURE DESCRIPTION=Container failed with state: EXITED_WITH_FAILURE APPID=application_1448220296684_0001 CONTAINERID=container_1448220296684_0001_02_000001 2015-11-22 19:25:18,899 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220296684_0001_02_000001 transitioned from EXITED_WITH_FAILURE to DONE 2015-11-22 19:25:18,899 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(346)) - Removing container_1448220296684_0001_02_000001 from application application_1448220296684_0001 2015-11-22 19:25:18,899 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_STOP for appId application_1448220296684_0001 2015-11-22 19:25:19,049 INFO [Thread-1756] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220302813, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:47975/proxy/application_1448220296684_0001/, appUser=ubuntu 2015-11-22 19:25:19,346 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220296684_0001_02_000001 Container Transitioned from RUNNING to COMPLETED 2015-11-22 19:25:19,346 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220296684_0001 CONTAINERID=container_1448220296684_0001_02_000001 2015-11-22 19:25:19,346 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:releaseContainer(248)) - Released container container_1448220296684_0001_02_000001 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:48952, which currently has 1 containers, used and available, release resources=true 2015-11-22 19:25:19,346 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:rememberTargetTransitionsAndStoreState(1258)) - Updating application attempt appattempt_1448220296684_0001_000002 with final state: FAILED, and exit status: 100 2015-11-22 19:25:19,346 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220296684_0001_000002 State change from RUNNING to FINAL_SAVING 2015-11-22 19:25:19,346 INFO [AsyncDispatcher event handler] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:unregisterAttempt(667)) - Unregistering app attempt : appattempt_1448220296684_0001_000002 2015-11-22 19:25:19,346 INFO [AsyncDispatcher event handler] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:applicationMasterFinished(124)) - Application finished, removing password for appattempt_1448220296684_0001_000002 2015-11-22 19:25:19,347 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220296684_0001_000002 State change from FINAL_SAVING to FAILED 2015-11-22 19:25:19,347 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:transition(1302)) - The number of failed attempts in previous 2500 milliseconds is 1. The max attempts is 2 2015-11-22 19:25:19,347 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220296684_0001 State change from RUNNING to ACCEPTED 2015-11-22 19:25:19,347 INFO [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:doneApplicationAttempt(844)) - Application Attempt appattempt_1448220296684_0001_000002 is done. finalState=FAILED 2015-11-22 19:25:19,347 INFO [AsyncDispatcher event handler] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:registerAppAttempt(661)) - Registering app attempt : appattempt_1448220296684_0001_000003 2015-11-22 19:25:19,347 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220296684_0001_02_000002 Container Transitioned from RUNNING to KILLED 2015-11-22 19:25:19,347 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220296684_0001 CONTAINERID=container_1448220296684_0001_02_000002 2015-11-22 19:25:19,347 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:releaseContainer(248)) - Released container container_1448220296684_0001_02_000002 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:48952, which currently has 0 containers, used and available, release resources=true 2015-11-22 19:25:19,348 INFO [ResourceManager Event Processor] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:clearRequests(125)) - Application application_1448220296684_0001 requests cleared 2015-11-22 19:25:19,348 INFO [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:removeApplicationAttempt(772)) - Application removed - appId: application_1448220296684_0001 user: ubuntu queue: default #user-pending-applications: 0 #user-active-applications: 0 #queue-pending-applications: 0 #queue-active-applications: 0 2015-11-22 19:25:19,347 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220296684_0001_000003 State change from NEW to SUBMITTED 2015-11-22 19:25:19,348 INFO [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:activateApplications(705)) - Application application_1448220296684_0001 from user: ubuntu activated in queue: default 2015-11-22 19:25:19,348 INFO [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:addApplicationAttempt(720)) - Application added - appId: application_1448220296684_0001 user: ubuntu, leaf-queue: default #user-pending-applications: 0 #user-active-applications: 1 #queue-pending-applications: 0 #queue-active-applications: 1 2015-11-22 19:25:19,349 INFO [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:addApplicationAttempt(805)) - Added Application Attempt appattempt_1448220296684_0001_000003 to scheduler from user ubuntu in queue default 2015-11-22 19:25:19,347 INFO [ApplicationMasterLauncher #3] amlauncher.AMLauncher (AMLauncher.java:run(268)) - Cleaning master appattempt_1448220296684_0001_000002 2015-11-22 19:25:19,349 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220296684_0001_000003 State change from SUBMITTED to SCHEDULED 2015-11-22 19:25:19,354 INFO [Socket Reader #1 for port 48952] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220296684_0001_000002 (auth:SIMPLE) 2015-11-22 19:25:19,355 INFO [IPC Server handler 2 on 48952] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:stopContainerInternal(1163)) - Stopping container with container Id: container_1448220296684_0001_02_000001 2015-11-22 19:25:19,355 INFO [IPC Server handler 2 on 48952] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu IP=172.31.20.42 OPERATION=Stop Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1448220296684_0001 CONTAINERID=container_1448220296684_0001_02_000001 2015-11-22 19:25:19,357 INFO [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:removeOrTrackCompletedContainersFromContext(605)) - Removed completed containers from NM context: [container_1448220296684_0001_02_000001] 2015-11-22 19:25:19,359 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220296684_0001_02_000002 transitioned from RUNNING to KILLING 2015-11-22 19:25:19,360 INFO [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(380)) - Cleaning up container container_1448220296684_0001_02_000002 2015-11-22 19:25:19,359 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220296684_0001_03_000001 Container Transitioned from NEW to ALLOCATED 2015-11-22 19:25:19,360 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220296684_0001 CONTAINERID=container_1448220296684_0001_03_000001 2015-11-22 19:25:19,360 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(154)) - Assigned container container_1448220296684_0001_03_000001 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:48952, which has 1 containers, used and available after allocation 2015-11-22 19:25:19,360 INFO [ResourceManager Event Processor] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(89)) - assignedContainer application attempt=appattempt_1448220296684_0001_000003 container=container_1448220296684_0001_03_000001 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@6471231d clusterResource= type=OFF_SWITCH 2015-11-22 19:25:19,361 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(589)) - Re-sorting assigned queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.125, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 2015-11-22 19:25:19,361 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(461)) - assignedContainer queue=root usedCapacity=0.125 absoluteUsedCapacity=0.125 used= cluster= 2015-11-22 19:25:19,361 INFO [AsyncDispatcher event handler] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:createAndGetNMToken(200)) - Sending NMToken for nodeId : ip-172-31-20-42.ap-northeast-1.compute.internal:48952 for container : container_1448220296684_0001_03_000001 2015-11-22 19:25:19,362 INFO [AsyncDispatcher event handler] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220296684_0001_03_000001 Container Transitioned from ALLOCATED to ACQUIRED 2015-11-22 19:25:19,362 INFO [AsyncDispatcher event handler] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:clearNodeSetForAttempt(146)) - Clear node set for appattempt_1448220296684_0001_000003 2015-11-22 19:25:19,362 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:storeAttempt(2026)) - Storing attempt: AppId: application_1448220296684_0001 AttemptId: appattempt_1448220296684_0001_000003 MasterContainer: Container: [ContainerId: container_1448220296684_0001_03_000001, NodeId: ip-172-31-20-42.ap-northeast-1.compute.internal:48952, NodeHttpAddress: ip-172-31-20-42.ap-northeast-1.compute.internal:36078, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.31.20.42:48952 }, ] 2015-11-22 19:25:19,363 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220296684_0001_000003 State change from SCHEDULED to ALLOCATED_SAVING 2015-11-22 19:25:19,363 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220296684_0001_000003 State change from ALLOCATED_SAVING to ALLOCATED 2015-11-22 19:25:19,363 INFO [ApplicationMasterLauncher #4] amlauncher.AMLauncher (AMLauncher.java:run(254)) - Launching masterappattempt_1448220296684_0001_000003 2015-11-22 19:25:19,365 INFO [ApplicationMasterLauncher #4] amlauncher.AMLauncher (AMLauncher.java:launch(106)) - Setting up container Container: [ContainerId: container_1448220296684_0001_03_000001, NodeId: ip-172-31-20-42.ap-northeast-1.compute.internal:48952, NodeHttpAddress: ip-172-31-20-42.ap-northeast-1.compute.internal:36078, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.31.20.42:48952 }, ] for AM appattempt_1448220296684_0001_000003 2015-11-22 19:25:19,365 INFO [ApplicationMasterLauncher #4] amlauncher.AMLauncher (AMLauncher.java:createAMContainerLaunchContext(188)) - Command to launch container container_1448220296684_0001_03_000001 : {{JAVA_HOME}}/bin/java -Xmx512m org.apache.hadoop.yarn.applications.distributedshell.TestDSSleepingAppMaster --container_memory 128 --container_vcores 1 --num_containers 1 --priority 0 1>/AppMaster.stdout 2>/AppMaster.stderr 2015-11-22 19:25:19,365 INFO [ApplicationMasterLauncher #4] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:createAndGetAMRMToken(195)) - Create AMRMToken for ApplicationAttempt: appattempt_1448220296684_0001_000003 2015-11-22 19:25:19,365 INFO [ApplicationMasterLauncher #4] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:createPassword(307)) - Creating password for appattempt_1448220296684_0001_000003 2015-11-22 19:25:19,368 WARN [ContainersLauncher #0] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(237)) - Exit code from container container_1448220296684_0001_02_000002 is : 143 2015-11-22 19:25:19,368 INFO [Socket Reader #1 for port 48952] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220296684_0001_000003 (auth:SIMPLE) 2015-11-22 19:25:19,374 INFO [IPC Server handler 3 on 48952] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:startContainerInternal(889)) - Start request for container_1448220296684_0001_03_000001 by user ubuntu 2015-11-22 19:25:19,375 INFO [IPC Server handler 3 on 48952] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu IP=172.31.20.42 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1448220296684_0001 CONTAINERID=container_1448220296684_0001_03_000001 2015-11-22 19:25:19,375 INFO [ApplicationMasterLauncher #4] amlauncher.AMLauncher (AMLauncher.java:launch(127)) - Done launching container Container: [ContainerId: container_1448220296684_0001_03_000001, NodeId: ip-172-31-20-42.ap-northeast-1.compute.internal:48952, NodeHttpAddress: ip-172-31-20-42.ap-northeast-1.compute.internal:36078, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.31.20.42:48952 }, ] for AM appattempt_1448220296684_0001_000003 2015-11-22 19:25:19,376 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220296684_0001_000003 State change from ALLOCATED to LAUNCHED 2015-11-22 19:25:19,380 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220296684_0001_02_000002 transitioned from KILLING to CONTAINER_CLEANEDUP_AFTER_KILL 2015-11-22 19:25:19,380 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(303)) - Adding container_1448220296684_0001_03_000001 to application application_1448220296684_0001 2015-11-22 19:25:19,380 INFO [DeletionService #0] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(480)) - Deleting absolute path : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220296684_0001/container_1448220296684_0001_02_000002 2015-11-22 19:25:19,381 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220296684_0001_03_000001 transitioned from NEW to LOCALIZING 2015-11-22 19:25:19,381 INFO [AsyncDispatcher event handler] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu OPERATION=Container Finished - Killed TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1448220296684_0001 CONTAINERID=container_1448220296684_0001_02_000002 2015-11-22 19:25:19,381 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220296684_0001_02_000002 transitioned from CONTAINER_CLEANEDUP_AFTER_KILL to DONE 2015-11-22 19:25:19,381 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_INIT for appId application_1448220296684_0001 2015-11-22 19:25:19,382 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(346)) - Removing container_1448220296684_0001_02_000002 from application application_1448220296684_0001 2015-11-22 19:25:19,382 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_STOP for appId application_1448220296684_0001 2015-11-22 19:25:19,382 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220296684_0001_03_000001 transitioned from LOCALIZING to LOCALIZED 2015-11-22 19:25:19,408 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220296684_0001_03_000001 transitioned from LOCALIZED to RUNNING 2015-11-22 19:25:19,436 INFO [ContainersLauncher #0] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:buildCommandExecutor(281)) - launchContainer: [nice, -n, 0, bash, /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220296684_0001/container_1448220296684_0001_03_000001/default_container_executor.sh] 2015-11-22 19:25:20,057 INFO [Thread-1756] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=N/A, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220302813, yarnAppState=ACCEPTED, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:47975/proxy/application_1448220296684_0001/, appUser=ubuntu 2015-11-22 19:25:20,359 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220296684_0001_03_000001 Container Transitioned from ACQUIRED to RUNNING 2015-11-22 19:25:20,359 INFO [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:completedContainer(1486)) - Container container_1448220296684_0001_02_000002 completed with event FINISHED 2015-11-22 19:25:21,059 INFO [Thread-1756] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=N/A, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220302813, yarnAppState=ACCEPTED, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:47975/proxy/application_1448220296684_0001/, appUser=ubuntu 2015-11-22 19:25:21,694 INFO [Socket Reader #1 for port 52849] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220296684_0001_000003 (auth:SIMPLE) 2015-11-22 19:25:21,706 INFO [IPC Server handler 8 on 52849] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:registerApplicationMaster(220)) - AM registration appattempt_1448220296684_0001_000003 2015-11-22 19:25:21,706 INFO [IPC Server handler 8 on 52849] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(133)) - USER=ubuntu IP=172.31.20.42 OPERATION=Register App Master TARGET=ApplicationMasterService RESULT=SUCCESS APPID=application_1448220296684_0001 APPATTEMPTID=appattempt_1448220296684_0001_000003 2015-11-22 19:25:21,706 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220296684_0001_000003 State change from LAUNCHED to RUNNING 2015-11-22 19:25:21,706 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220296684_0001 State change from ACCEPTED to RUNNING 2015-11-22 19:25:22,061 INFO [Thread-1756] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220302813, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:47975/proxy/application_1448220296684_0001/, appUser=ubuntu 2015-11-22 19:25:23,063 INFO [Thread-1756] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220302813, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:47975/proxy/application_1448220296684_0001/, appUser=ubuntu 2015-11-22 19:25:23,360 INFO [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:removeOrTrackCompletedContainersFromContext(605)) - Removed completed containers from NM context: [container_1448220296684_0001_02_000002] 2015-11-22 19:25:23,361 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220296684_0001_03_000002 Container Transitioned from NEW to ALLOCATED 2015-11-22 19:25:23,361 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220296684_0001 CONTAINERID=container_1448220296684_0001_03_000002 2015-11-22 19:25:23,361 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(154)) - Assigned container container_1448220296684_0001_03_000002 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:48952, which has 2 containers, used and available after allocation 2015-11-22 19:25:23,362 INFO [ResourceManager Event Processor] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(89)) - assignedContainer application attempt=appattempt_1448220296684_0001_000003 container=container_1448220296684_0001_03_000002 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@6471231d clusterResource= type=OFF_SWITCH 2015-11-22 19:25:23,362 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(589)) - Re-sorting assigned queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.15625, absoluteUsedCapacity=0.15625, numApps=1, numContainers=2 2015-11-22 19:25:23,362 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(461)) - assignedContainer queue=root usedCapacity=0.15625 absoluteUsedCapacity=0.15625 used= cluster= 2015-11-22 19:25:23,764 INFO [IPC Server handler 1 on 52849] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:createAndGetNMToken(200)) - Sending NMToken for nodeId : ip-172-31-20-42.ap-northeast-1.compute.internal:48952 for container : container_1448220296684_0001_03_000002 2015-11-22 19:25:23,765 INFO [IPC Server handler 1 on 52849] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220296684_0001_03_000002 Container Transitioned from ALLOCATED to ACQUIRED 2015-11-22 19:25:23,829 INFO [Socket Reader #1 for port 48952] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220296684_0001_000003 (auth:SIMPLE) 2015-11-22 19:25:23,832 INFO [IPC Server handler 6 on 48952] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:startContainerInternal(889)) - Start request for container_1448220296684_0001_03_000002 by user ubuntu 2015-11-22 19:25:23,832 INFO [IPC Server handler 6 on 48952] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu IP=172.31.20.42 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1448220296684_0001 CONTAINERID=container_1448220296684_0001_03_000002 2015-11-22 19:25:23,833 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(303)) - Adding container_1448220296684_0001_03_000002 to application application_1448220296684_0001 2015-11-22 19:25:23,833 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220296684_0001_03_000002 transitioned from NEW to LOCALIZED 2015-11-22 19:25:23,833 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_INIT for appId application_1448220296684_0001 2015-11-22 19:25:23,871 INFO [Socket Reader #1 for port 48952] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220296684_0001_000003 (auth:SIMPLE) 2015-11-22 19:25:23,876 INFO [IPC Server handler 8 on 48952] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:getContainerStatusInternal(1221)) - Getting container-status for container_1448220296684_0001_03_000002 2015-11-22 19:25:23,876 INFO [IPC Server handler 8 on 48952] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:getContainerStatusInternal(1235)) - Returning ContainerStatus: [ContainerId: container_1448220296684_0001_03_000002, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ] 2015-11-22 19:25:23,906 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220296684_0001_03_000002 transitioned from LOCALIZED to RUNNING 2015-11-22 19:25:23,967 INFO [ContainersLauncher #1] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:buildCommandExecutor(281)) - launchContainer: [nice, -n, 0, bash, /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220296684_0001/container_1448220296684_0001_03_000002/default_container_executor.sh] 2015-11-22 19:25:24,065 INFO [Thread-1756] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220302813, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:47975/proxy/application_1448220296684_0001/, appUser=ubuntu 2015-11-22 19:25:24,361 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220296684_0001_03_000002 Container Transitioned from ACQUIRED to RUNNING 2015-11-22 19:25:25,067 INFO [Thread-1756] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220302813, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:47975/proxy/application_1448220296684_0001/, appUser=ubuntu 2015-11-22 19:25:26,069 INFO [Thread-1756] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220302813, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:47975/proxy/application_1448220296684_0001/, appUser=ubuntu 2015-11-22 19:25:27,071 INFO [Thread-1756] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220302813, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:47975/proxy/application_1448220296684_0001/, appUser=ubuntu 2015-11-22 19:25:28,073 INFO [Thread-1756] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220302813, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:47975/proxy/application_1448220296684_0001/, appUser=ubuntu 2015-11-22 19:25:29,074 INFO [Thread-1756] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220302813, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:47975/proxy/application_1448220296684_0001/, appUser=ubuntu 2015-11-22 19:25:30,076 INFO [Thread-1756] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220302813, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:47975/proxy/application_1448220296684_0001/, appUser=ubuntu 2015-11-22 19:25:31,078 INFO [Thread-1756] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220302813, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:47975/proxy/application_1448220296684_0001/, appUser=ubuntu 2015-11-22 19:25:31,977 INFO [ContainersLauncher #1] launcher.ContainerLaunch (ContainerLaunch.java:call(356)) - Container container_1448220296684_0001_03_000002 succeeded 2015-11-22 19:25:31,977 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220296684_0001_03_000002 transitioned from RUNNING to EXITED_WITH_SUCCESS 2015-11-22 19:25:31,977 INFO [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(380)) - Cleaning up container container_1448220296684_0001_03_000002 2015-11-22 19:25:31,991 INFO [DeletionService #2] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(480)) - Deleting absolute path : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220296684_0001/container_1448220296684_0001_03_000002 2015-11-22 19:25:31,991 INFO [AsyncDispatcher event handler] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu OPERATION=Container Finished - Succeeded TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1448220296684_0001 CONTAINERID=container_1448220296684_0001_03_000002 2015-11-22 19:25:31,991 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220296684_0001_03_000002 transitioned from EXITED_WITH_SUCCESS to DONE 2015-11-22 19:25:31,991 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(346)) - Removing container_1448220296684_0001_03_000002 from application application_1448220296684_0001 2015-11-22 19:25:31,991 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_STOP for appId application_1448220296684_0001 2015-11-22 19:25:32,079 INFO [Thread-1756] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220302813, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:47975/proxy/application_1448220296684_0001/, appUser=ubuntu 2015-11-22 19:25:32,365 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220296684_0001_03_000002 Container Transitioned from RUNNING to COMPLETED 2015-11-22 19:25:32,365 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220296684_0001 CONTAINERID=container_1448220296684_0001_03_000002 2015-11-22 19:25:32,365 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:releaseContainer(248)) - Released container container_1448220296684_0001_03_000002 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:48952, which currently has 1 containers, used and available, release resources=true 2015-11-22 19:25:32,998 INFO [Socket Reader #1 for port 48952] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220296684_0001_000003 (auth:SIMPLE) 2015-11-22 19:25:33,002 INFO [IPC Server handler 12 on 48952] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:stopContainerInternal(1163)) - Stopping container with container Id: container_1448220296684_0001_03_000002 2015-11-22 19:25:33,002 INFO [IPC Server handler 12 on 48952] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu IP=172.31.20.42 OPERATION=Stop Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1448220296684_0001 CONTAINERID=container_1448220296684_0001_03_000002 2015-11-22 19:25:33,014 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:rememberTargetTransitionsAndStoreState(1258)) - Updating application attempt appattempt_1448220296684_0001_000003 with final state: FINISHING, and exit status: -1000 2015-11-22 19:25:33,014 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220296684_0001_000003 State change from RUNNING to FINAL_SAVING 2015-11-22 19:25:33,014 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:rememberTargetTransitionsAndStoreState(1098)) - Updating application application_1448220296684_0001 with final state: FINISHING 2015-11-22 19:25:33,015 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220296684_0001 State change from RUNNING to FINAL_SAVING 2015-11-22 19:25:33,015 INFO [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:transition(237)) - Updating info for app: application_1448220296684_0001 2015-11-22 19:25:33,015 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220296684_0001_000003 State change from FINAL_SAVING to FINISHING 2015-11-22 19:25:33,015 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220296684_0001 State change from FINAL_SAVING to FINISHING 2015-11-22 19:25:33,081 INFO [Thread-1756] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220302813, yarnAppState=FINISHED, distributedFinalState=SUCCEEDED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:47975/proxy/application_1448220296684_0001/, appUser=ubuntu 2015-11-22 19:25:33,081 INFO [Thread-1756] distributedshell.Client (Client.java:monitorApplication(759)) - Application has completed successfully. Breaking monitoring loop 2015-11-22 19:25:33,081 INFO [Thread-1756] distributedshell.TestDistributedShell (TestDistributedShell.java:testDSAttemptFailuresValidityIntervalSucess(406)) - Client run completed. Result=true 2015-11-22 19:25:33,084 ERROR [Thread[Thread-1752,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(670)) - ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted 2015-11-22 19:25:33,085 INFO [main] mortbay.log (Slf4jLog.java:info(67)) - Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:0 2015-11-22 19:25:33,088 INFO [main] applicationhistoryservice.ApplicationHistoryManagerImpl (ApplicationHistoryManagerImpl.java:serviceStop(78)) - Stopping ApplicationHistory 2015-11-22 19:25:33,088 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 32776 2015-11-22 19:25:33,090 INFO [IPC Server listener on 32776] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 32776 2015-11-22 19:25:33,090 ERROR [Thread[Thread-1734,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(670)) - ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted 2015-11-22 19:25:33,090 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:25:33,091 INFO [main] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:unRegisterNM(268)) - Successfully Unregistered the Node ip-172-31-20-42.ap-northeast-1.compute.internal:48952 with ResourceManager. 2015-11-22 19:25:33,095 ERROR [Thread[Thread-1727,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(670)) - ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted 2015-11-22 19:25:33,096 INFO [main] mortbay.log (Slf4jLog.java:info(67)) - Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:0 2015-11-22 19:25:33,117 INFO [IPC Server handler 18 on 52849] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:finishApplicationMaster(300)) - application_1448220296684_0001 unregistered successfully. 2015-11-22 19:25:33,132 INFO [ContainersLauncher #0] launcher.ContainerLaunch (ContainerLaunch.java:call(356)) - Container container_1448220296684_0001_03_000001 succeeded 2015-11-22 19:25:33,132 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220296684_0001_03_000001 transitioned from RUNNING to EXITED_WITH_SUCCESS 2015-11-22 19:25:33,132 INFO [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(380)) - Cleaning up container container_1448220296684_0001_03_000001 2015-11-22 19:25:33,145 INFO [DeletionService #0] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(480)) - Deleting absolute path : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220296684_0001/container_1448220296684_0001_03_000001 2015-11-22 19:25:33,145 INFO [AsyncDispatcher event handler] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu OPERATION=Container Finished - Succeeded TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1448220296684_0001 CONTAINERID=container_1448220296684_0001_03_000001 2015-11-22 19:25:33,145 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220296684_0001_03_000001 transitioned from EXITED_WITH_SUCCESS to DONE 2015-11-22 19:25:33,145 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(346)) - Removing container_1448220296684_0001_03_000001 from application application_1448220296684_0001 2015-11-22 19:25:33,146 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_STOP for appId application_1448220296684_0001 2015-11-22 19:25:33,197 INFO [main] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:cleanUpApplicationsOnNMShutDown(556)) - Applications still running : [application_1448220296684_0001] 2015-11-22 19:25:33,198 INFO [main] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:cleanUpApplicationsOnNMShutDown(572)) - Waiting for Applications to be Finished 2015-11-22 19:25:33,198 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(463)) - Application application_1448220296684_0001 transitioned from RUNNING to APPLICATION_RESOURCES_CLEANINGUP 2015-11-22 19:25:33,198 INFO [DeletionService #3] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(480)) - Deleting absolute path : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220296684_0001 2015-11-22 19:25:33,198 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event APPLICATION_STOP for appId application_1448220296684_0001 2015-11-22 19:25:33,198 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(463)) - Application application_1448220296684_0001 transitioned from APPLICATION_RESOURCES_CLEANINGUP to FINISHED 2015-11-22 19:25:33,198 INFO [AsyncDispatcher event handler] loghandler.NonAggregatingLogHandler (NonAggregatingLogHandler.java:handle(167)) - Scheduling Log Deletion for application: application_1448220296684_0001, with delay of 10800 seconds 2015-11-22 19:25:37,198 INFO [main] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:cleanUpApplicationsOnNMShutDown(589)) - Done waiting for Applications to be Finished. Still alive: [application_1448220296684_0001] 2015-11-22 19:25:37,199 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 48952 2015-11-22 19:25:37,202 INFO [IPC Server listener on 48952] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 48952 2015-11-22 19:25:37,203 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:25:47,206 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 47193 2015-11-22 19:25:47,207 INFO [IPC Server listener on 47193] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 47193 2015-11-22 19:25:47,207 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:25:47,209 WARN [Node Resource Monitor] nodemanager.NodeResourceMonitorImpl (NodeResourceMonitorImpl.java:run(154)) - org.apache.hadoop.yarn.server.nodemanager.NodeResourceMonitorImpl is interrupted. Exiting. 2015-11-22 19:25:47,209 INFO [Public Localizer] localizer.ResourceLocalizationService (ResourceLocalizationService.java:run(908)) - Public cache exiting 2015-11-22 19:25:52,212 WARN [main] server.MiniYARNCluster (MiniYARNCluster.java:waitForAppMastersToFinish(474)) - Stopping RM while some app masters are still alive 2015-11-22 19:25:52,214 ERROR [Thread[Thread-1677,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(670)) - ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted 2015-11-22 19:25:52,215 INFO [main] mortbay.log (Slf4jLog.java:info(67)) - Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:0 2015-11-22 19:25:52,317 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 51403 2015-11-22 19:25:52,320 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:25:52,320 INFO [IPC Server listener on 51403] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 51403 2015-11-22 19:25:52,321 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 49153 2015-11-22 19:25:52,322 INFO [IPC Server listener on 49153] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 49153 2015-11-22 19:25:52,322 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:25:52,323 INFO [main] resourcemanager.ResourceManager (ResourceManager.java:transitionToStandby(1051)) - Transitioning to standby state 2015-11-22 19:25:52,324 WARN [ApplicationMaster Launcher] amlauncher.ApplicationMasterLauncher (ApplicationMasterLauncher.java:run(122)) - org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher$LauncherThread interrupted. Returning. 2015-11-22 19:25:52,325 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 52849 2015-11-22 19:25:52,328 INFO [IPC Server listener on 52849] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 52849 2015-11-22 19:25:52,328 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 56122 2015-11-22 19:25:52,328 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:25:52,331 INFO [IPC Server listener on 56122] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 56122 2015-11-22 19:25:52,331 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:25:52,331 ERROR [ResourceManager Event Processor] resourcemanager.ResourceManager (ResourceManager.java:run(691)) - Returning, interrupted : java.lang.InterruptedException 2015-11-22 19:25:52,331 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:serviceStop(141)) - AsyncDispatcher is draining to stop, igonring any new events. 2015-11-22 19:25:52,332 INFO [Ping Checker] util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(135)) - NMLivelinessMonitor thread interrupted 2015-11-22 19:25:52,332 INFO [Ping Checker] util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(135)) - AMLivelinessMonitor thread interrupted 2015-11-22 19:25:52,332 INFO [Ping Checker] util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(135)) - org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.ContainerAllocationExpirer thread interrupted 2015-11-22 19:25:52,333 ERROR [Thread[Thread-1508,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(670)) - ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted 2015-11-22 19:25:52,333 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stop(211)) - Stopping ApplicationHistoryServer metrics system... 2015-11-22 19:25:52,334 INFO [Ping Checker] util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(135)) - AMLivelinessMonitor thread interrupted 2015-11-22 19:25:52,338 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stop(217)) - ApplicationHistoryServer metrics system stopped. 2015-11-22 19:25:52,338 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:shutdown(608)) - ApplicationHistoryServer metrics system shutdown complete. 2015-11-22 19:25:52,338 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:serviceStop(141)) - AsyncDispatcher is draining to stop, igonring any new events. 2015-11-22 19:25:52,338 INFO [main] resourcemanager.ResourceManager (ResourceManager.java:transitionToStandby(1058)) - Transitioned to standby state 2015-11-22 19:25:52,349 INFO [main] distributedshell.TestDistributedShell (TestDistributedShell.java:setupInternal(77)) - Starting up YARN cluster 2015-11-22 19:25:52,357 INFO [main] net.ServerSocketUtil (ServerSocketUtil.java:getPort(52)) - Using port 10021 2015-11-22 19:25:52,368 INFO [main] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2363)) - found resource core-site.xml at file:/home/ubuntu/hadoop-dev/hadoop-common-project/hadoop-common/target/test-classes/core-site.xml 2015-11-22 19:25:52,378 WARN [main] conf.Configuration (Configuration.java:loadProperty(2687)) - core-site.xml:an attempt to override final parameter: hadoop.tmp.dir; Ignoring. 2015-11-22 19:25:52,378 INFO [main] security.Groups (Groups.java:refresh(256)) - clearing userToGroupsMap cache 2015-11-22 19:25:52,379 INFO [main] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2363)) - found resource yarn-site.xml at file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/test-classes/yarn-site.xml 2015-11-22 19:25:52,393 WARN [main] conf.Configuration (Configuration.java:loadProperty(2687)) - yarn-site.xml:an attempt to override final parameter: hadoop.tmp.dir; Ignoring. 2015-11-22 19:25:52,393 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.RMFatalEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMFatalEventDispatcher 2015-11-22 19:25:52,394 INFO [main] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:(75)) - NMTokenKeyRollingInterval: 86400000ms and NMTokenKeyActivationDelay: 900000ms 2015-11-22 19:25:52,394 INFO [main] security.RMContainerTokenSecretManager (RMContainerTokenSecretManager.java:(78)) - ContainerTokenKeyRollingInterval: 86400000ms and ContainerTokenKeyActivationDelay: 900000ms 2015-11-22 19:25:52,394 INFO [main] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:(94)) - AMRMTokenKeyRollingInterval: 86400000ms and AMRMTokenKeyActivationDelay: 900000 ms 2015-11-22 19:25:52,394 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStoreEventType for class org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$ForwardingEventHandler 2015-11-22 19:25:52,394 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.NodesListManagerEventType for class org.apache.hadoop.yarn.server.resourcemanager.NodesListManager 2015-11-22 19:25:52,395 INFO [main] resourcemanager.ResourceManager (ResourceManager.java:createScheduler(295)) - Using Scheduler: org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler 2015-11-22 19:25:52,395 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.SchedulerEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher 2015-11-22 19:25:52,395 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationEventDispatcher 2015-11-22 19:25:52,395 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationAttemptEventDispatcher 2015-11-22 19:25:52,395 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$NodeEventDispatcher 2015-11-22 19:25:52,396 INFO [main] impl.MetricsConfig (MetricsConfig.java:loadFirst(115)) - loaded properties from hadoop-metrics2.properties 2015-11-22 19:25:52,397 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:startTimer(374)) - Scheduled snapshot period at 10 second(s). 2015-11-22 19:25:52,397 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:start(192)) - ResourceManager metrics system started 2015-11-22 19:25:52,398 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.RMAppManagerEventType for class org.apache.hadoop.yarn.server.resourcemanager.RMAppManager 2015-11-22 19:25:52,398 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncherEventType for class org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher 2015-11-22 19:25:52,398 WARN [main] util.MBeans (MBeans.java:register(73)) - Failed to register MBean "Hadoop:service=ResourceManager,name=RMNMInfo": Instance already exists. 2015-11-22 19:25:52,399 INFO [main] resourcemanager.RMNMInfo (RMNMInfo.java:(63)) - Registered RMNMInfo MBean 2015-11-22 19:25:52,399 INFO [main] util.HostsFileReader (HostsFileReader.java:refresh(132)) - Refreshing hosts (include/exclude) list 2015-11-22 19:25:52,400 INFO [main] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2363)) - found resource capacity-scheduler.xml at jar:file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/target/hadoop-yarn-server-tests-2.8.0-SNAPSHOT-tests.jar!/capacity-scheduler.xml 2015-11-22 19:25:52,411 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(647)) - max alloc mb per queue for root is undefined 2015-11-22 19:25:52,411 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(651)) - max alloc vcore per queue for root is undefined 2015-11-22 19:25:52,412 INFO [main] capacity.ParentQueue (ParentQueue.java:setupQueueConfigs(125)) - root, capacity=1.0, asboluteCapacity=1.0, maxCapacity=1.0, asboluteMaxCapacity=1.0, state=RUNNING, acls=ADMINISTER_QUEUE:*SUBMIT_APP:*, labels=*, , reservationsContinueLooking=true 2015-11-22 19:25:52,412 INFO [main] capacity.ParentQueue (ParentQueue.java:(104)) - Initialized parent-queue root name=root, fullname=root 2015-11-22 19:25:52,413 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(647)) - max alloc mb per queue for root.default is undefined 2015-11-22 19:25:52,413 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(651)) - max alloc vcore per queue for root.default is undefined 2015-11-22 19:25:52,414 INFO [main] capacity.LeafQueue (LeafQueue.java:setupQueueConfigs(217)) - Initializing default capacity = 1.0 [= (float) configuredCapacity / 100 ] asboluteCapacity = 1.0 [= parentAbsoluteCapacity * capacity ] maxCapacity = 1.0 [= configuredMaxCapacity ] absoluteMaxCapacity = 1.0 [= 1.0 maximumCapacity undefined, (parentAbsoluteMaxCapacity * maximumCapacity) / 100 otherwise ] userLimit = 100 [= configuredUserLimit ] userLimitFactor = 1.0 [= configuredUserLimitFactor ] maxApplications = 10000 [= configuredMaximumSystemApplicationsPerQueue or (int)(configuredMaximumSystemApplications * absoluteCapacity)] maxApplicationsPerUser = 10000 [= (int)(maxApplications * (userLimit / 100.0f) * userLimitFactor) ] usedCapacity = 0.0 [= usedResourcesMemory / (clusterResourceMemory * absoluteCapacity)] absoluteUsedCapacity = 0.0 [= usedResourcesMemory / clusterResourceMemory] maxAMResourcePerQueuePercent = 0.1 [= configuredMaximumAMResourcePercent ] minimumAllocationFactor = 0.984375 [= (float)(maximumAllocationMemory - minimumAllocationMemory) / maximumAllocationMemory ] maximumAllocation = [= configuredMaxAllocation ] numContainers = 0 [= currentNumContainers ] state = RUNNING [= configuredState ] acls = ADMINISTER_QUEUE:*SUBMIT_APP:* [= configuredAcls ] nodeLocalityDelay = -1 labels=*, nodeLocalityDelay = -1 reservationsContinueLooking = true preemptionDisabled = true defaultAppPriorityPerQueue = 0 2015-11-22 19:25:52,414 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:parseQueue(657)) - Initialized queue: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=0, numContainers=0 2015-11-22 19:25:52,414 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:parseQueue(657)) - Initialized queue: root: numChildQueue= 1, capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.0, numApps=0, numContainers=0 2015-11-22 19:25:52,415 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:initializeQueues(501)) - Initialized root queue root: numChildQueue= 1, capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.0, numApps=0, numContainers=0 2015-11-22 19:25:52,415 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:getUserGroupMappingPlacementRule(449)) - Initialized queue mappings, override: false 2015-11-22 19:25:52,415 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:initScheduler(314)) - Initialized CapacityScheduler with calculator=class org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator, minimumAllocation=<>, maximumAllocation=<>, asynchronousScheduling=false, asyncScheduleInterval=5ms 2015-11-22 19:25:52,415 INFO [main] metrics.SystemMetricsPublisher (SystemMetricsPublisher.java:serviceInit(95)) - YARN system metrics publishing service is not enabled 2015-11-22 19:25:52,415 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEventType for class org.apache.hadoop.yarn.server.MiniYARNCluster$1 2015-11-22 19:25:52,416 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:prepareDirs(558)) - Created localDir in /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0 2015-11-22 19:25:52,416 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:prepareDirs(558)) - Created logDir in /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-logDir-nm-0_0 2015-11-22 19:25:52,416 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceInit(540)) - Starting NM: 0 2015-11-22 19:25:52,426 INFO [main] nodemanager.NodeManager (NodeManager.java:getNodeHealthScriptRunner(255)) - Node Manager health check script is not available or doesn't have execute permission, so not starting the node health script runner. 2015-11-22 19:25:52,427 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ContainerEventDispatcher 2015-11-22 19:25:52,427 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ApplicationEventDispatcher 2015-11-22 19:25:52,427 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizationEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService 2015-11-22 19:25:52,427 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServicesEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices 2015-11-22 19:25:52,427 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl 2015-11-22 19:25:52,427 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncherEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncher 2015-11-22 19:25:52,427 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.ContainerManagerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl 2015-11-22 19:25:52,427 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.NodeManagerEventType for class org.apache.hadoop.yarn.server.MiniYARNCluster$ShortCircuitedNodeManager 2015-11-22 19:25:52,427 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:init(159)) - NodeManager metrics system started (again) 2015-11-22 19:25:52,429 INFO [main] nodemanager.NodeResourceMonitorImpl (NodeResourceMonitorImpl.java:serviceInit(76)) - Using ResourceCalculatorPlugin : org.apache.hadoop.yarn.util.ResourceCalculatorPlugin@4e3958e7 2015-11-22 19:25:52,429 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.event.LogHandlerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.NonAggregatingLogHandler 2015-11-22 19:25:52,429 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.sharedcache.SharedCacheUploadEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.sharedcache.SharedCacheUploadService 2015-11-22 19:25:52,429 INFO [main] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:serviceInit(261)) - AMRMProxyService is disabled 2015-11-22 19:25:52,429 INFO [main] localizer.ResourceLocalizationService (ResourceLocalizationService.java:validateConf(224)) - per directory file limit = 8192 2015-11-22 19:25:52,440 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerTracker 2015-11-22 19:25:52,440 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(116)) - Using ResourceCalculatorPlugin : org.apache.hadoop.yarn.util.ResourceCalculatorPlugin@77f80c04 2015-11-22 19:25:52,441 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(121)) - Using ResourceCalculatorProcessTree : null 2015-11-22 19:25:52,441 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(158)) - Physical memory check enabled: false 2015-11-22 19:25:52,441 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(159)) - Virtual memory check enabled: false 2015-11-22 19:25:52,441 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:isEnabled(206)) - Neither virutal-memory nor physical-memory monitoring is needed. Not running the monitor-thread 2015-11-22 19:25:52,441 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(162)) - ContainersMonitor enabled: false 2015-11-22 19:25:52,441 INFO [main] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:serviceInit(175)) - Nodemanager resources: memory set to 4096MB. 2015-11-22 19:25:52,441 INFO [main] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:serviceInit(176)) - Nodemanager resources: vcores set to 8. 2015-11-22 19:25:52,442 INFO [main] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:serviceInit(208)) - Initialized nodemanager with : physical-memory=4096 virtual-memory=8602 virtual-cores=8 2015-11-22 19:25:52,442 WARN [main] applicationhistoryservice.ApplicationHistoryServer (ApplicationHistoryServer.java:createApplicationHistoryManager(218)) - The filesystem based application history store is deprecated. 2015-11-22 19:25:52,442 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:init(159)) - ApplicationHistoryServer metrics system started (again) 2015-11-22 19:25:52,443 INFO [main] applicationhistoryservice.ApplicationHistoryManagerImpl (ApplicationHistoryManagerImpl.java:serviceInit(61)) - ApplicationHistory Init 2015-11-22 19:25:52,443 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:startResourceManager(326)) - Waiting for RM to start... 2015-11-22 19:25:52,443 INFO [RM-0] resourcemanager.ResourceManager (ResourceManager.java:transitionToActive(1024)) - Transitioning to active state 2015-11-22 19:25:52,444 INFO [RM-0] recovery.RMStateStore (RMStateStore.java:transition(482)) - Updating AMRMToken 2015-11-22 19:25:52,444 INFO [RM-0] security.RMContainerTokenSecretManager (RMContainerTokenSecretManager.java:rollMasterKey(106)) - Rolling master-key for container-tokens 2015-11-22 19:25:52,444 INFO [RM-0] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:rollMasterKey(95)) - Rolling master-key for nm-tokens 2015-11-22 19:25:52,445 INFO [RM-0] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:25:52,445 INFO [RM-0] security.RMDelegationTokenSecretManager (RMDelegationTokenSecretManager.java:storeNewMasterKey(87)) - storing master key with keyID 1 2015-11-22 19:25:52,445 INFO [RM-0] recovery.RMStateStore (RMStateStore.java:transition(433)) - Storing RMDTMasterKey. 2015-11-22 19:25:52,445 INFO [Thread[Thread-1953,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2015-11-22 19:25:52,445 INFO [Thread[Thread-1953,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:25:52,446 INFO [Thread[Thread-1953,5,main]] security.RMDelegationTokenSecretManager (RMDelegationTokenSecretManager.java:storeNewMasterKey(87)) - storing master key with keyID 2 2015-11-22 19:25:52,446 INFO [Thread[Thread-1953,5,main]] recovery.RMStateStore (RMStateStore.java:transition(433)) - Storing RMDTMasterKey. 2015-11-22 19:25:52,447 INFO [RM-0] nodelabels.CommonNodeLabelsManager (CommonNodeLabelsManager.java:addToCluserNodeLabels(309)) - Add labels: [] 2015-11-22 19:25:52,447 INFO [RM-0] nodelabels.CommonNodeLabelsManager (CommonNodeLabelsManager.java:internalUpdateLabelsOnNodes(638)) - REPLACE labels on nodes: 2015-11-22 19:25:52,447 INFO [RM-0] nodelabels.CommonNodeLabelsManager (CommonNodeLabelsManager.java:internalUpdateLabelsOnNodes(640)) - NM=ip-172-31-20-42.ap-northeast-1.compute.internal:58694, labels=[x] 2015-11-22 19:25:52,448 INFO [RM-0] nodelabels.FileSystemNodeLabelsStore (FileSystemNodeLabelsStore.java:recover(279)) - Finished write mirror at:file:/tmp/hadoop-yarn-ubuntu/node-labels/nodelabel.mirror 2015-11-22 19:25:52,448 INFO [RM-0] nodelabels.FileSystemNodeLabelsStore (FileSystemNodeLabelsStore.java:recover(280)) - Finished create editlog file at:file:/tmp/hadoop-yarn-ubuntu/node-labels/nodelabel.editlog 2015-11-22 19:25:52,448 INFO [RM-0] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.nodelabels.event.NodeLabelsStoreEventType for class org.apache.hadoop.yarn.nodelabels.CommonNodeLabelsManager$ForwardingEventHandler 2015-11-22 19:25:52,449 INFO [RM-0] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:25:52,450 INFO [Socket Reader #1 for port 42955] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 42955 2015-11-22 19:25:52,451 INFO [RM-0] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.server.api.ResourceTrackerPB to the server 2015-11-22 19:25:52,451 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:25:52,451 INFO [IPC Server listener on 42955] ipc.Server (Server.java:run(794)) - IPC Server listener on 42955: starting 2015-11-22 19:25:52,460 INFO [RM-0] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:25:52,461 INFO [Socket Reader #1 for port 48123] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 48123 2015-11-22 19:25:52,463 INFO [RM-0] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB to the server 2015-11-22 19:25:52,463 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:25:52,463 INFO [IPC Server listener on 48123] ipc.Server (Server.java:run(794)) - IPC Server listener on 48123: starting 2015-11-22 19:25:52,467 INFO [RM-0] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:25:52,468 INFO [Socket Reader #1 for port 43161] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 43161 2015-11-22 19:25:52,469 INFO [RM-0] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.api.ApplicationClientProtocolPB to the server 2015-11-22 19:25:52,469 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:25:52,469 INFO [IPC Server listener on 43161] ipc.Server (Server.java:run(794)) - IPC Server listener on 43161: starting 2015-11-22 19:25:52,474 INFO [RM-0] resourcemanager.ResourceManager (ResourceManager.java:transitionToActive(1040)) - Transitioned to active state 2015-11-22 19:25:52,475 INFO [RM-0] server.AuthenticationFilter (AuthenticationFilter.java:constructSecretProvider(304)) - Unable to initialize FileSignerSecretProvider, falling back to use random secrets. 2015-11-22 19:25:52,475 INFO [RM-0] http.HttpRequestLog (HttpRequestLog.java:getRequestLog(80)) - Http request log for http.requests.resourcemanager is not defined 2015-11-22 19:25:52,476 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter) 2015-11-22 19:25:52,477 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context cluster 2015-11-22 19:25:52,477 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context logs 2015-11-22 19:25:52,477 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context static 2015-11-22 19:25:52,477 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context cluster 2015-11-22 19:25:52,477 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2015-11-22 19:25:52,478 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2015-11-22 19:25:52,478 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'Timeline Authentication Filter' (class=org.apache.hadoop.yarn.server.timeline.security.TimelineAuthenticationFilter) 2015-11-22 19:25:52,479 INFO [RM-0] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /cluster/* 2015-11-22 19:25:52,479 INFO [RM-0] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /ws/* 2015-11-22 19:25:52,484 INFO [RM-0] webapp.WebApps (WebApps.java:build(288)) - Registered webapp guice modules 2015-11-22 19:25:52,484 INFO [RM-0] http.HttpServer2 (HttpServer2.java:openListeners(917)) - Jetty bound to port 37870 2015-11-22 19:25:52,484 INFO [RM-0] mortbay.log (Slf4jLog.java:info(67)) - jetty-6.1.26 2015-11-22 19:25:52,496 INFO [RM-0] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:25:52,496 INFO [Thread[Thread-2122,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2015-11-22 19:25:52,497 INFO [Thread[Thread-2122,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:25:52,498 INFO [RM-0] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:25:52,498 INFO [Thread[Thread-2124,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) Nov 22, 2015 7:25:52 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.resourcemanager.webapp.JAXBContextResolver as a provider class Nov 22, 2015 7:25:52 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices as a root resource class Nov 22, 2015 7:25:52 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a provider class Nov 22, 2015 7:25:52 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate INFO: Initiating Jersey application, version 'Jersey: 1.9 09/02/2011 11:17 AM' Nov 22, 2015 7:25:52 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.resourcemanager.webapp.JAXBContextResolver to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:25:52 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.webapp.GenericExceptionHandler to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:25:52 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices to GuiceManagedComponentProvider with the scope "Singleton" 2015-11-22 19:25:52,730 INFO [RM-0] mortbay.log (Slf4jLog.java:info(67)) - Started HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:37870 2015-11-22 19:25:52,731 INFO [RM-0] webapp.WebApps (WebApps.java:start(307)) - Web app cluster started at 37870 2015-11-22 19:25:52,734 INFO [RM-0] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:25:52,735 INFO [Socket Reader #1 for port 52421] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 52421 2015-11-22 19:25:52,736 INFO [RM-0] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.server.api.ResourceManagerAdministrationProtocolPB to the server 2015-11-22 19:25:52,736 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:25:52,741 INFO [IPC Server listener on 52421] ipc.Server (Server.java:run(794)) - IPC Server listener on 52421: starting 2015-11-22 19:25:53,944 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:startResourceManager(339)) - MiniYARN ResourceManager address: ip-172-31-20-42.ap-northeast-1.compute.internal:43161 2015-11-22 19:25:53,944 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:startResourceManager(341)) - MiniYARN ResourceManager web address: ip-172-31-20-42.ap-northeast-1.compute.internal:37870 2015-11-22 19:25:53,944 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(459)) - MiniYARN ResourceManager address: ip-172-31-20-42.ap-northeast-1.compute.internal:43161 2015-11-22 19:25:53,944 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(461)) - MiniYARN ResourceManager web address: ip-172-31-20-42.ap-northeast-1.compute.internal:37870 2015-11-22 19:25:53,944 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(575)) - Waiting for NM 0 to start... 2015-11-22 19:25:53,947 INFO [Thread-2131] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:25:53,947 INFO [Socket Reader #1 for port 49391] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 49391 2015-11-22 19:25:53,948 INFO [Thread-2131] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.api.ContainerManagementProtocolPB to the server 2015-11-22 19:25:53,949 INFO [Thread-2131] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:serviceStart(464)) - Blocking new container-requests as container manager rpc server is still starting. 2015-11-22 19:25:53,949 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:25:53,949 INFO [IPC Server listener on 49391] ipc.Server (Server.java:run(794)) - IPC Server listener on 49391: starting 2015-11-22 19:25:53,957 INFO [Thread-2131] security.NMContainerTokenSecretManager (NMContainerTokenSecretManager.java:setNodeId(260)) - Updating node address : ip-172-31-20-42.ap-northeast-1.compute.internal:49391 2015-11-22 19:25:53,958 INFO [Thread-2131] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:25:53,958 INFO [Socket Reader #1 for port 40586] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 40586 2015-11-22 19:25:53,961 INFO [Thread-2131] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB to the server 2015-11-22 19:25:53,961 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:25:53,961 INFO [IPC Server listener on 40586] ipc.Server (Server.java:run(794)) - IPC Server listener on 40586: starting 2015-11-22 19:25:53,964 INFO [Thread-2131] localizer.ResourceLocalizationService (ResourceLocalizationService.java:serviceStart(363)) - Localizer started on port 40586 2015-11-22 19:25:53,974 INFO [Thread-2131] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:serviceStart(509)) - ContainerManager started at /172.31.20.42:49391 2015-11-22 19:25:53,974 INFO [Thread-2131] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:serviceStart(510)) - ContainerManager bound to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:0 2015-11-22 19:25:53,974 INFO [Thread-2131] webapp.WebServer (WebServer.java:serviceStart(71)) - Instantiating NMWebApp at ip-172-31-20-42.ap-northeast-1.compute.internal:0 2015-11-22 19:25:53,976 INFO [Thread-2131] server.AuthenticationFilter (AuthenticationFilter.java:constructSecretProvider(304)) - Unable to initialize FileSignerSecretProvider, falling back to use random secrets. 2015-11-22 19:25:53,976 INFO [Thread-2131] http.HttpRequestLog (HttpRequestLog.java:getRequestLog(80)) - Http request log for http.requests.nodemanager is not defined 2015-11-22 19:25:53,976 INFO [Thread-2131] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter) 2015-11-22 19:25:53,977 INFO [Thread-2131] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context node 2015-11-22 19:25:53,977 INFO [Thread-2131] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context logs 2015-11-22 19:25:53,977 INFO [Thread-2131] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context static 2015-11-22 19:25:53,977 INFO [Thread-2131] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context node 2015-11-22 19:25:53,978 INFO [Thread-2131] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2015-11-22 19:25:53,978 INFO [Thread-2131] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2015-11-22 19:25:53,978 INFO [Thread-2131] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'Timeline Authentication Filter' (class=org.apache.hadoop.yarn.server.timeline.security.TimelineAuthenticationFilter) 2015-11-22 19:25:53,979 INFO [Thread-2131] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /node/* 2015-11-22 19:25:53,979 INFO [Thread-2131] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /ws/* Nov 22, 2015 7:25:53 PM com.google.inject.servlet.GuiceFilter setPipeline WARNING: Multiple Servlet injectors detected. This is a warning indicating that you have more than one GuiceFilter running in your web application. If this is deliberate, you may safely ignore this message. If this is NOT deliberate however, your application may not work as expected. 2015-11-22 19:25:53,985 INFO [Thread-2131] webapp.WebApps (WebApps.java:build(288)) - Registered webapp guice modules 2015-11-22 19:25:53,986 INFO [Thread-2131] http.HttpServer2 (HttpServer2.java:openListeners(917)) - Jetty bound to port 39768 2015-11-22 19:25:53,986 INFO [Thread-2131] mortbay.log (Slf4jLog.java:info(67)) - jetty-6.1.26 2015-11-22 19:25:54,000 INFO [Thread-2131] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:25:54,000 INFO [Thread[Thread-2172,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2015-11-22 19:25:54,001 INFO [Thread[Thread-2172,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:25:54,001 INFO [Thread-2131] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:25:54,002 INFO [Thread[Thread-2174,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) Nov 22, 2015 7:25:54 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.nodemanager.webapp.NMWebServices as a root resource class Nov 22, 2015 7:25:54 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a provider class Nov 22, 2015 7:25:54 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.nodemanager.webapp.JAXBContextResolver as a provider class Nov 22, 2015 7:25:54 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate INFO: Initiating Jersey application, version 'Jersey: 1.9 09/02/2011 11:17 AM' Nov 22, 2015 7:25:54 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.nodemanager.webapp.JAXBContextResolver to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:25:54 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.webapp.GenericExceptionHandler to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:25:54 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.nodemanager.webapp.NMWebServices to GuiceManagedComponentProvider with the scope "Singleton" 2015-11-22 19:25:54,097 INFO [Thread-2131] mortbay.log (Slf4jLog.java:info(67)) - Started HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:39768 2015-11-22 19:25:54,098 INFO [Thread-2131] webapp.WebApps (WebApps.java:start(307)) - Web app node started at 39768 2015-11-22 19:25:54,101 INFO [Thread-2131] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:serviceStart(222)) - Node ID assigned is : ip-172-31-20-42.ap-northeast-1.compute.internal:49391 2015-11-22 19:25:54,101 INFO [Thread-2131] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNMContainerStatuses(550)) - Sending out 0 NM container statuses: [] 2015-11-22 19:25:54,101 INFO [Thread-2131] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:registerWithRM(330)) - Registering with RM using containers :[] 2015-11-22 19:25:54,102 INFO [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(566)) - ip-172-31-20-42.ap-northeast-1.compute.internal:49391 Node Transitioned from NEW to RUNNING 2015-11-22 19:25:54,102 INFO [Thread-2131] resourcemanager.ResourceTrackerService (ResourceTrackerService.java:registerNodeManager(375)) - NodeManager from node ip-172-31-20-42.ap-northeast-1.compute.internal(cmPort: 49391 httpPort: 39768) registered with capability: , assigned nodeId ip-172-31-20-42.ap-northeast-1.compute.internal:49391 2015-11-22 19:25:54,103 INFO [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:addNode(1427)) - Added node ip-172-31-20-42.ap-northeast-1.compute.internal:49391 clusterResource: 2015-11-22 19:25:54,104 INFO [Thread-2131] security.NMContainerTokenSecretManager (NMContainerTokenSecretManager.java:setMasterKey(138)) - Rolling master-key for container-tokens, got key with id 645143115 2015-11-22 19:25:54,104 INFO [Thread-2131] security.NMTokenSecretManagerInNM (NMTokenSecretManagerInNM.java:setMasterKey(135)) - Rolling master-key for container-tokens, got key with id 390440421 2015-11-22 19:25:54,105 INFO [Thread-2131] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:registerWithRM(391)) - Registered with ResourceManager as ip-172-31-20-42.ap-northeast-1.compute.internal:49391 with total resource of 2015-11-22 19:25:54,105 INFO [Thread-2131] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:registerWithRM(392)) - Notifying ContainerManager to unblock new container-requests 2015-11-22 19:25:54,945 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(728)) - Waiting for Timeline Server to start... 2015-11-22 19:25:54,946 INFO [Thread-2177] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:25:54,946 INFO [Thread[Thread-2179,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2015-11-22 19:25:54,946 INFO [Thread[Thread-2179,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:25:54,946 INFO [Thread-2177] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:25:54,947 INFO [Socket Reader #1 for port 48370] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 48370 2015-11-22 19:25:54,949 INFO [Thread-2177] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.api.ApplicationHistoryProtocolPB to the server 2015-11-22 19:25:54,949 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:25:54,949 INFO [IPC Server listener on 48370] ipc.Server (Server.java:run(794)) - IPC Server listener on 48370: starting 2015-11-22 19:25:54,953 INFO [Thread-2177] applicationhistoryservice.ApplicationHistoryClientService (ApplicationHistoryClientService.java:serviceStart(111)) - Instantiated ApplicationHistoryClientService at ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:48370 2015-11-22 19:25:54,953 INFO [Thread-2177] applicationhistoryservice.ApplicationHistoryManagerImpl (ApplicationHistoryManagerImpl.java:serviceStart(71)) - Starting ApplicationHistory 2015-11-22 19:25:54,954 INFO [Thread-2177] server.AuthenticationFilter (AuthenticationFilter.java:constructSecretProvider(304)) - Unable to initialize FileSignerSecretProvider, falling back to use random secrets. 2015-11-22 19:25:54,954 INFO [Thread-2177] http.HttpRequestLog (HttpRequestLog.java:getRequestLog(80)) - Http request log for http.requests.applicationhistory is not defined 2015-11-22 19:25:54,954 INFO [Thread-2177] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter) 2015-11-22 19:25:54,955 INFO [Thread-2177] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context applicationhistory 2015-11-22 19:25:54,955 INFO [Thread-2177] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context logs 2015-11-22 19:25:54,955 INFO [Thread-2177] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context static 2015-11-22 19:25:54,955 INFO [Thread-2177] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context applicationhistory 2015-11-22 19:25:54,956 INFO [Thread-2177] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2015-11-22 19:25:54,956 INFO [Thread-2177] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2015-11-22 19:25:54,956 INFO [Thread-2177] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'Timeline Authentication Filter' (class=org.apache.hadoop.yarn.server.timeline.security.TimelineAuthenticationFilter) 2015-11-22 19:25:54,957 INFO [Thread-2177] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /applicationhistory/* 2015-11-22 19:25:54,957 INFO [Thread-2177] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /ws/* Nov 22, 2015 7:25:54 PM com.google.inject.servlet.GuiceFilter setPipeline WARNING: Multiple Servlet injectors detected. This is a warning indicating that you have more than one GuiceFilter running in your web application. If this is deliberate, you may safely ignore this message. If this is NOT deliberate however, your application may not work as expected. 2015-11-22 19:25:54,961 INFO [Thread-2177] webapp.WebApps (WebApps.java:build(288)) - Registered webapp guice modules 2015-11-22 19:25:54,961 INFO [Thread-2177] http.HttpServer2 (HttpServer2.java:openListeners(917)) - Jetty bound to port 49401 2015-11-22 19:25:54,962 INFO [Thread-2177] mortbay.log (Slf4jLog.java:info(67)) - jetty-6.1.26 2015-11-22 19:25:54,972 INFO [Thread-2177] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:25:54,972 INFO [Thread[Thread-2197,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2015-11-22 19:25:54,973 INFO [Thread[Thread-2197,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:25:54,973 INFO [Thread-2177] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:25:54,974 INFO [Thread[Thread-2199,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) Nov 22, 2015 7:25:54 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider as a provider class Nov 22, 2015 7:25:54 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.AHSWebServices as a root resource class Nov 22, 2015 7:25:54 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.timeline.webapp.TimelineWebServices as a root resource class Nov 22, 2015 7:25:54 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a provider class Nov 22, 2015 7:25:54 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate INFO: Initiating Jersey application, version 'Jersey: 1.9 09/02/2011 11:17 AM' Nov 22, 2015 7:25:54 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.webapp.GenericExceptionHandler to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:25:54 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:25:55 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.AHSWebServices to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:25:55 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.timeline.webapp.TimelineWebServices to GuiceManagedComponentProvider with the scope "Singleton" 2015-11-22 19:25:55,053 INFO [Thread-2177] mortbay.log (Slf4jLog.java:info(67)) - Started HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:49401 2015-11-22 19:25:55,054 INFO [Thread-2177] applicationhistoryservice.ApplicationHistoryServer (ApplicationHistoryServer.java:startWebApp(342)) - Instantiating AHSWebApp at 49401 2015-11-22 19:25:56,445 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(739)) - MiniYARN ApplicationHistoryServer address: ip-172-31-20-42.ap-northeast-1.compute.internal:48370 2015-11-22 19:25:56,445 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(741)) - MiniYARN ApplicationHistoryServer web address: ip-172-31-20-42.ap-northeast-1.compute.internal:49401 2015-11-22 19:25:58,468 INFO [Thread-2201] distributedshell.TestDistributedShell (TestDistributedShell.java:testDSShell(191)) - Initializing DS Client 2015-11-22 19:25:58,514 INFO [Thread-2201] impl.TimelineClientImpl (TimelineClientImpl.java:serviceInit(296)) - Timeline service address: http://ip-172-31-20-42.ap-northeast-1.compute.internal:49401/ws/v1/timeline/ 2015-11-22 19:25:58,515 INFO [Thread-2201] distributedshell.TestDistributedShell (TestDistributedShell.java:testDSShell(195)) - Running DS Client 2015-11-22 19:25:58,515 INFO [Thread-2202] distributedshell.Client (Client.java:run(443)) - Running Client 2015-11-22 19:25:58,527 INFO [Thread-2202] client.RMProxy (RMProxy.java:createRMProxy(98)) - Connecting to ResourceManager at ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:43161 2015-11-22 19:25:58,533 INFO [Thread-2202] distributedshell.Client (Client.java:run(447)) - Got Cluster metric info from ASM, numNodeManagers=1 2015-11-22 19:25:58,535 INFO [Thread-2202] distributedshell.Client (Client.java:run(452)) - Got Cluster node info from ASM 2015-11-22 19:25:58,535 INFO [Thread-2202] distributedshell.Client (Client.java:run(454)) - Got node report from ASM for, nodeId=ip-172-31-20-42.ap-northeast-1.compute.internal:49391, nodeAddress=ip-172-31-20-42.ap-northeast-1.compute.internal:39768, nodeRackName=/default-rack, nodeNumContainers=0 2015-11-22 19:25:58,536 INFO [Thread-2202] distributedshell.Client (Client.java:run(462)) - Queue info, queueName=default, queueCurrentCapacity=0.0, queueMaxCapacity=1.0, queueApplicationCount=0, queueChildQueueCount=0 2015-11-22 19:25:58,537 INFO [Thread-2202] distributedshell.Client (Client.java:run(472)) - User ACL Info for Queue, queueName=root, userAcl=SUBMIT_APPLICATIONS 2015-11-22 19:25:58,537 INFO [Thread-2202] distributedshell.Client (Client.java:run(472)) - User ACL Info for Queue, queueName=root, userAcl=ADMINISTER_QUEUE 2015-11-22 19:25:58,538 INFO [Thread-2202] distributedshell.Client (Client.java:run(472)) - User ACL Info for Queue, queueName=default, userAcl=SUBMIT_APPLICATIONS 2015-11-22 19:25:58,538 INFO [Thread-2202] distributedshell.Client (Client.java:run(472)) - User ACL Info for Queue, queueName=default, userAcl=ADMINISTER_QUEUE 2015-11-22 19:25:58,574 INFO [Thread-2201] impl.TimelineClientImpl (TimelineClientImpl.java:serviceInit(296)) - Timeline service address: http://ip-172-31-20-42.ap-northeast-1.compute.internal:49401/ws/v1/timeline/ 2015-11-22 19:25:58,592 INFO [Thread-2201] client.RMProxy (RMProxy.java:createRMProxy(98)) - Connecting to ResourceManager at ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:43161 2015-11-22 19:25:58,602 INFO [Thread-2202] impl.TimelineClientImpl (TimelineClientImpl.java:serviceInit(296)) - Timeline service address: http://ip-172-31-20-42.ap-northeast-1.compute.internal:49401/ws/v1/timeline/ 2015-11-22 19:25:58,704 INFO [Thread-2202] distributedshell.Client (Client.java:prepareTimelineDomain(853)) - Put the timeline domain: {"id":"TEST_DOMAIN","readers":"reader_user reader_group","writers":"writer_user writer_group"} 2015-11-22 19:25:58,705 INFO [IPC Server handler 14 on 43161] resourcemanager.ClientRMService (ClientRMService.java:getNewApplicationId(305)) - Allocated new applicationId: 1 2015-11-22 19:25:58,705 INFO [Thread-2202] distributedshell.Client (Client.java:run(491)) - Max mem capability of resources in this cluster 4096 2015-11-22 19:25:58,705 INFO [Thread-2202] distributedshell.Client (Client.java:run(502)) - Max virtual cores capability of resources in this cluster 4 2015-11-22 19:25:58,706 INFO [Thread-2202] distributedshell.Client (Client.java:run(528)) - Copy App Master jar from local filesystem and add to local environment 2015-11-22 19:25:58,713 INFO [Thread-2202] distributedshell.Client (Client.java:run(577)) - Set the environment for the application master 2015-11-22 19:25:58,713 INFO [Thread-2202] distributedshell.Client (Client.java:run(619)) - Setting up app master command 2015-11-22 19:25:58,713 INFO [Thread-2202] distributedshell.Client (Client.java:run(650)) - Completed setting up app master command {{JAVA_HOME}}/bin/java -Xmx512m org.apache.hadoop.yarn.applications.distributedshell.ApplicationMaster --container_memory 128 --container_vcores 1 --num_containers 2 --priority 0 1>/AppMaster.stdout 2>/AppMaster.stderr 2015-11-22 19:25:58,713 INFO [Thread-2202] distributedshell.Client (Client.java:run(706)) - Submitting application to ASM 2015-11-22 19:25:58,714 INFO [IPC Server handler 15 on 43161] capacity.CapacityScheduler (CapacityScheduler.java:checkAndGetApplicationPriority(1941)) - Priority '0' is acceptable in queue : default for application: application_1448220352443_0001 for the user: ubuntu 2015-11-22 19:25:58,714 WARN [IPC Server handler 15 on 43161] rmapp.RMAppImpl (RMAppImpl.java:(420)) - The specific max attempts: 0 for application: 1 is invalid, because it is out of the range [1, 2]. Use the global max attempts instead. 2015-11-22 19:25:58,714 INFO [IPC Server handler 15 on 43161] resourcemanager.ClientRMService (ClientRMService.java:submitApplication(602)) - Application with id 1 submitted by user ubuntu 2015-11-22 19:25:58,715 INFO [IPC Server handler 15 on 43161] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(154)) - USER=ubuntu IP=172.31.20.42 OPERATION=Submit Application Request TARGET=ClientRMService RESULT=SUCCESS APPID=application_1448220352443_0001 2015-11-22 19:25:58,715 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:transition(1079)) - Storing application with id application_1448220352443_0001 2015-11-22 19:25:58,715 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220352443_0001 State change from NEW to NEW_SAVING 2015-11-22 19:25:58,715 INFO [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:transition(208)) - Storing info for app: application_1448220352443_0001 2015-11-22 19:25:58,715 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220352443_0001 State change from NEW_SAVING to SUBMITTED 2015-11-22 19:25:58,715 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:addApplication(349)) - Application added - appId: application_1448220352443_0001 user: ubuntu leaf-queue of parent: root #applications: 1 2015-11-22 19:25:58,715 INFO [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:addApplication(765)) - Accepted application application_1448220352443_0001 from user: ubuntu, in queue: default 2015-11-22 19:25:58,716 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220352443_0001 State change from SUBMITTED to ACCEPTED 2015-11-22 19:25:58,716 INFO [AsyncDispatcher event handler] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:registerAppAttempt(661)) - Registering app attempt : appattempt_1448220352443_0001_000001 2015-11-22 19:25:58,716 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220352443_0001_000001 State change from NEW to SUBMITTED 2015-11-22 19:25:58,717 INFO [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:activateApplications(705)) - Application application_1448220352443_0001 from user: ubuntu activated in queue: default 2015-11-22 19:25:58,717 INFO [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:addApplicationAttempt(720)) - Application added - appId: application_1448220352443_0001 user: ubuntu, leaf-queue: default #user-pending-applications: 0 #user-active-applications: 1 #queue-pending-applications: 0 #queue-active-applications: 1 2015-11-22 19:25:58,717 INFO [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:addApplicationAttempt(805)) - Added Application Attempt appattempt_1448220352443_0001_000001 to scheduler from user ubuntu in queue default 2015-11-22 19:25:58,718 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220352443_0001_000001 State change from SUBMITTED to SCHEDULED 2015-11-22 19:25:58,918 INFO [Thread-2202] impl.YarnClientImpl (YarnClientImpl.java:submitApplication(279)) - Submitted application application_1448220352443_0001 2015-11-22 19:25:59,109 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220352443_0001_01_000001 Container Transitioned from NEW to ALLOCATED 2015-11-22 19:25:59,109 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220352443_0001 CONTAINERID=container_1448220352443_0001_01_000001 2015-11-22 19:25:59,110 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(154)) - Assigned container container_1448220352443_0001_01_000001 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:49391, which has 1 containers, used and available after allocation 2015-11-22 19:25:59,110 INFO [ResourceManager Event Processor] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(89)) - assignedContainer application attempt=appattempt_1448220352443_0001_000001 container=container_1448220352443_0001_01_000001 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@23d8cfaf clusterResource= type=OFF_SWITCH 2015-11-22 19:25:59,110 INFO [AsyncDispatcher event handler] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:createAndGetNMToken(200)) - Sending NMToken for nodeId : ip-172-31-20-42.ap-northeast-1.compute.internal:49391 for container : container_1448220352443_0001_01_000001 2015-11-22 19:25:59,111 INFO [AsyncDispatcher event handler] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220352443_0001_01_000001 Container Transitioned from ALLOCATED to ACQUIRED 2015-11-22 19:25:59,111 INFO [AsyncDispatcher event handler] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:clearNodeSetForAttempt(146)) - Clear node set for appattempt_1448220352443_0001_000001 2015-11-22 19:25:59,111 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(589)) - Re-sorting assigned queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.125, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 2015-11-22 19:25:59,111 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:storeAttempt(2026)) - Storing attempt: AppId: application_1448220352443_0001 AttemptId: appattempt_1448220352443_0001_000001 MasterContainer: Container: [ContainerId: container_1448220352443_0001_01_000001, NodeId: ip-172-31-20-42.ap-northeast-1.compute.internal:49391, NodeHttpAddress: ip-172-31-20-42.ap-northeast-1.compute.internal:39768, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.31.20.42:49391 }, ] 2015-11-22 19:25:59,111 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(461)) - assignedContainer queue=root usedCapacity=0.125 absoluteUsedCapacity=0.125 used= cluster= 2015-11-22 19:25:59,111 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220352443_0001_000001 State change from SCHEDULED to ALLOCATED_SAVING 2015-11-22 19:25:59,112 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220352443_0001_000001 State change from ALLOCATED_SAVING to ALLOCATED 2015-11-22 19:25:59,112 INFO [ApplicationMasterLauncher #0] amlauncher.AMLauncher (AMLauncher.java:run(254)) - Launching masterappattempt_1448220352443_0001_000001 2015-11-22 19:25:59,114 INFO [ApplicationMasterLauncher #0] amlauncher.AMLauncher (AMLauncher.java:launch(106)) - Setting up container Container: [ContainerId: container_1448220352443_0001_01_000001, NodeId: ip-172-31-20-42.ap-northeast-1.compute.internal:49391, NodeHttpAddress: ip-172-31-20-42.ap-northeast-1.compute.internal:39768, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.31.20.42:49391 }, ] for AM appattempt_1448220352443_0001_000001 2015-11-22 19:25:59,114 INFO [ApplicationMasterLauncher #0] amlauncher.AMLauncher (AMLauncher.java:createAMContainerLaunchContext(188)) - Command to launch container container_1448220352443_0001_01_000001 : {{JAVA_HOME}}/bin/java -Xmx512m org.apache.hadoop.yarn.applications.distributedshell.ApplicationMaster --container_memory 128 --container_vcores 1 --num_containers 2 --priority 0 1>/AppMaster.stdout 2>/AppMaster.stderr 2015-11-22 19:25:59,115 INFO [ApplicationMasterLauncher #0] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:createAndGetAMRMToken(195)) - Create AMRMToken for ApplicationAttempt: appattempt_1448220352443_0001_000001 2015-11-22 19:25:59,115 INFO [ApplicationMasterLauncher #0] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:createPassword(307)) - Creating password for appattempt_1448220352443_0001_000001 2015-11-22 19:25:59,118 INFO [Socket Reader #1 for port 49391] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220352443_0001_000001 (auth:SIMPLE) 2015-11-22 19:25:59,122 INFO [IPC Server handler 0 on 49391] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:startContainerInternal(889)) - Start request for container_1448220352443_0001_01_000001 by user ubuntu 2015-11-22 19:25:59,123 INFO [IPC Server handler 0 on 49391] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:startContainerInternal(930)) - Creating a new application reference for app application_1448220352443_0001 2015-11-22 19:25:59,123 INFO [IPC Server handler 0 on 49391] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu IP=172.31.20.42 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1448220352443_0001 CONTAINERID=container_1448220352443_0001_01_000001 2015-11-22 19:25:59,123 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(463)) - Application application_1448220352443_0001 transitioned from NEW to INITING 2015-11-22 19:25:59,125 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(303)) - Adding container_1448220352443_0001_01_000001 to application application_1448220352443_0001 2015-11-22 19:25:59,126 INFO [ApplicationMasterLauncher #0] amlauncher.AMLauncher (AMLauncher.java:launch(127)) - Done launching container Container: [ContainerId: container_1448220352443_0001_01_000001, NodeId: ip-172-31-20-42.ap-northeast-1.compute.internal:49391, NodeHttpAddress: ip-172-31-20-42.ap-northeast-1.compute.internal:39768, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.31.20.42:49391 }, ] for AM appattempt_1448220352443_0001_000001 2015-11-22 19:25:59,126 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(463)) - Application application_1448220352443_0001 transitioned from INITING to RUNNING 2015-11-22 19:25:59,126 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220352443_0001_000001 State change from ALLOCATED to LAUNCHED 2015-11-22 19:25:59,127 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220352443_0001_01_000001 transitioned from NEW to LOCALIZING 2015-11-22 19:25:59,127 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_INIT for appId application_1448220352443_0001 2015-11-22 19:25:59,127 INFO [AsyncDispatcher event handler] localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource file:/home/ubuntu/DistributedShell/application_1448220352443_0001/AppMaster.jar transitioned from INIT to DOWNLOADING 2015-11-22 19:25:59,127 INFO [AsyncDispatcher event handler] localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource file:/home/ubuntu/DistributedShell/application_1448220352443_0001/shellCommands transitioned from INIT to DOWNLOADING 2015-11-22 19:25:59,128 INFO [AsyncDispatcher event handler] localizer.ResourceLocalizationService (ResourceLocalizationService.java:handle(742)) - Created localizer for container_1448220352443_0001_01_000001 2015-11-22 19:25:59,132 INFO [LocalizerRunner for container_1448220352443_0001_01_000001] localizer.ResourceLocalizationService (ResourceLocalizationService.java:writeCredentials(1221)) - Writing credentials to the nmPrivate file /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/container_1448220352443_0001_01_000001.tokens. Credentials list: 2015-11-22 19:25:59,133 INFO [LocalizerRunner for container_1448220352443_0001_01_000001] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:createUserCacheDirs(632)) - Initializing user ubuntu 2015-11-22 19:25:59,141 INFO [LocalizerRunner for container_1448220352443_0001_01_000001] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:startLocalizer(126)) - Copying from /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/container_1448220352443_0001_01_000001.tokens to /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220352443_0001/container_1448220352443_0001_01_000001.tokens 2015-11-22 19:25:59,141 INFO [LocalizerRunner for container_1448220352443_0001_01_000001] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:startLocalizer(133)) - Localizer CWD set to /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220352443_0001 = file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220352443_0001 2015-11-22 19:25:59,175 INFO [IPC Server handler 2 on 40586] localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource file:/home/ubuntu/DistributedShell/application_1448220352443_0001/AppMaster.jar(->/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220352443_0001/filecache/10/AppMaster.jar) transitioned from DOWNLOADING to LOCALIZED 2015-11-22 19:25:59,189 INFO [IPC Server handler 3 on 40586] localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource file:/home/ubuntu/DistributedShell/application_1448220352443_0001/shellCommands(->/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220352443_0001/filecache/11/shellCommands) transitioned from DOWNLOADING to LOCALIZED 2015-11-22 19:25:59,189 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220352443_0001_01_000001 transitioned from LOCALIZING to LOCALIZED 2015-11-22 19:25:59,212 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220352443_0001_01_000001 transitioned from LOCALIZED to RUNNING 2015-11-22 19:25:59,230 INFO [ContainersLauncher #0] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:buildCommandExecutor(281)) - launchContainer: [nice, -n, 0, bash, /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220352443_0001/container_1448220352443_0001_01_000001/default_container_executor.sh] 2015-11-22 19:25:59,920 INFO [Thread-2202] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=N/A, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220358714, yarnAppState=ACCEPTED, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:37870/proxy/application_1448220352443_0001/, appUser=ubuntu 2015-11-22 19:26:00,108 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220352443_0001_01_000001 Container Transitioned from ACQUIRED to RUNNING 2015-11-22 19:26:00,921 INFO [Thread-2202] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=N/A, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220358714, yarnAppState=ACCEPTED, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:37870/proxy/application_1448220352443_0001/, appUser=ubuntu 2015-11-22 19:26:01,770 INFO [Socket Reader #1 for port 48123] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220352443_0001_000001 (auth:SIMPLE) 2015-11-22 19:26:01,777 INFO [IPC Server handler 1 on 48123] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:registerApplicationMaster(220)) - AM registration appattempt_1448220352443_0001_000001 2015-11-22 19:26:01,777 INFO [IPC Server handler 1 on 48123] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(133)) - USER=ubuntu IP=172.31.20.42 OPERATION=Register App Master TARGET=ApplicationMasterService RESULT=SUCCESS APPID=application_1448220352443_0001 APPATTEMPTID=appattempt_1448220352443_0001_000001 2015-11-22 19:26:01,778 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220352443_0001_000001 State change from LAUNCHED to RUNNING 2015-11-22 19:26:01,778 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220352443_0001 State change from ACCEPTED to RUNNING 2015-11-22 19:26:01,922 INFO [Thread-2202] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220358714, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:37870/proxy/application_1448220352443_0001/, appUser=ubuntu 2015-11-22 19:26:02,924 INFO [Thread-2202] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220358714, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:37870/proxy/application_1448220352443_0001/, appUser=ubuntu 2015-11-22 19:26:03,111 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220352443_0001_01_000002 Container Transitioned from NEW to ALLOCATED 2015-11-22 19:26:03,111 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220352443_0001 CONTAINERID=container_1448220352443_0001_01_000002 2015-11-22 19:26:03,111 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(154)) - Assigned container container_1448220352443_0001_01_000002 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:49391, which has 2 containers, used and available after allocation 2015-11-22 19:26:03,111 INFO [ResourceManager Event Processor] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(89)) - assignedContainer application attempt=appattempt_1448220352443_0001_000001 container=container_1448220352443_0001_01_000002 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@23d8cfaf clusterResource= type=OFF_SWITCH 2015-11-22 19:26:03,111 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(589)) - Re-sorting assigned queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.15625, absoluteUsedCapacity=0.15625, numApps=1, numContainers=2 2015-11-22 19:26:03,112 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(461)) - assignedContainer queue=root usedCapacity=0.15625 absoluteUsedCapacity=0.15625 used= cluster= 2015-11-22 19:26:03,832 INFO [IPC Server handler 3 on 48123] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:createAndGetNMToken(200)) - Sending NMToken for nodeId : ip-172-31-20-42.ap-northeast-1.compute.internal:49391 for container : container_1448220352443_0001_01_000002 2015-11-22 19:26:03,833 INFO [IPC Server handler 3 on 48123] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220352443_0001_01_000002 Container Transitioned from ALLOCATED to ACQUIRED 2015-11-22 19:26:03,911 INFO [Socket Reader #1 for port 49391] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220352443_0001_000001 (auth:SIMPLE) 2015-11-22 19:26:03,917 INFO [IPC Server handler 1 on 49391] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:startContainerInternal(889)) - Start request for container_1448220352443_0001_01_000002 by user ubuntu 2015-11-22 19:26:03,917 INFO [IPC Server handler 1 on 49391] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu IP=172.31.20.42 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1448220352443_0001 CONTAINERID=container_1448220352443_0001_01_000002 2015-11-22 19:26:03,928 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(303)) - Adding container_1448220352443_0001_01_000002 to application application_1448220352443_0001 2015-11-22 19:26:03,928 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220352443_0001_01_000002 transitioned from NEW to LOCALIZED 2015-11-22 19:26:03,928 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_INIT for appId application_1448220352443_0001 2015-11-22 19:26:03,935 INFO [Thread-2202] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220358714, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:37870/proxy/application_1448220352443_0001/, appUser=ubuntu 2015-11-22 19:26:03,977 INFO [Socket Reader #1 for port 49391] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220352443_0001_000001 (auth:SIMPLE) 2015-11-22 19:26:03,981 INFO [IPC Server handler 0 on 49391] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:getContainerStatusInternal(1221)) - Getting container-status for container_1448220352443_0001_01_000002 2015-11-22 19:26:03,982 INFO [IPC Server handler 0 on 49391] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:getContainerStatusInternal(1235)) - Returning ContainerStatus: [ContainerId: container_1448220352443_0001_01_000002, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ] 2015-11-22 19:26:03,989 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220352443_0001_01_000002 transitioned from LOCALIZED to RUNNING 2015-11-22 19:26:04,023 INFO [ContainersLauncher #1] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:buildCommandExecutor(281)) - launchContainer: [nice, -n, 0, bash, /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220352443_0001/container_1448220352443_0001_01_000002/default_container_executor.sh] 2015-11-22 19:26:04,033 INFO [ContainersLauncher #1] launcher.ContainerLaunch (ContainerLaunch.java:call(356)) - Container container_1448220352443_0001_01_000002 succeeded 2015-11-22 19:26:04,033 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220352443_0001_01_000002 transitioned from RUNNING to EXITED_WITH_SUCCESS 2015-11-22 19:26:04,033 INFO [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(380)) - Cleaning up container container_1448220352443_0001_01_000002 2015-11-22 19:26:04,046 INFO [DeletionService #0] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(480)) - Deleting absolute path : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220352443_0001/container_1448220352443_0001_01_000002 2015-11-22 19:26:04,047 INFO [AsyncDispatcher event handler] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu OPERATION=Container Finished - Succeeded TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1448220352443_0001 CONTAINERID=container_1448220352443_0001_01_000002 2015-11-22 19:26:04,047 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220352443_0001_01_000002 transitioned from EXITED_WITH_SUCCESS to DONE 2015-11-22 19:26:04,047 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(346)) - Removing container_1448220352443_0001_01_000002 from application application_1448220352443_0001 2015-11-22 19:26:04,048 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_STOP for appId application_1448220352443_0001 2015-11-22 19:26:04,110 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220352443_0001_01_000002 Container Transitioned from ACQUIRED to COMPLETED 2015-11-22 19:26:04,110 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220352443_0001 CONTAINERID=container_1448220352443_0001_01_000002 2015-11-22 19:26:04,110 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:releaseContainer(248)) - Released container container_1448220352443_0001_01_000002 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:49391, which currently has 1 containers, used and available, release resources=true 2015-11-22 19:26:04,111 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220352443_0001_01_000003 Container Transitioned from NEW to ALLOCATED 2015-11-22 19:26:04,111 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220352443_0001 CONTAINERID=container_1448220352443_0001_01_000003 2015-11-22 19:26:04,111 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(154)) - Assigned container container_1448220352443_0001_01_000003 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:49391, which has 2 containers, used and available after allocation 2015-11-22 19:26:04,111 INFO [ResourceManager Event Processor] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(89)) - assignedContainer application attempt=appattempt_1448220352443_0001_000001 container=container_1448220352443_0001_01_000003 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@23d8cfaf clusterResource= type=OFF_SWITCH 2015-11-22 19:26:04,112 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(589)) - Re-sorting assigned queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.15625, absoluteUsedCapacity=0.15625, numApps=1, numContainers=2 2015-11-22 19:26:04,112 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(461)) - assignedContainer queue=root usedCapacity=0.15625 absoluteUsedCapacity=0.15625 used= cluster= 2015-11-22 19:26:04,842 INFO [IPC Server handler 5 on 48123] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220352443_0001_01_000003 Container Transitioned from ALLOCATED to ACQUIRED 2015-11-22 19:26:04,893 INFO [Socket Reader #1 for port 49391] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220352443_0001_000001 (auth:SIMPLE) 2015-11-22 19:26:04,897 INFO [IPC Server handler 1 on 49391] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:startContainerInternal(889)) - Start request for container_1448220352443_0001_01_000003 by user ubuntu 2015-11-22 19:26:04,897 INFO [IPC Server handler 1 on 49391] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu IP=172.31.20.42 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1448220352443_0001 CONTAINERID=container_1448220352443_0001_01_000003 2015-11-22 19:26:04,897 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(303)) - Adding container_1448220352443_0001_01_000003 to application application_1448220352443_0001 2015-11-22 19:26:04,898 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220352443_0001_01_000003 transitioned from NEW to LOCALIZED 2015-11-22 19:26:04,898 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_INIT for appId application_1448220352443_0001 2015-11-22 19:26:04,912 INFO [Socket Reader #1 for port 49391] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220352443_0001_000001 (auth:SIMPLE) 2015-11-22 19:26:04,928 INFO [IPC Server handler 2 on 49391] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:getContainerStatusInternal(1221)) - Getting container-status for container_1448220352443_0001_01_000003 2015-11-22 19:26:04,928 INFO [IPC Server handler 2 on 49391] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:getContainerStatusInternal(1235)) - Returning ContainerStatus: [ContainerId: container_1448220352443_0001_01_000003, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ] 2015-11-22 19:26:04,937 INFO [Thread-2202] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220358714, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:37870/proxy/application_1448220352443_0001/, appUser=ubuntu 2015-11-22 19:26:04,942 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220352443_0001_01_000003 transitioned from LOCALIZED to RUNNING 2015-11-22 19:26:04,980 INFO [ContainersLauncher #1] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:buildCommandExecutor(281)) - launchContainer: [nice, -n, 0, bash, /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220352443_0001/container_1448220352443_0001_01_000003/default_container_executor.sh] 2015-11-22 19:26:04,992 INFO [ContainersLauncher #1] launcher.ContainerLaunch (ContainerLaunch.java:call(356)) - Container container_1448220352443_0001_01_000003 succeeded 2015-11-22 19:26:04,993 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220352443_0001_01_000003 transitioned from RUNNING to EXITED_WITH_SUCCESS 2015-11-22 19:26:04,993 INFO [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(380)) - Cleaning up container container_1448220352443_0001_01_000003 2015-11-22 19:26:05,013 INFO [DeletionService #0] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(480)) - Deleting absolute path : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220352443_0001/container_1448220352443_0001_01_000003 2015-11-22 19:26:05,014 INFO [AsyncDispatcher event handler] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu OPERATION=Container Finished - Succeeded TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1448220352443_0001 CONTAINERID=container_1448220352443_0001_01_000003 2015-11-22 19:26:05,014 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220352443_0001_01_000003 transitioned from EXITED_WITH_SUCCESS to DONE 2015-11-22 19:26:05,014 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(346)) - Removing container_1448220352443_0001_01_000003 from application application_1448220352443_0001 2015-11-22 19:26:05,014 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_STOP for appId application_1448220352443_0001 2015-11-22 19:26:05,111 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220352443_0001_01_000003 Container Transitioned from ACQUIRED to COMPLETED 2015-11-22 19:26:05,111 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220352443_0001 CONTAINERID=container_1448220352443_0001_01_000003 2015-11-22 19:26:05,111 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:releaseContainer(248)) - Released container container_1448220352443_0001_01_000003 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:49391, which currently has 1 containers, used and available, release resources=true 2015-11-22 19:26:05,938 INFO [Thread-2202] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220358714, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:37870/proxy/application_1448220352443_0001/, appUser=ubuntu 2015-11-22 19:26:06,049 INFO [Socket Reader #1 for port 49391] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220352443_0001_000001 (auth:SIMPLE) 2015-11-22 19:26:06,056 INFO [IPC Server handler 1 on 49391] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:stopContainerInternal(1163)) - Stopping container with container Id: container_1448220352443_0001_01_000003 2015-11-22 19:26:06,056 INFO [IPC Server handler 1 on 49391] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu IP=172.31.20.42 OPERATION=Stop Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1448220352443_0001 CONTAINERID=container_1448220352443_0001_01_000003 2015-11-22 19:26:06,057 INFO [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:removeOrTrackCompletedContainersFromContext(605)) - Removed completed containers from NM context: [container_1448220352443_0001_01_000002] 2015-11-22 19:26:06,063 INFO [Socket Reader #1 for port 49391] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220352443_0001_000001 (auth:SIMPLE) 2015-11-22 19:26:06,067 INFO [IPC Server handler 2 on 49391] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:stopContainerInternal(1163)) - Stopping container with container Id: container_1448220352443_0001_01_000002 2015-11-22 19:26:06,074 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:rememberTargetTransitionsAndStoreState(1258)) - Updating application attempt appattempt_1448220352443_0001_000001 with final state: FINISHING, and exit status: -1000 2015-11-22 19:26:06,074 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220352443_0001_000001 State change from RUNNING to FINAL_SAVING 2015-11-22 19:26:06,074 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:rememberTargetTransitionsAndStoreState(1098)) - Updating application application_1448220352443_0001 with final state: FINISHING 2015-11-22 19:26:06,074 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220352443_0001 State change from RUNNING to FINAL_SAVING 2015-11-22 19:26:06,074 INFO [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:transition(237)) - Updating info for app: application_1448220352443_0001 2015-11-22 19:26:06,074 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220352443_0001_000001 State change from FINAL_SAVING to FINISHING 2015-11-22 19:26:06,074 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220352443_0001 State change from FINAL_SAVING to FINISHING 2015-11-22 19:26:06,178 INFO [IPC Server handler 11 on 48123] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:finishApplicationMaster(300)) - application_1448220352443_0001 unregistered successfully. 2015-11-22 19:26:06,194 INFO [ContainersLauncher #0] launcher.ContainerLaunch (ContainerLaunch.java:call(356)) - Container container_1448220352443_0001_01_000001 succeeded 2015-11-22 19:26:06,194 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220352443_0001_01_000001 transitioned from RUNNING to EXITED_WITH_SUCCESS 2015-11-22 19:26:06,194 INFO [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(380)) - Cleaning up container container_1448220352443_0001_01_000001 2015-11-22 19:26:06,210 INFO [DeletionService #1] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(480)) - Deleting absolute path : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220352443_0001/container_1448220352443_0001_01_000001 2015-11-22 19:26:06,210 INFO [AsyncDispatcher event handler] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu OPERATION=Container Finished - Succeeded TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1448220352443_0001 CONTAINERID=container_1448220352443_0001_01_000001 2015-11-22 19:26:06,211 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220352443_0001_01_000001 transitioned from EXITED_WITH_SUCCESS to DONE 2015-11-22 19:26:06,211 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(346)) - Removing container_1448220352443_0001_01_000001 from application application_1448220352443_0001 2015-11-22 19:26:06,211 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_STOP for appId application_1448220352443_0001 2015-11-22 19:26:06,940 INFO [Thread-2202] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220358714, yarnAppState=FINISHED, distributedFinalState=SUCCEEDED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:37870/proxy/application_1448220352443_0001/, appUser=ubuntu 2015-11-22 19:26:06,940 INFO [Thread-2202] distributedshell.Client (Client.java:monitorApplication(759)) - Application has completed successfully. Breaking monitoring loop 2015-11-22 19:26:06,940 INFO [Thread-2201] distributedshell.TestDistributedShell (TestDistributedShell.java:testDSShell(239)) - Client run completed. Result=true 2015-11-22 19:26:06,943 ERROR [Thread[Thread-2197,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(670)) - ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted 2015-11-22 19:26:06,945 INFO [main] mortbay.log (Slf4jLog.java:info(67)) - Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:0 2015-11-22 19:26:07,047 INFO [main] applicationhistoryservice.ApplicationHistoryManagerImpl (ApplicationHistoryManagerImpl.java:serviceStop(78)) - Stopping ApplicationHistory 2015-11-22 19:26:07,047 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 48370 2015-11-22 19:26:07,049 INFO [IPC Server listener on 48370] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 48370 2015-11-22 19:26:07,049 ERROR [Thread[Thread-2179,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(670)) - ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted 2015-11-22 19:26:07,050 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:26:07,050 INFO [main] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:unRegisterNM(268)) - Successfully Unregistered the Node ip-172-31-20-42.ap-northeast-1.compute.internal:49391 with ResourceManager. 2015-11-22 19:26:07,057 ERROR [Thread[Thread-2172,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(670)) - ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted 2015-11-22 19:26:07,059 INFO [main] mortbay.log (Slf4jLog.java:info(67)) - Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:0 2015-11-22 19:26:07,161 INFO [main] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:cleanUpApplicationsOnNMShutDown(556)) - Applications still running : [application_1448220352443_0001] 2015-11-22 19:26:07,161 INFO [main] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:cleanUpApplicationsOnNMShutDown(572)) - Waiting for Applications to be Finished 2015-11-22 19:26:07,161 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(463)) - Application application_1448220352443_0001 transitioned from RUNNING to APPLICATION_RESOURCES_CLEANINGUP 2015-11-22 19:26:07,162 INFO [DeletionService #2] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(480)) - Deleting absolute path : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220352443_0001 2015-11-22 19:26:07,162 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event APPLICATION_STOP for appId application_1448220352443_0001 2015-11-22 19:26:07,162 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(463)) - Application application_1448220352443_0001 transitioned from APPLICATION_RESOURCES_CLEANINGUP to FINISHED 2015-11-22 19:26:07,162 INFO [AsyncDispatcher event handler] loghandler.NonAggregatingLogHandler (NonAggregatingLogHandler.java:handle(167)) - Scheduling Log Deletion for application: application_1448220352443_0001, with delay of 10800 seconds 2015-11-22 19:26:11,162 INFO [main] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:cleanUpApplicationsOnNMShutDown(589)) - Done waiting for Applications to be Finished. Still alive: [application_1448220352443_0001] 2015-11-22 19:26:11,163 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 49391 2015-11-22 19:26:11,166 INFO [IPC Server listener on 49391] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 49391 2015-11-22 19:26:11,166 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:26:21,167 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 40586 2015-11-22 19:26:21,168 INFO [Public Localizer] localizer.ResourceLocalizationService (ResourceLocalizationService.java:run(908)) - Public cache exiting 2015-11-22 19:26:21,168 WARN [Node Resource Monitor] nodemanager.NodeResourceMonitorImpl (NodeResourceMonitorImpl.java:run(154)) - org.apache.hadoop.yarn.server.nodemanager.NodeResourceMonitorImpl is interrupted. Exiting. 2015-11-22 19:26:21,168 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:26:21,169 INFO [IPC Server listener on 40586] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 40586 2015-11-22 19:26:21,175 ERROR [Thread[Thread-2122,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(670)) - ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted 2015-11-22 19:26:21,177 INFO [main] mortbay.log (Slf4jLog.java:info(67)) - Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:0 2015-11-22 19:26:21,280 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 43161 2015-11-22 19:26:21,283 INFO [IPC Server listener on 43161] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 43161 2015-11-22 19:26:21,284 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 52421 2015-11-22 19:26:21,283 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:26:21,285 INFO [IPC Server listener on 52421] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 52421 2015-11-22 19:26:21,285 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:26:21,286 INFO [main] resourcemanager.ResourceManager (ResourceManager.java:transitionToStandby(1051)) - Transitioning to standby state 2015-11-22 19:26:21,286 WARN [ApplicationMaster Launcher] amlauncher.ApplicationMasterLauncher (ApplicationMasterLauncher.java:run(122)) - org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher$LauncherThread interrupted. Returning. 2015-11-22 19:26:21,287 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 48123 2015-11-22 19:26:21,289 INFO [IPC Server listener on 48123] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 48123 2015-11-22 19:26:21,289 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:26:21,289 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 42955 2015-11-22 19:26:21,292 INFO [IPC Server listener on 42955] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 42955 2015-11-22 19:26:21,292 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:26:21,292 ERROR [ResourceManager Event Processor] resourcemanager.ResourceManager (ResourceManager.java:run(691)) - Returning, interrupted : java.lang.InterruptedException 2015-11-22 19:26:21,292 INFO [Ping Checker] util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(135)) - NMLivelinessMonitor thread interrupted 2015-11-22 19:26:21,294 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:serviceStop(141)) - AsyncDispatcher is draining to stop, igonring any new events. 2015-11-22 19:26:21,296 ERROR [Thread[Thread-1953,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(670)) - ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted 2015-11-22 19:26:21,296 INFO [Ping Checker] util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(135)) - AMLivelinessMonitor thread interrupted 2015-11-22 19:26:21,296 INFO [Ping Checker] util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(135)) - org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.ContainerAllocationExpirer thread interrupted 2015-11-22 19:26:21,297 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stop(211)) - Stopping ApplicationHistoryServer metrics system... 2015-11-22 19:26:21,297 INFO [Ping Checker] util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(135)) - AMLivelinessMonitor thread interrupted 2015-11-22 19:26:21,302 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stop(217)) - ApplicationHistoryServer metrics system stopped. 2015-11-22 19:26:21,302 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:shutdown(608)) - ApplicationHistoryServer metrics system shutdown complete. 2015-11-22 19:26:21,302 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:serviceStop(141)) - AsyncDispatcher is draining to stop, igonring any new events. 2015-11-22 19:26:21,303 INFO [main] resourcemanager.ResourceManager (ResourceManager.java:transitionToStandby(1058)) - Transitioned to standby state 2015-11-22 19:26:21,313 INFO [main] distributedshell.TestDistributedShell (TestDistributedShell.java:setupInternal(77)) - Starting up YARN cluster 2015-11-22 19:26:21,338 INFO [main] net.ServerSocketUtil (ServerSocketUtil.java:getPort(52)) - Using port 10021 2015-11-22 19:26:21,359 INFO [main] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2363)) - found resource core-site.xml at file:/home/ubuntu/hadoop-dev/hadoop-common-project/hadoop-common/target/test-classes/core-site.xml 2015-11-22 19:26:21,376 WARN [main] conf.Configuration (Configuration.java:loadProperty(2687)) - core-site.xml:an attempt to override final parameter: hadoop.tmp.dir; Ignoring. 2015-11-22 19:26:21,376 INFO [main] security.Groups (Groups.java:refresh(256)) - clearing userToGroupsMap cache 2015-11-22 19:26:21,378 INFO [main] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2363)) - found resource yarn-site.xml at file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/test-classes/yarn-site.xml 2015-11-22 19:26:21,403 WARN [main] conf.Configuration (Configuration.java:loadProperty(2687)) - yarn-site.xml:an attempt to override final parameter: hadoop.tmp.dir; Ignoring. 2015-11-22 19:26:21,404 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.RMFatalEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMFatalEventDispatcher 2015-11-22 19:26:21,405 INFO [main] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:(75)) - NMTokenKeyRollingInterval: 86400000ms and NMTokenKeyActivationDelay: 900000ms 2015-11-22 19:26:21,405 INFO [main] security.RMContainerTokenSecretManager (RMContainerTokenSecretManager.java:(78)) - ContainerTokenKeyRollingInterval: 86400000ms and ContainerTokenKeyActivationDelay: 900000ms 2015-11-22 19:26:21,405 INFO [main] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:(94)) - AMRMTokenKeyRollingInterval: 86400000ms and AMRMTokenKeyActivationDelay: 900000 ms 2015-11-22 19:26:21,406 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStoreEventType for class org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$ForwardingEventHandler 2015-11-22 19:26:21,406 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.NodesListManagerEventType for class org.apache.hadoop.yarn.server.resourcemanager.NodesListManager 2015-11-22 19:26:21,406 INFO [main] resourcemanager.ResourceManager (ResourceManager.java:createScheduler(295)) - Using Scheduler: org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler 2015-11-22 19:26:21,406 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.SchedulerEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher 2015-11-22 19:26:21,406 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationEventDispatcher 2015-11-22 19:26:21,406 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationAttemptEventDispatcher 2015-11-22 19:26:21,407 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$NodeEventDispatcher 2015-11-22 19:26:21,408 INFO [main] impl.MetricsConfig (MetricsConfig.java:loadFirst(115)) - loaded properties from hadoop-metrics2.properties 2015-11-22 19:26:21,410 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:startTimer(374)) - Scheduled snapshot period at 10 second(s). 2015-11-22 19:26:21,410 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:start(192)) - ResourceManager metrics system started 2015-11-22 19:26:21,411 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.RMAppManagerEventType for class org.apache.hadoop.yarn.server.resourcemanager.RMAppManager 2015-11-22 19:26:21,411 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncherEventType for class org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher 2015-11-22 19:26:21,411 WARN [main] util.MBeans (MBeans.java:register(73)) - Failed to register MBean "Hadoop:service=ResourceManager,name=RMNMInfo": Instance already exists. 2015-11-22 19:26:21,412 INFO [main] resourcemanager.RMNMInfo (RMNMInfo.java:(63)) - Registered RMNMInfo MBean 2015-11-22 19:26:21,412 INFO [main] util.HostsFileReader (HostsFileReader.java:refresh(132)) - Refreshing hosts (include/exclude) list 2015-11-22 19:26:21,413 INFO [main] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2363)) - found resource capacity-scheduler.xml at jar:file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/target/hadoop-yarn-server-tests-2.8.0-SNAPSHOT-tests.jar!/capacity-scheduler.xml 2015-11-22 19:26:21,425 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(647)) - max alloc mb per queue for root is undefined 2015-11-22 19:26:21,425 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(651)) - max alloc vcore per queue for root is undefined 2015-11-22 19:26:21,426 INFO [main] capacity.ParentQueue (ParentQueue.java:setupQueueConfigs(125)) - root, capacity=1.0, asboluteCapacity=1.0, maxCapacity=1.0, asboluteMaxCapacity=1.0, state=RUNNING, acls=ADMINISTER_QUEUE:*SUBMIT_APP:*, labels=*, , reservationsContinueLooking=true 2015-11-22 19:26:21,426 INFO [main] capacity.ParentQueue (ParentQueue.java:(104)) - Initialized parent-queue root name=root, fullname=root 2015-11-22 19:26:21,427 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(647)) - max alloc mb per queue for root.default is undefined 2015-11-22 19:26:21,427 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(651)) - max alloc vcore per queue for root.default is undefined 2015-11-22 19:26:21,427 INFO [main] capacity.LeafQueue (LeafQueue.java:setupQueueConfigs(217)) - Initializing default capacity = 1.0 [= (float) configuredCapacity / 100 ] asboluteCapacity = 1.0 [= parentAbsoluteCapacity * capacity ] maxCapacity = 1.0 [= configuredMaxCapacity ] absoluteMaxCapacity = 1.0 [= 1.0 maximumCapacity undefined, (parentAbsoluteMaxCapacity * maximumCapacity) / 100 otherwise ] userLimit = 100 [= configuredUserLimit ] userLimitFactor = 1.0 [= configuredUserLimitFactor ] maxApplications = 10000 [= configuredMaximumSystemApplicationsPerQueue or (int)(configuredMaximumSystemApplications * absoluteCapacity)] maxApplicationsPerUser = 10000 [= (int)(maxApplications * (userLimit / 100.0f) * userLimitFactor) ] usedCapacity = 0.0 [= usedResourcesMemory / (clusterResourceMemory * absoluteCapacity)] absoluteUsedCapacity = 0.0 [= usedResourcesMemory / clusterResourceMemory] maxAMResourcePerQueuePercent = 0.1 [= configuredMaximumAMResourcePercent ] minimumAllocationFactor = 0.984375 [= (float)(maximumAllocationMemory - minimumAllocationMemory) / maximumAllocationMemory ] maximumAllocation = [= configuredMaxAllocation ] numContainers = 0 [= currentNumContainers ] state = RUNNING [= configuredState ] acls = ADMINISTER_QUEUE:*SUBMIT_APP:* [= configuredAcls ] nodeLocalityDelay = -1 labels=*, nodeLocalityDelay = -1 reservationsContinueLooking = true preemptionDisabled = true defaultAppPriorityPerQueue = 0 2015-11-22 19:26:21,427 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:parseQueue(657)) - Initialized queue: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=0, numContainers=0 2015-11-22 19:26:21,428 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:parseQueue(657)) - Initialized queue: root: numChildQueue= 1, capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.0, numApps=0, numContainers=0 2015-11-22 19:26:21,428 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:initializeQueues(501)) - Initialized root queue root: numChildQueue= 1, capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.0, numApps=0, numContainers=0 2015-11-22 19:26:21,428 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:getUserGroupMappingPlacementRule(449)) - Initialized queue mappings, override: false 2015-11-22 19:26:21,428 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:initScheduler(314)) - Initialized CapacityScheduler with calculator=class org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator, minimumAllocation=<>, maximumAllocation=<>, asynchronousScheduling=false, asyncScheduleInterval=5ms 2015-11-22 19:26:21,429 INFO [main] metrics.SystemMetricsPublisher (SystemMetricsPublisher.java:serviceInit(95)) - YARN system metrics publishing service is not enabled 2015-11-22 19:26:21,429 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEventType for class org.apache.hadoop.yarn.server.MiniYARNCluster$1 2015-11-22 19:26:21,429 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:prepareDirs(558)) - Created localDir in /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0 2015-11-22 19:26:21,429 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:prepareDirs(558)) - Created logDir in /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-logDir-nm-0_0 2015-11-22 19:26:21,429 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceInit(540)) - Starting NM: 0 2015-11-22 19:26:21,439 INFO [main] nodemanager.NodeManager (NodeManager.java:getNodeHealthScriptRunner(255)) - Node Manager health check script is not available or doesn't have execute permission, so not starting the node health script runner. 2015-11-22 19:26:21,439 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ContainerEventDispatcher 2015-11-22 19:26:21,439 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ApplicationEventDispatcher 2015-11-22 19:26:21,439 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizationEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService 2015-11-22 19:26:21,439 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServicesEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices 2015-11-22 19:26:21,440 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl 2015-11-22 19:26:21,440 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncherEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncher 2015-11-22 19:26:21,440 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.ContainerManagerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl 2015-11-22 19:26:21,440 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.NodeManagerEventType for class org.apache.hadoop.yarn.server.MiniYARNCluster$ShortCircuitedNodeManager 2015-11-22 19:26:21,440 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:init(159)) - NodeManager metrics system started (again) 2015-11-22 19:26:21,441 INFO [main] nodemanager.NodeResourceMonitorImpl (NodeResourceMonitorImpl.java:serviceInit(76)) - Using ResourceCalculatorPlugin : org.apache.hadoop.yarn.util.ResourceCalculatorPlugin@fa36558 2015-11-22 19:26:21,441 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.event.LogHandlerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.NonAggregatingLogHandler 2015-11-22 19:26:21,441 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.sharedcache.SharedCacheUploadEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.sharedcache.SharedCacheUploadService 2015-11-22 19:26:21,441 INFO [main] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:serviceInit(261)) - AMRMProxyService is disabled 2015-11-22 19:26:21,441 INFO [main] localizer.ResourceLocalizationService (ResourceLocalizationService.java:validateConf(224)) - per directory file limit = 8192 2015-11-22 19:26:21,452 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerTracker 2015-11-22 19:26:21,453 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(116)) - Using ResourceCalculatorPlugin : org.apache.hadoop.yarn.util.ResourceCalculatorPlugin@672872e1 2015-11-22 19:26:21,453 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(121)) - Using ResourceCalculatorProcessTree : null 2015-11-22 19:26:21,453 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(158)) - Physical memory check enabled: false 2015-11-22 19:26:21,453 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(159)) - Virtual memory check enabled: false 2015-11-22 19:26:21,453 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:isEnabled(206)) - Neither virutal-memory nor physical-memory monitoring is needed. Not running the monitor-thread 2015-11-22 19:26:21,453 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(162)) - ContainersMonitor enabled: false 2015-11-22 19:26:21,453 INFO [main] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:serviceInit(175)) - Nodemanager resources: memory set to 4096MB. 2015-11-22 19:26:21,453 INFO [main] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:serviceInit(176)) - Nodemanager resources: vcores set to 8. 2015-11-22 19:26:21,453 INFO [main] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:serviceInit(208)) - Initialized nodemanager with : physical-memory=4096 virtual-memory=8602 virtual-cores=8 2015-11-22 19:26:21,454 WARN [main] applicationhistoryservice.ApplicationHistoryServer (ApplicationHistoryServer.java:createApplicationHistoryManager(218)) - The filesystem based application history store is deprecated. 2015-11-22 19:26:21,454 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:init(159)) - ApplicationHistoryServer metrics system started (again) 2015-11-22 19:26:21,454 INFO [main] applicationhistoryservice.ApplicationHistoryManagerImpl (ApplicationHistoryManagerImpl.java:serviceInit(61)) - ApplicationHistory Init 2015-11-22 19:26:21,455 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:startResourceManager(326)) - Waiting for RM to start... 2015-11-22 19:26:21,455 INFO [RM-0] resourcemanager.ResourceManager (ResourceManager.java:transitionToActive(1024)) - Transitioning to active state 2015-11-22 19:26:21,456 INFO [RM-0] recovery.RMStateStore (RMStateStore.java:transition(482)) - Updating AMRMToken 2015-11-22 19:26:21,456 INFO [RM-0] security.RMContainerTokenSecretManager (RMContainerTokenSecretManager.java:rollMasterKey(106)) - Rolling master-key for container-tokens 2015-11-22 19:26:21,456 INFO [RM-0] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:rollMasterKey(95)) - Rolling master-key for nm-tokens 2015-11-22 19:26:21,456 INFO [RM-0] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:26:21,456 INFO [RM-0] security.RMDelegationTokenSecretManager (RMDelegationTokenSecretManager.java:storeNewMasterKey(87)) - storing master key with keyID 1 2015-11-22 19:26:21,456 INFO [RM-0] recovery.RMStateStore (RMStateStore.java:transition(433)) - Storing RMDTMasterKey. 2015-11-22 19:26:21,456 INFO [Thread[Thread-2328,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2015-11-22 19:26:21,457 INFO [Thread[Thread-2328,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:26:21,457 INFO [Thread[Thread-2328,5,main]] security.RMDelegationTokenSecretManager (RMDelegationTokenSecretManager.java:storeNewMasterKey(87)) - storing master key with keyID 2 2015-11-22 19:26:21,457 INFO [Thread[Thread-2328,5,main]] recovery.RMStateStore (RMStateStore.java:transition(433)) - Storing RMDTMasterKey. 2015-11-22 19:26:21,458 INFO [RM-0] nodelabels.CommonNodeLabelsManager (CommonNodeLabelsManager.java:addToCluserNodeLabels(309)) - Add labels: [] 2015-11-22 19:26:21,458 INFO [RM-0] nodelabels.CommonNodeLabelsManager (CommonNodeLabelsManager.java:internalUpdateLabelsOnNodes(638)) - REPLACE labels on nodes: 2015-11-22 19:26:21,458 INFO [RM-0] nodelabels.CommonNodeLabelsManager (CommonNodeLabelsManager.java:internalUpdateLabelsOnNodes(640)) - NM=ip-172-31-20-42.ap-northeast-1.compute.internal:58694, labels=[x] 2015-11-22 19:26:21,459 INFO [RM-0] nodelabels.FileSystemNodeLabelsStore (FileSystemNodeLabelsStore.java:recover(279)) - Finished write mirror at:file:/tmp/hadoop-yarn-ubuntu/node-labels/nodelabel.mirror 2015-11-22 19:26:21,459 INFO [RM-0] nodelabels.FileSystemNodeLabelsStore (FileSystemNodeLabelsStore.java:recover(280)) - Finished create editlog file at:file:/tmp/hadoop-yarn-ubuntu/node-labels/nodelabel.editlog 2015-11-22 19:26:21,459 INFO [RM-0] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.nodelabels.event.NodeLabelsStoreEventType for class org.apache.hadoop.yarn.nodelabels.CommonNodeLabelsManager$ForwardingEventHandler 2015-11-22 19:26:21,460 INFO [RM-0] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:26:21,461 INFO [Socket Reader #1 for port 37685] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 37685 2015-11-22 19:26:21,462 INFO [RM-0] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.server.api.ResourceTrackerPB to the server 2015-11-22 19:26:21,462 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:26:21,462 INFO [IPC Server listener on 37685] ipc.Server (Server.java:run(794)) - IPC Server listener on 37685: starting 2015-11-22 19:26:21,469 INFO [RM-0] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:26:21,469 INFO [Socket Reader #1 for port 47916] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 47916 2015-11-22 19:26:21,470 INFO [RM-0] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB to the server 2015-11-22 19:26:21,471 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:26:21,472 INFO [IPC Server listener on 47916] ipc.Server (Server.java:run(794)) - IPC Server listener on 47916: starting 2015-11-22 19:26:21,479 INFO [RM-0] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:26:21,480 INFO [Socket Reader #1 for port 39999] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 39999 2015-11-22 19:26:21,481 INFO [RM-0] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.api.ApplicationClientProtocolPB to the server 2015-11-22 19:26:21,482 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:26:21,482 INFO [IPC Server listener on 39999] ipc.Server (Server.java:run(794)) - IPC Server listener on 39999: starting 2015-11-22 19:26:21,487 INFO [RM-0] resourcemanager.ResourceManager (ResourceManager.java:transitionToActive(1040)) - Transitioned to active state 2015-11-22 19:26:21,488 INFO [RM-0] server.AuthenticationFilter (AuthenticationFilter.java:constructSecretProvider(304)) - Unable to initialize FileSignerSecretProvider, falling back to use random secrets. 2015-11-22 19:26:21,488 INFO [RM-0] http.HttpRequestLog (HttpRequestLog.java:getRequestLog(80)) - Http request log for http.requests.resourcemanager is not defined 2015-11-22 19:26:21,488 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter) 2015-11-22 19:26:21,489 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context cluster 2015-11-22 19:26:21,489 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context static 2015-11-22 19:26:21,489 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context logs 2015-11-22 19:26:21,489 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context cluster 2015-11-22 19:26:21,489 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2015-11-22 19:26:21,489 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2015-11-22 19:26:21,490 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'Timeline Authentication Filter' (class=org.apache.hadoop.yarn.server.timeline.security.TimelineAuthenticationFilter) 2015-11-22 19:26:21,490 INFO [RM-0] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /cluster/* 2015-11-22 19:26:21,490 INFO [RM-0] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /ws/* 2015-11-22 19:26:21,496 INFO [RM-0] webapp.WebApps (WebApps.java:build(288)) - Registered webapp guice modules 2015-11-22 19:26:21,497 INFO [RM-0] http.HttpServer2 (HttpServer2.java:openListeners(917)) - Jetty bound to port 50090 2015-11-22 19:26:21,497 INFO [RM-0] mortbay.log (Slf4jLog.java:info(67)) - jetty-6.1.26 2015-11-22 19:26:21,512 INFO [RM-0] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:26:21,512 INFO [Thread[Thread-2497,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2015-11-22 19:26:21,513 INFO [Thread[Thread-2497,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:26:21,513 INFO [RM-0] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:26:21,513 INFO [Thread[Thread-2499,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) Nov 22, 2015 7:26:21 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.resourcemanager.webapp.JAXBContextResolver as a provider class Nov 22, 2015 7:26:21 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices as a root resource class Nov 22, 2015 7:26:21 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a provider class Nov 22, 2015 7:26:21 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate INFO: Initiating Jersey application, version 'Jersey: 1.9 09/02/2011 11:17 AM' Nov 22, 2015 7:26:21 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.resourcemanager.webapp.JAXBContextResolver to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:26:21 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.webapp.GenericExceptionHandler to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:26:21 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices to GuiceManagedComponentProvider with the scope "Singleton" 2015-11-22 19:26:21,659 INFO [RM-0] mortbay.log (Slf4jLog.java:info(67)) - Started HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:50090 2015-11-22 19:26:21,660 INFO [RM-0] webapp.WebApps (WebApps.java:start(307)) - Web app cluster started at 50090 2015-11-22 19:26:21,661 INFO [RM-0] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:26:21,662 INFO [Socket Reader #1 for port 60879] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 60879 2015-11-22 19:26:21,662 INFO [RM-0] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.server.api.ResourceManagerAdministrationProtocolPB to the server 2015-11-22 19:26:21,663 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:26:21,663 INFO [IPC Server listener on 60879] ipc.Server (Server.java:run(794)) - IPC Server listener on 60879: starting 2015-11-22 19:26:22,955 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:startResourceManager(339)) - MiniYARN ResourceManager address: ip-172-31-20-42.ap-northeast-1.compute.internal:39999 2015-11-22 19:26:22,956 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:startResourceManager(341)) - MiniYARN ResourceManager web address: ip-172-31-20-42.ap-northeast-1.compute.internal:50090 2015-11-22 19:26:22,956 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(459)) - MiniYARN ResourceManager address: ip-172-31-20-42.ap-northeast-1.compute.internal:39999 2015-11-22 19:26:22,956 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(461)) - MiniYARN ResourceManager web address: ip-172-31-20-42.ap-northeast-1.compute.internal:50090 2015-11-22 19:26:22,956 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(575)) - Waiting for NM 0 to start... 2015-11-22 19:26:22,958 INFO [Thread-2506] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:26:22,958 INFO [Socket Reader #1 for port 44404] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 44404 2015-11-22 19:26:22,960 INFO [Thread-2506] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.api.ContainerManagementProtocolPB to the server 2015-11-22 19:26:22,961 INFO [Thread-2506] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:serviceStart(464)) - Blocking new container-requests as container manager rpc server is still starting. 2015-11-22 19:26:22,961 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:26:22,961 INFO [IPC Server listener on 44404] ipc.Server (Server.java:run(794)) - IPC Server listener on 44404: starting 2015-11-22 19:26:22,964 INFO [Thread-2506] security.NMContainerTokenSecretManager (NMContainerTokenSecretManager.java:setNodeId(260)) - Updating node address : ip-172-31-20-42.ap-northeast-1.compute.internal:44404 2015-11-22 19:26:22,964 INFO [Thread-2506] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:26:22,965 INFO [Socket Reader #1 for port 49635] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 49635 2015-11-22 19:26:22,966 INFO [Thread-2506] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB to the server 2015-11-22 19:26:22,966 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:26:22,966 INFO [IPC Server listener on 49635] ipc.Server (Server.java:run(794)) - IPC Server listener on 49635: starting 2015-11-22 19:26:22,966 INFO [Thread-2506] localizer.ResourceLocalizationService (ResourceLocalizationService.java:serviceStart(363)) - Localizer started on port 49635 2015-11-22 19:26:22,975 INFO [Thread-2506] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:serviceStart(509)) - ContainerManager started at /172.31.20.42:44404 2015-11-22 19:26:22,975 INFO [Thread-2506] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:serviceStart(510)) - ContainerManager bound to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:0 2015-11-22 19:26:22,975 INFO [Thread-2506] webapp.WebServer (WebServer.java:serviceStart(71)) - Instantiating NMWebApp at ip-172-31-20-42.ap-northeast-1.compute.internal:0 2015-11-22 19:26:22,975 INFO [Thread-2506] server.AuthenticationFilter (AuthenticationFilter.java:constructSecretProvider(304)) - Unable to initialize FileSignerSecretProvider, falling back to use random secrets. 2015-11-22 19:26:22,976 INFO [Thread-2506] http.HttpRequestLog (HttpRequestLog.java:getRequestLog(80)) - Http request log for http.requests.nodemanager is not defined 2015-11-22 19:26:22,976 INFO [Thread-2506] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter) 2015-11-22 19:26:22,977 INFO [Thread-2506] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context node 2015-11-22 19:26:22,977 INFO [Thread-2506] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context logs 2015-11-22 19:26:22,977 INFO [Thread-2506] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context static 2015-11-22 19:26:22,977 INFO [Thread-2506] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context node 2015-11-22 19:26:22,977 INFO [Thread-2506] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2015-11-22 19:26:22,977 INFO [Thread-2506] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2015-11-22 19:26:22,978 INFO [Thread-2506] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'Timeline Authentication Filter' (class=org.apache.hadoop.yarn.server.timeline.security.TimelineAuthenticationFilter) 2015-11-22 19:26:22,978 INFO [Thread-2506] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /node/* 2015-11-22 19:26:22,978 INFO [Thread-2506] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /ws/* Nov 22, 2015 7:26:22 PM com.google.inject.servlet.GuiceFilter setPipeline WARNING: Multiple Servlet injectors detected. This is a warning indicating that you have more than one GuiceFilter running in your web application. If this is deliberate, you may safely ignore this message. If this is NOT deliberate however, your application may not work as expected. 2015-11-22 19:26:22,982 INFO [Thread-2506] webapp.WebApps (WebApps.java:build(288)) - Registered webapp guice modules 2015-11-22 19:26:22,983 INFO [Thread-2506] http.HttpServer2 (HttpServer2.java:openListeners(917)) - Jetty bound to port 60054 2015-11-22 19:26:22,983 INFO [Thread-2506] mortbay.log (Slf4jLog.java:info(67)) - jetty-6.1.26 2015-11-22 19:26:22,993 INFO [Thread-2506] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:26:22,994 INFO [Thread[Thread-2547,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2015-11-22 19:26:22,994 INFO [Thread[Thread-2547,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:26:22,995 INFO [Thread-2506] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:26:22,995 INFO [Thread[Thread-2549,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) Nov 22, 2015 7:26:22 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.nodemanager.webapp.NMWebServices as a root resource class Nov 22, 2015 7:26:22 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a provider class Nov 22, 2015 7:26:22 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.nodemanager.webapp.JAXBContextResolver as a provider class Nov 22, 2015 7:26:22 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate INFO: Initiating Jersey application, version 'Jersey: 1.9 09/02/2011 11:17 AM' Nov 22, 2015 7:26:23 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.nodemanager.webapp.JAXBContextResolver to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:26:23 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.webapp.GenericExceptionHandler to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:26:23 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.nodemanager.webapp.NMWebServices to GuiceManagedComponentProvider with the scope "Singleton" 2015-11-22 19:26:23,078 INFO [Thread-2506] mortbay.log (Slf4jLog.java:info(67)) - Started HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:60054 2015-11-22 19:26:23,080 INFO [Thread-2506] webapp.WebApps (WebApps.java:start(307)) - Web app node started at 60054 2015-11-22 19:26:23,081 INFO [Thread-2506] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:serviceStart(222)) - Node ID assigned is : ip-172-31-20-42.ap-northeast-1.compute.internal:44404 2015-11-22 19:26:23,081 INFO [Thread-2506] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNMContainerStatuses(550)) - Sending out 0 NM container statuses: [] 2015-11-22 19:26:23,081 INFO [Thread-2506] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:registerWithRM(330)) - Registering with RM using containers :[] 2015-11-22 19:26:23,082 INFO [Thread-2506] resourcemanager.ResourceTrackerService (ResourceTrackerService.java:registerNodeManager(375)) - NodeManager from node ip-172-31-20-42.ap-northeast-1.compute.internal(cmPort: 44404 httpPort: 60054) registered with capability: , assigned nodeId ip-172-31-20-42.ap-northeast-1.compute.internal:44404 2015-11-22 19:26:23,082 INFO [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(566)) - ip-172-31-20-42.ap-northeast-1.compute.internal:44404 Node Transitioned from NEW to RUNNING 2015-11-22 19:26:23,082 INFO [Thread-2506] security.NMContainerTokenSecretManager (NMContainerTokenSecretManager.java:setMasterKey(138)) - Rolling master-key for container-tokens, got key with id -1261666866 2015-11-22 19:26:23,083 INFO [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:addNode(1427)) - Added node ip-172-31-20-42.ap-northeast-1.compute.internal:44404 clusterResource: 2015-11-22 19:26:23,083 INFO [Thread-2506] security.NMTokenSecretManagerInNM (NMTokenSecretManagerInNM.java:setMasterKey(135)) - Rolling master-key for container-tokens, got key with id 203047445 2015-11-22 19:26:23,083 INFO [Thread-2506] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:registerWithRM(391)) - Registered with ResourceManager as ip-172-31-20-42.ap-northeast-1.compute.internal:44404 with total resource of 2015-11-22 19:26:23,083 INFO [Thread-2506] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:registerWithRM(392)) - Notifying ContainerManager to unblock new container-requests 2015-11-22 19:26:23,956 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(728)) - Waiting for Timeline Server to start... 2015-11-22 19:26:23,957 INFO [Thread-2552] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:26:23,957 INFO [Thread[Thread-2554,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2015-11-22 19:26:23,958 INFO [Thread[Thread-2554,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:26:23,958 INFO [Thread-2552] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:26:23,958 INFO [Socket Reader #1 for port 43026] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 43026 2015-11-22 19:26:23,960 INFO [Thread-2552] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.api.ApplicationHistoryProtocolPB to the server 2015-11-22 19:26:23,960 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:26:23,960 INFO [IPC Server listener on 43026] ipc.Server (Server.java:run(794)) - IPC Server listener on 43026: starting 2015-11-22 19:26:23,962 INFO [Thread-2552] applicationhistoryservice.ApplicationHistoryClientService (ApplicationHistoryClientService.java:serviceStart(111)) - Instantiated ApplicationHistoryClientService at ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:43026 2015-11-22 19:26:23,962 INFO [Thread-2552] applicationhistoryservice.ApplicationHistoryManagerImpl (ApplicationHistoryManagerImpl.java:serviceStart(71)) - Starting ApplicationHistory 2015-11-22 19:26:23,963 INFO [Thread-2552] server.AuthenticationFilter (AuthenticationFilter.java:constructSecretProvider(304)) - Unable to initialize FileSignerSecretProvider, falling back to use random secrets. 2015-11-22 19:26:23,964 INFO [Thread-2552] http.HttpRequestLog (HttpRequestLog.java:getRequestLog(80)) - Http request log for http.requests.applicationhistory is not defined 2015-11-22 19:26:23,964 INFO [Thread-2552] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter) 2015-11-22 19:26:23,965 INFO [Thread-2552] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context applicationhistory 2015-11-22 19:26:23,965 INFO [Thread-2552] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context logs 2015-11-22 19:26:23,965 INFO [Thread-2552] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context static 2015-11-22 19:26:23,965 INFO [Thread-2552] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context applicationhistory 2015-11-22 19:26:23,965 INFO [Thread-2552] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2015-11-22 19:26:23,965 INFO [Thread-2552] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2015-11-22 19:26:23,965 INFO [Thread-2552] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'Timeline Authentication Filter' (class=org.apache.hadoop.yarn.server.timeline.security.TimelineAuthenticationFilter) 2015-11-22 19:26:23,966 INFO [Thread-2552] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /applicationhistory/* 2015-11-22 19:26:23,966 INFO [Thread-2552] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /ws/* Nov 22, 2015 7:26:23 PM com.google.inject.servlet.GuiceFilter setPipeline WARNING: Multiple Servlet injectors detected. This is a warning indicating that you have more than one GuiceFilter running in your web application. If this is deliberate, you may safely ignore this message. If this is NOT deliberate however, your application may not work as expected. 2015-11-22 19:26:23,971 INFO [Thread-2552] webapp.WebApps (WebApps.java:build(288)) - Registered webapp guice modules 2015-11-22 19:26:23,972 INFO [Thread-2552] http.HttpServer2 (HttpServer2.java:openListeners(917)) - Jetty bound to port 51186 2015-11-22 19:26:23,972 INFO [Thread-2552] mortbay.log (Slf4jLog.java:info(67)) - jetty-6.1.26 2015-11-22 19:26:23,982 INFO [Thread-2552] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:26:23,983 INFO [Thread[Thread-2572,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2015-11-22 19:26:23,983 INFO [Thread[Thread-2572,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:26:23,984 INFO [Thread-2552] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:26:23,984 INFO [Thread[Thread-2574,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) Nov 22, 2015 7:26:23 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider as a provider class Nov 22, 2015 7:26:23 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.AHSWebServices as a root resource class Nov 22, 2015 7:26:23 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.timeline.webapp.TimelineWebServices as a root resource class Nov 22, 2015 7:26:23 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a provider class Nov 22, 2015 7:26:23 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate INFO: Initiating Jersey application, version 'Jersey: 1.9 09/02/2011 11:17 AM' Nov 22, 2015 7:26:23 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.webapp.GenericExceptionHandler to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:26:23 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:26:24 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.AHSWebServices to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:26:24 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.timeline.webapp.TimelineWebServices to GuiceManagedComponentProvider with the scope "Singleton" 2015-11-22 19:26:24,059 INFO [Thread-2552] mortbay.log (Slf4jLog.java:info(67)) - Started HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:51186 2015-11-22 19:26:24,060 INFO [Thread-2552] applicationhistoryservice.ApplicationHistoryServer (ApplicationHistoryServer.java:startWebApp(342)) - Instantiating AHSWebApp at 51186 2015-11-22 19:26:25,457 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(739)) - MiniYARN ApplicationHistoryServer address: ip-172-31-20-42.ap-northeast-1.compute.internal:43026 2015-11-22 19:26:25,457 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(741)) - MiniYARN ApplicationHistoryServer web address: ip-172-31-20-42.ap-northeast-1.compute.internal:51186 2015-11-22 19:26:27,503 INFO [Thread-2576] distributedshell.TestDistributedShell (TestDistributedShell.java:testDebugFlag(836)) - Initializing DS Client 2015-11-22 19:26:27,548 INFO [Thread-2576] impl.TimelineClientImpl (TimelineClientImpl.java:serviceInit(296)) - Timeline service address: http://ip-172-31-20-42.ap-northeast-1.compute.internal:51186/ws/v1/timeline/ 2015-11-22 19:26:27,548 INFO [Thread-2576] distributedshell.TestDistributedShell (TestDistributedShell.java:testDebugFlag(839)) - Running DS Client 2015-11-22 19:26:27,549 INFO [Thread-2576] distributedshell.Client (Client.java:run(443)) - Running Client 2015-11-22 19:26:27,564 INFO [Thread-2576] client.RMProxy (RMProxy.java:createRMProxy(98)) - Connecting to ResourceManager at ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:39999 2015-11-22 19:26:27,568 INFO [Thread-2576] distributedshell.Client (Client.java:run(447)) - Got Cluster metric info from ASM, numNodeManagers=1 2015-11-22 19:26:27,569 INFO [Thread-2576] distributedshell.Client (Client.java:run(452)) - Got Cluster node info from ASM 2015-11-22 19:26:27,570 INFO [Thread-2576] distributedshell.Client (Client.java:run(454)) - Got node report from ASM for, nodeId=ip-172-31-20-42.ap-northeast-1.compute.internal:44404, nodeAddress=ip-172-31-20-42.ap-northeast-1.compute.internal:60054, nodeRackName=/default-rack, nodeNumContainers=0 2015-11-22 19:26:27,570 INFO [Thread-2576] distributedshell.Client (Client.java:run(462)) - Queue info, queueName=default, queueCurrentCapacity=0.0, queueMaxCapacity=1.0, queueApplicationCount=0, queueChildQueueCount=0 2015-11-22 19:26:27,571 INFO [Thread-2576] distributedshell.Client (Client.java:run(472)) - User ACL Info for Queue, queueName=root, userAcl=SUBMIT_APPLICATIONS 2015-11-22 19:26:27,571 INFO [Thread-2576] distributedshell.Client (Client.java:run(472)) - User ACL Info for Queue, queueName=root, userAcl=ADMINISTER_QUEUE 2015-11-22 19:26:27,571 INFO [Thread-2576] distributedshell.Client (Client.java:run(472)) - User ACL Info for Queue, queueName=default, userAcl=SUBMIT_APPLICATIONS 2015-11-22 19:26:27,571 INFO [Thread-2576] distributedshell.Client (Client.java:run(472)) - User ACL Info for Queue, queueName=default, userAcl=ADMINISTER_QUEUE 2015-11-22 19:26:27,572 INFO [IPC Server handler 3 on 39999] resourcemanager.ClientRMService (ClientRMService.java:getNewApplicationId(305)) - Allocated new applicationId: 1 2015-11-22 19:26:27,572 INFO [Thread-2576] distributedshell.Client (Client.java:run(491)) - Max mem capability of resources in this cluster 4096 2015-11-22 19:26:27,572 INFO [Thread-2576] distributedshell.Client (Client.java:run(502)) - Max virtual cores capability of resources in this cluster 4 2015-11-22 19:26:27,572 INFO [Thread-2576] distributedshell.Client (Client.java:run(528)) - Copy App Master jar from local filesystem and add to local environment 2015-11-22 19:26:27,579 INFO [Thread-2576] distributedshell.Client (Client.java:run(577)) - Set the environment for the application master 2015-11-22 19:26:27,579 INFO [Thread-2576] distributedshell.Client (Client.java:run(619)) - Setting up app master command 2015-11-22 19:26:27,579 INFO [Thread-2576] distributedshell.Client (Client.java:run(650)) - Completed setting up app master command {{JAVA_HOME}}/bin/java -Xmx512m org.apache.hadoop.yarn.applications.distributedshell.ApplicationMaster --container_memory 128 --container_vcores 1 --num_containers 2 --priority 0 --debug 1>/AppMaster.stdout 2>/AppMaster.stderr 2015-11-22 19:26:27,579 INFO [Thread-2576] distributedshell.Client (Client.java:run(706)) - Submitting application to ASM 2015-11-22 19:26:27,580 INFO [IPC Server handler 4 on 39999] capacity.CapacityScheduler (CapacityScheduler.java:checkAndGetApplicationPriority(1941)) - Priority '0' is acceptable in queue : default for application: application_1448220381455_0001 for the user: ubuntu 2015-11-22 19:26:27,580 WARN [IPC Server handler 4 on 39999] rmapp.RMAppImpl (RMAppImpl.java:(420)) - The specific max attempts: 0 for application: 1 is invalid, because it is out of the range [1, 2]. Use the global max attempts instead. 2015-11-22 19:26:27,581 INFO [IPC Server handler 4 on 39999] resourcemanager.ClientRMService (ClientRMService.java:submitApplication(602)) - Application with id 1 submitted by user ubuntu 2015-11-22 19:26:27,581 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:transition(1079)) - Storing application with id application_1448220381455_0001 2015-11-22 19:26:27,581 INFO [IPC Server handler 4 on 39999] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(154)) - USER=ubuntu IP=172.31.20.42 OPERATION=Submit Application Request TARGET=ClientRMService RESULT=SUCCESS APPID=application_1448220381455_0001 2015-11-22 19:26:27,581 INFO [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:transition(208)) - Storing info for app: application_1448220381455_0001 2015-11-22 19:26:27,581 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220381455_0001 State change from NEW to NEW_SAVING 2015-11-22 19:26:27,581 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220381455_0001 State change from NEW_SAVING to SUBMITTED 2015-11-22 19:26:27,581 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:addApplication(349)) - Application added - appId: application_1448220381455_0001 user: ubuntu leaf-queue of parent: root #applications: 1 2015-11-22 19:26:27,581 INFO [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:addApplication(765)) - Accepted application application_1448220381455_0001 from user: ubuntu, in queue: default 2015-11-22 19:26:27,582 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220381455_0001 State change from SUBMITTED to ACCEPTED 2015-11-22 19:26:27,582 INFO [AsyncDispatcher event handler] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:registerAppAttempt(661)) - Registering app attempt : appattempt_1448220381455_0001_000001 2015-11-22 19:26:27,582 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220381455_0001_000001 State change from NEW to SUBMITTED 2015-11-22 19:26:27,582 INFO [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:activateApplications(705)) - Application application_1448220381455_0001 from user: ubuntu activated in queue: default 2015-11-22 19:26:27,582 INFO [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:addApplicationAttempt(720)) - Application added - appId: application_1448220381455_0001 user: ubuntu, leaf-queue: default #user-pending-applications: 0 #user-active-applications: 1 #queue-pending-applications: 0 #queue-active-applications: 1 2015-11-22 19:26:27,583 INFO [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:addApplicationAttempt(805)) - Added Application Attempt appattempt_1448220381455_0001_000001 to scheduler from user ubuntu in queue default 2015-11-22 19:26:27,584 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220381455_0001_000001 State change from SUBMITTED to SCHEDULED 2015-11-22 19:26:27,784 INFO [Thread-2576] impl.YarnClientImpl (YarnClientImpl.java:submitApplication(279)) - Submitted application application_1448220381455_0001 2015-11-22 19:26:28,088 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220381455_0001_01_000001 Container Transitioned from NEW to ALLOCATED 2015-11-22 19:26:28,088 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220381455_0001 CONTAINERID=container_1448220381455_0001_01_000001 2015-11-22 19:26:28,088 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(154)) - Assigned container container_1448220381455_0001_01_000001 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:44404, which has 1 containers, used and available after allocation 2015-11-22 19:26:28,089 INFO [ResourceManager Event Processor] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(89)) - assignedContainer application attempt=appattempt_1448220381455_0001_000001 container=container_1448220381455_0001_01_000001 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@62646a2e clusterResource= type=OFF_SWITCH 2015-11-22 19:26:28,090 INFO [AsyncDispatcher event handler] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:createAndGetNMToken(200)) - Sending NMToken for nodeId : ip-172-31-20-42.ap-northeast-1.compute.internal:44404 for container : container_1448220381455_0001_01_000001 2015-11-22 19:26:28,090 INFO [AsyncDispatcher event handler] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220381455_0001_01_000001 Container Transitioned from ALLOCATED to ACQUIRED 2015-11-22 19:26:28,091 INFO [AsyncDispatcher event handler] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:clearNodeSetForAttempt(146)) - Clear node set for appattempt_1448220381455_0001_000001 2015-11-22 19:26:28,091 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(589)) - Re-sorting assigned queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.125, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 2015-11-22 19:26:28,091 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:storeAttempt(2026)) - Storing attempt: AppId: application_1448220381455_0001 AttemptId: appattempt_1448220381455_0001_000001 MasterContainer: Container: [ContainerId: container_1448220381455_0001_01_000001, NodeId: ip-172-31-20-42.ap-northeast-1.compute.internal:44404, NodeHttpAddress: ip-172-31-20-42.ap-northeast-1.compute.internal:60054, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.31.20.42:44404 }, ] 2015-11-22 19:26:28,091 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(461)) - assignedContainer queue=root usedCapacity=0.125 absoluteUsedCapacity=0.125 used= cluster= 2015-11-22 19:26:28,091 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220381455_0001_000001 State change from SCHEDULED to ALLOCATED_SAVING 2015-11-22 19:26:28,092 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220381455_0001_000001 State change from ALLOCATED_SAVING to ALLOCATED 2015-11-22 19:26:28,092 INFO [ApplicationMasterLauncher #0] amlauncher.AMLauncher (AMLauncher.java:run(254)) - Launching masterappattempt_1448220381455_0001_000001 2015-11-22 19:26:28,094 INFO [ApplicationMasterLauncher #0] amlauncher.AMLauncher (AMLauncher.java:launch(106)) - Setting up container Container: [ContainerId: container_1448220381455_0001_01_000001, NodeId: ip-172-31-20-42.ap-northeast-1.compute.internal:44404, NodeHttpAddress: ip-172-31-20-42.ap-northeast-1.compute.internal:60054, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.31.20.42:44404 }, ] for AM appattempt_1448220381455_0001_000001 2015-11-22 19:26:28,094 INFO [ApplicationMasterLauncher #0] amlauncher.AMLauncher (AMLauncher.java:createAMContainerLaunchContext(188)) - Command to launch container container_1448220381455_0001_01_000001 : {{JAVA_HOME}}/bin/java -Xmx512m org.apache.hadoop.yarn.applications.distributedshell.ApplicationMaster --container_memory 128 --container_vcores 1 --num_containers 2 --priority 0 --debug 1>/AppMaster.stdout 2>/AppMaster.stderr 2015-11-22 19:26:28,094 INFO [ApplicationMasterLauncher #0] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:createAndGetAMRMToken(195)) - Create AMRMToken for ApplicationAttempt: appattempt_1448220381455_0001_000001 2015-11-22 19:26:28,095 INFO [ApplicationMasterLauncher #0] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:createPassword(307)) - Creating password for appattempt_1448220381455_0001_000001 2015-11-22 19:26:28,098 INFO [Socket Reader #1 for port 44404] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220381455_0001_000001 (auth:SIMPLE) 2015-11-22 19:26:28,101 INFO [IPC Server handler 0 on 44404] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:startContainerInternal(889)) - Start request for container_1448220381455_0001_01_000001 by user ubuntu 2015-11-22 19:26:28,102 INFO [IPC Server handler 0 on 44404] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:startContainerInternal(930)) - Creating a new application reference for app application_1448220381455_0001 2015-11-22 19:26:28,102 INFO [IPC Server handler 0 on 44404] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu IP=172.31.20.42 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1448220381455_0001 CONTAINERID=container_1448220381455_0001_01_000001 2015-11-22 19:26:28,102 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(463)) - Application application_1448220381455_0001 transitioned from NEW to INITING 2015-11-22 19:26:28,103 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(303)) - Adding container_1448220381455_0001_01_000001 to application application_1448220381455_0001 2015-11-22 19:26:28,104 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(463)) - Application application_1448220381455_0001 transitioned from INITING to RUNNING 2015-11-22 19:26:28,104 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220381455_0001_01_000001 transitioned from NEW to LOCALIZING 2015-11-22 19:26:28,104 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_INIT for appId application_1448220381455_0001 2015-11-22 19:26:28,105 INFO [AsyncDispatcher event handler] localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource file:/home/ubuntu/DistributedShell/application_1448220381455_0001/AppMaster.jar transitioned from INIT to DOWNLOADING 2015-11-22 19:26:28,105 INFO [AsyncDispatcher event handler] localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource file:/home/ubuntu/DistributedShell/application_1448220381455_0001/shellCommands transitioned from INIT to DOWNLOADING 2015-11-22 19:26:28,105 INFO [AsyncDispatcher event handler] localizer.ResourceLocalizationService (ResourceLocalizationService.java:handle(742)) - Created localizer for container_1448220381455_0001_01_000001 2015-11-22 19:26:28,105 INFO [ApplicationMasterLauncher #0] amlauncher.AMLauncher (AMLauncher.java:launch(127)) - Done launching container Container: [ContainerId: container_1448220381455_0001_01_000001, NodeId: ip-172-31-20-42.ap-northeast-1.compute.internal:44404, NodeHttpAddress: ip-172-31-20-42.ap-northeast-1.compute.internal:60054, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.31.20.42:44404 }, ] for AM appattempt_1448220381455_0001_000001 2015-11-22 19:26:28,106 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220381455_0001_000001 State change from ALLOCATED to LAUNCHED 2015-11-22 19:26:28,109 INFO [LocalizerRunner for container_1448220381455_0001_01_000001] localizer.ResourceLocalizationService (ResourceLocalizationService.java:writeCredentials(1221)) - Writing credentials to the nmPrivate file /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/container_1448220381455_0001_01_000001.tokens. Credentials list: 2015-11-22 19:26:28,111 INFO [LocalizerRunner for container_1448220381455_0001_01_000001] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:createUserCacheDirs(632)) - Initializing user ubuntu 2015-11-22 19:26:28,118 INFO [LocalizerRunner for container_1448220381455_0001_01_000001] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:startLocalizer(126)) - Copying from /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/container_1448220381455_0001_01_000001.tokens to /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220381455_0001/container_1448220381455_0001_01_000001.tokens 2015-11-22 19:26:28,118 INFO [LocalizerRunner for container_1448220381455_0001_01_000001] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:startLocalizer(133)) - Localizer CWD set to /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220381455_0001 = file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220381455_0001 2015-11-22 19:26:28,158 INFO [IPC Server handler 4 on 49635] localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource file:/home/ubuntu/DistributedShell/application_1448220381455_0001/AppMaster.jar(->/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220381455_0001/filecache/10/AppMaster.jar) transitioned from DOWNLOADING to LOCALIZED 2015-11-22 19:26:28,171 INFO [IPC Server handler 3 on 49635] localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource file:/home/ubuntu/DistributedShell/application_1448220381455_0001/shellCommands(->/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220381455_0001/filecache/11/shellCommands) transitioned from DOWNLOADING to LOCALIZED 2015-11-22 19:26:28,173 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220381455_0001_01_000001 transitioned from LOCALIZING to LOCALIZED 2015-11-22 19:26:28,191 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220381455_0001_01_000001 transitioned from LOCALIZED to RUNNING 2015-11-22 19:26:28,207 INFO [ContainersLauncher #0] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:buildCommandExecutor(281)) - launchContainer: [nice, -n, 0, bash, /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220381455_0001/container_1448220381455_0001_01_000001/default_container_executor.sh] 2015-11-22 19:26:28,786 INFO [Thread-2576] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=N/A, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220387580, yarnAppState=ACCEPTED, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:50090/proxy/application_1448220381455_0001/, appUser=ubuntu 2015-11-22 19:26:29,088 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220381455_0001_01_000001 Container Transitioned from ACQUIRED to RUNNING 2015-11-22 19:26:29,788 INFO [Thread-2576] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=N/A, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220387580, yarnAppState=ACCEPTED, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:50090/proxy/application_1448220381455_0001/, appUser=ubuntu 2015-11-22 19:26:30,731 INFO [Socket Reader #1 for port 47916] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220381455_0001_000001 (auth:SIMPLE) 2015-11-22 19:26:30,738 INFO [IPC Server handler 1 on 47916] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:registerApplicationMaster(220)) - AM registration appattempt_1448220381455_0001_000001 2015-11-22 19:26:30,738 INFO [IPC Server handler 1 on 47916] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(133)) - USER=ubuntu IP=172.31.20.42 OPERATION=Register App Master TARGET=ApplicationMasterService RESULT=SUCCESS APPID=application_1448220381455_0001 APPATTEMPTID=appattempt_1448220381455_0001_000001 2015-11-22 19:26:30,739 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220381455_0001_000001 State change from LAUNCHED to RUNNING 2015-11-22 19:26:30,740 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220381455_0001 State change from ACCEPTED to RUNNING 2015-11-22 19:26:30,789 INFO [Thread-2576] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220387580, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:50090/proxy/application_1448220381455_0001/, appUser=ubuntu 2015-11-22 19:26:31,791 INFO [Thread-2576] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220387580, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:50090/proxy/application_1448220381455_0001/, appUser=ubuntu 2015-11-22 19:26:32,089 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220381455_0001_01_000002 Container Transitioned from NEW to ALLOCATED 2015-11-22 19:26:32,089 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220381455_0001 CONTAINERID=container_1448220381455_0001_01_000002 2015-11-22 19:26:32,089 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(154)) - Assigned container container_1448220381455_0001_01_000002 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:44404, which has 2 containers, used and available after allocation 2015-11-22 19:26:32,089 INFO [ResourceManager Event Processor] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(89)) - assignedContainer application attempt=appattempt_1448220381455_0001_000001 container=container_1448220381455_0001_01_000002 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@62646a2e clusterResource= type=OFF_SWITCH 2015-11-22 19:26:32,090 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(589)) - Re-sorting assigned queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.15625, absoluteUsedCapacity=0.15625, numApps=1, numContainers=2 2015-11-22 19:26:32,090 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(461)) - assignedContainer queue=root usedCapacity=0.15625 absoluteUsedCapacity=0.15625 used= cluster= 2015-11-22 19:26:32,778 INFO [IPC Server handler 0 on 47916] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:createAndGetNMToken(200)) - Sending NMToken for nodeId : ip-172-31-20-42.ap-northeast-1.compute.internal:44404 for container : container_1448220381455_0001_01_000002 2015-11-22 19:26:32,779 INFO [IPC Server handler 0 on 47916] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220381455_0001_01_000002 Container Transitioned from ALLOCATED to ACQUIRED 2015-11-22 19:26:32,792 INFO [Thread-2576] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220387580, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:50090/proxy/application_1448220381455_0001/, appUser=ubuntu 2015-11-22 19:26:32,841 INFO [Socket Reader #1 for port 44404] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220381455_0001_000001 (auth:SIMPLE) 2015-11-22 19:26:32,845 INFO [IPC Server handler 1 on 44404] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:startContainerInternal(889)) - Start request for container_1448220381455_0001_01_000002 by user ubuntu 2015-11-22 19:26:32,846 INFO [IPC Server handler 1 on 44404] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu IP=172.31.20.42 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1448220381455_0001 CONTAINERID=container_1448220381455_0001_01_000002 2015-11-22 19:26:32,846 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(303)) - Adding container_1448220381455_0001_01_000002 to application application_1448220381455_0001 2015-11-22 19:26:32,847 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220381455_0001_01_000002 transitioned from NEW to LOCALIZED 2015-11-22 19:26:32,847 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_INIT for appId application_1448220381455_0001 2015-11-22 19:26:32,881 INFO [Socket Reader #1 for port 44404] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220381455_0001_000001 (auth:SIMPLE) 2015-11-22 19:26:32,886 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220381455_0001_01_000002 transitioned from LOCALIZED to RUNNING 2015-11-22 19:26:32,892 INFO [IPC Server handler 3 on 44404] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:getContainerStatusInternal(1221)) - Getting container-status for container_1448220381455_0001_01_000002 2015-11-22 19:26:32,892 INFO [IPC Server handler 3 on 44404] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:getContainerStatusInternal(1235)) - Returning ContainerStatus: [ContainerId: container_1448220381455_0001_01_000002, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ] 2015-11-22 19:26:32,918 INFO [ContainersLauncher #1] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:buildCommandExecutor(281)) - launchContainer: [nice, -n, 0, bash, /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220381455_0001/container_1448220381455_0001_01_000002/default_container_executor.sh] 2015-11-22 19:26:32,927 INFO [ContainersLauncher #1] launcher.ContainerLaunch (ContainerLaunch.java:call(356)) - Container container_1448220381455_0001_01_000002 succeeded 2015-11-22 19:26:32,928 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220381455_0001_01_000002 transitioned from RUNNING to EXITED_WITH_SUCCESS 2015-11-22 19:26:32,928 INFO [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(380)) - Cleaning up container container_1448220381455_0001_01_000002 2015-11-22 19:26:32,941 INFO [DeletionService #0] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(480)) - Deleting absolute path : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220381455_0001/container_1448220381455_0001_01_000002 2015-11-22 19:26:32,942 INFO [AsyncDispatcher event handler] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu OPERATION=Container Finished - Succeeded TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1448220381455_0001 CONTAINERID=container_1448220381455_0001_01_000002 2015-11-22 19:26:32,942 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220381455_0001_01_000002 transitioned from EXITED_WITH_SUCCESS to DONE 2015-11-22 19:26:32,942 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(346)) - Removing container_1448220381455_0001_01_000002 from application application_1448220381455_0001 2015-11-22 19:26:32,942 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_STOP for appId application_1448220381455_0001 2015-11-22 19:26:33,089 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220381455_0001_01_000002 Container Transitioned from ACQUIRED to COMPLETED 2015-11-22 19:26:33,089 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220381455_0001 CONTAINERID=container_1448220381455_0001_01_000002 2015-11-22 19:26:33,089 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:releaseContainer(248)) - Released container container_1448220381455_0001_01_000002 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:44404, which currently has 1 containers, used and available, release resources=true 2015-11-22 19:26:33,090 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220381455_0001_01_000003 Container Transitioned from NEW to ALLOCATED 2015-11-22 19:26:33,090 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220381455_0001 CONTAINERID=container_1448220381455_0001_01_000003 2015-11-22 19:26:33,090 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(154)) - Assigned container container_1448220381455_0001_01_000003 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:44404, which has 2 containers, used and available after allocation 2015-11-22 19:26:33,090 INFO [ResourceManager Event Processor] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(89)) - assignedContainer application attempt=appattempt_1448220381455_0001_000001 container=container_1448220381455_0001_01_000003 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@62646a2e clusterResource= type=OFF_SWITCH 2015-11-22 19:26:33,090 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(589)) - Re-sorting assigned queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.15625, absoluteUsedCapacity=0.15625, numApps=1, numContainers=2 2015-11-22 19:26:33,090 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(461)) - assignedContainer queue=root usedCapacity=0.15625 absoluteUsedCapacity=0.15625 used= cluster= 2015-11-22 19:26:33,787 INFO [IPC Server handler 2 on 47916] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220381455_0001_01_000003 Container Transitioned from ALLOCATED to ACQUIRED 2015-11-22 19:26:33,794 INFO [Thread-2576] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220387580, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:50090/proxy/application_1448220381455_0001/, appUser=ubuntu 2015-11-22 19:26:33,831 INFO [Socket Reader #1 for port 44404] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220381455_0001_000001 (auth:SIMPLE) 2015-11-22 19:26:33,833 INFO [IPC Server handler 1 on 44404] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:startContainerInternal(889)) - Start request for container_1448220381455_0001_01_000003 by user ubuntu 2015-11-22 19:26:33,833 INFO [IPC Server handler 1 on 44404] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu IP=172.31.20.42 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1448220381455_0001 CONTAINERID=container_1448220381455_0001_01_000003 2015-11-22 19:26:33,834 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(303)) - Adding container_1448220381455_0001_01_000003 to application application_1448220381455_0001 2015-11-22 19:26:33,834 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220381455_0001_01_000003 transitioned from NEW to LOCALIZED 2015-11-22 19:26:33,834 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_INIT for appId application_1448220381455_0001 2015-11-22 19:26:33,847 INFO [Socket Reader #1 for port 44404] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220381455_0001_000001 (auth:SIMPLE) 2015-11-22 19:26:33,854 INFO [IPC Server handler 3 on 44404] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:getContainerStatusInternal(1221)) - Getting container-status for container_1448220381455_0001_01_000003 2015-11-22 19:26:33,854 INFO [IPC Server handler 3 on 44404] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:getContainerStatusInternal(1235)) - Returning ContainerStatus: [ContainerId: container_1448220381455_0001_01_000003, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ] 2015-11-22 19:26:33,861 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220381455_0001_01_000003 transitioned from LOCALIZED to RUNNING 2015-11-22 19:26:33,893 INFO [ContainersLauncher #1] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:buildCommandExecutor(281)) - launchContainer: [nice, -n, 0, bash, /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220381455_0001/container_1448220381455_0001_01_000003/default_container_executor.sh] 2015-11-22 19:26:33,902 INFO [ContainersLauncher #1] launcher.ContainerLaunch (ContainerLaunch.java:call(356)) - Container container_1448220381455_0001_01_000003 succeeded 2015-11-22 19:26:33,902 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220381455_0001_01_000003 transitioned from RUNNING to EXITED_WITH_SUCCESS 2015-11-22 19:26:33,902 INFO [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(380)) - Cleaning up container container_1448220381455_0001_01_000003 2015-11-22 19:26:33,915 INFO [DeletionService #0] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(480)) - Deleting absolute path : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220381455_0001/container_1448220381455_0001_01_000003 2015-11-22 19:26:33,915 INFO [AsyncDispatcher event handler] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu OPERATION=Container Finished - Succeeded TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1448220381455_0001 CONTAINERID=container_1448220381455_0001_01_000003 2015-11-22 19:26:33,916 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220381455_0001_01_000003 transitioned from EXITED_WITH_SUCCESS to DONE 2015-11-22 19:26:33,916 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(346)) - Removing container_1448220381455_0001_01_000003 from application application_1448220381455_0001 2015-11-22 19:26:33,916 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_STOP for appId application_1448220381455_0001 2015-11-22 19:26:34,089 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220381455_0001_01_000003 Container Transitioned from ACQUIRED to COMPLETED 2015-11-22 19:26:34,090 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220381455_0001 CONTAINERID=container_1448220381455_0001_01_000003 2015-11-22 19:26:34,090 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:releaseContainer(248)) - Released container container_1448220381455_0001_01_000003 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:44404, which currently has 1 containers, used and available, release resources=true 2015-11-22 19:26:34,795 INFO [Thread-2576] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220387580, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:50090/proxy/application_1448220381455_0001/, appUser=ubuntu 2015-11-22 19:26:35,008 INFO [Socket Reader #1 for port 44404] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220381455_0001_000001 (auth:SIMPLE) 2015-11-22 19:26:35,012 INFO [IPC Server handler 0 on 44404] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:stopContainerInternal(1163)) - Stopping container with container Id: container_1448220381455_0001_01_000003 2015-11-22 19:26:35,012 INFO [IPC Server handler 0 on 44404] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu IP=172.31.20.42 OPERATION=Stop Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1448220381455_0001 CONTAINERID=container_1448220381455_0001_01_000003 2015-11-22 19:26:35,014 INFO [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:removeOrTrackCompletedContainersFromContext(605)) - Removed completed containers from NM context: [container_1448220381455_0001_01_000002] 2015-11-22 19:26:35,020 INFO [Socket Reader #1 for port 44404] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220381455_0001_000001 (auth:SIMPLE) 2015-11-22 19:26:35,022 INFO [IPC Server handler 1 on 44404] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:stopContainerInternal(1163)) - Stopping container with container Id: container_1448220381455_0001_01_000002 2015-11-22 19:26:35,027 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:rememberTargetTransitionsAndStoreState(1258)) - Updating application attempt appattempt_1448220381455_0001_000001 with final state: FINISHING, and exit status: -1000 2015-11-22 19:26:35,027 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220381455_0001_000001 State change from RUNNING to FINAL_SAVING 2015-11-22 19:26:35,027 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:rememberTargetTransitionsAndStoreState(1098)) - Updating application application_1448220381455_0001 with final state: FINISHING 2015-11-22 19:26:35,028 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220381455_0001 State change from RUNNING to FINAL_SAVING 2015-11-22 19:26:35,032 INFO [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:transition(237)) - Updating info for app: application_1448220381455_0001 2015-11-22 19:26:35,032 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220381455_0001_000001 State change from FINAL_SAVING to FINISHING 2015-11-22 19:26:35,032 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220381455_0001 State change from FINAL_SAVING to FINISHING 2015-11-22 19:26:35,129 INFO [IPC Server handler 6 on 47916] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:finishApplicationMaster(300)) - application_1448220381455_0001 unregistered successfully. 2015-11-22 19:26:35,145 INFO [ContainersLauncher #0] launcher.ContainerLaunch (ContainerLaunch.java:call(356)) - Container container_1448220381455_0001_01_000001 succeeded 2015-11-22 19:26:35,145 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220381455_0001_01_000001 transitioned from RUNNING to EXITED_WITH_SUCCESS 2015-11-22 19:26:35,145 INFO [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(380)) - Cleaning up container container_1448220381455_0001_01_000001 2015-11-22 19:26:35,163 INFO [DeletionService #1] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(480)) - Deleting absolute path : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220381455_0001/container_1448220381455_0001_01_000001 2015-11-22 19:26:35,163 INFO [AsyncDispatcher event handler] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu OPERATION=Container Finished - Succeeded TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1448220381455_0001 CONTAINERID=container_1448220381455_0001_01_000001 2015-11-22 19:26:35,163 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220381455_0001_01_000001 transitioned from EXITED_WITH_SUCCESS to DONE 2015-11-22 19:26:35,164 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(346)) - Removing container_1448220381455_0001_01_000001 from application application_1448220381455_0001 2015-11-22 19:26:35,164 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_STOP for appId application_1448220381455_0001 2015-11-22 19:26:35,797 INFO [Thread-2576] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220387580, yarnAppState=FINISHED, distributedFinalState=SUCCEEDED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:50090/proxy/application_1448220381455_0001/, appUser=ubuntu 2015-11-22 19:26:35,797 INFO [Thread-2576] distributedshell.Client (Client.java:monitorApplication(759)) - Application has completed successfully. Breaking monitoring loop 2015-11-22 19:26:35,799 ERROR [Thread[Thread-2572,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(670)) - ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted 2015-11-22 19:26:35,801 INFO [main] mortbay.log (Slf4jLog.java:info(67)) - Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:0 2015-11-22 19:26:35,903 INFO [main] applicationhistoryservice.ApplicationHistoryManagerImpl (ApplicationHistoryManagerImpl.java:serviceStop(78)) - Stopping ApplicationHistory 2015-11-22 19:26:35,904 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 43026 2015-11-22 19:26:35,906 INFO [IPC Server listener on 43026] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 43026 2015-11-22 19:26:35,907 ERROR [Thread[Thread-2554,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(670)) - ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted 2015-11-22 19:26:35,907 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:26:35,908 INFO [main] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:unRegisterNM(268)) - Successfully Unregistered the Node ip-172-31-20-42.ap-northeast-1.compute.internal:44404 with ResourceManager. 2015-11-22 19:26:35,911 ERROR [Thread[Thread-2547,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(670)) - ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted 2015-11-22 19:26:35,913 INFO [main] mortbay.log (Slf4jLog.java:info(67)) - Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:0 2015-11-22 19:26:36,015 INFO [main] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:cleanUpApplicationsOnNMShutDown(556)) - Applications still running : [application_1448220381455_0001] 2015-11-22 19:26:36,015 INFO [main] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:cleanUpApplicationsOnNMShutDown(572)) - Waiting for Applications to be Finished 2015-11-22 19:26:36,015 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(463)) - Application application_1448220381455_0001 transitioned from RUNNING to APPLICATION_RESOURCES_CLEANINGUP 2015-11-22 19:26:36,016 INFO [DeletionService #3] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(480)) - Deleting absolute path : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220381455_0001 2015-11-22 19:26:36,016 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event APPLICATION_STOP for appId application_1448220381455_0001 2015-11-22 19:26:36,016 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(463)) - Application application_1448220381455_0001 transitioned from APPLICATION_RESOURCES_CLEANINGUP to FINISHED 2015-11-22 19:26:36,016 INFO [AsyncDispatcher event handler] loghandler.NonAggregatingLogHandler (NonAggregatingLogHandler.java:handle(167)) - Scheduling Log Deletion for application: application_1448220381455_0001, with delay of 10800 seconds 2015-11-22 19:26:40,016 INFO [main] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:cleanUpApplicationsOnNMShutDown(589)) - Done waiting for Applications to be Finished. Still alive: [application_1448220381455_0001] 2015-11-22 19:26:40,017 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 44404 2015-11-22 19:26:40,021 INFO [IPC Server listener on 44404] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 44404 2015-11-22 19:26:40,022 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:26:50,023 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 49635 2015-11-22 19:26:50,025 INFO [IPC Server listener on 49635] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 49635 2015-11-22 19:26:50,025 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:26:50,026 INFO [Public Localizer] localizer.ResourceLocalizationService (ResourceLocalizationService.java:run(908)) - Public cache exiting 2015-11-22 19:26:50,026 WARN [Node Resource Monitor] nodemanager.NodeResourceMonitorImpl (NodeResourceMonitorImpl.java:run(154)) - org.apache.hadoop.yarn.server.nodemanager.NodeResourceMonitorImpl is interrupted. Exiting. 2015-11-22 19:26:50,031 ERROR [Thread[Thread-2497,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(670)) - ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted 2015-11-22 19:26:50,033 INFO [main] mortbay.log (Slf4jLog.java:info(67)) - Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:0 2015-11-22 19:26:50,135 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 39999 2015-11-22 19:26:50,136 INFO [IPC Server listener on 39999] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 39999 2015-11-22 19:26:50,136 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:26:50,137 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 60879 2015-11-22 19:26:50,139 INFO [IPC Server listener on 60879] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 60879 2015-11-22 19:26:50,140 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:26:50,141 INFO [main] resourcemanager.ResourceManager (ResourceManager.java:transitionToStandby(1051)) - Transitioning to standby state 2015-11-22 19:26:50,141 WARN [ApplicationMaster Launcher] amlauncher.ApplicationMasterLauncher (ApplicationMasterLauncher.java:run(122)) - org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher$LauncherThread interrupted. Returning. 2015-11-22 19:26:50,142 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 47916 2015-11-22 19:26:50,144 INFO [IPC Server listener on 47916] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 47916 2015-11-22 19:26:50,144 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:26:50,144 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 37685 2015-11-22 19:26:50,147 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:26:50,147 INFO [IPC Server listener on 37685] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 37685 2015-11-22 19:26:50,147 INFO [Ping Checker] util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(135)) - NMLivelinessMonitor thread interrupted 2015-11-22 19:26:50,147 ERROR [ResourceManager Event Processor] resourcemanager.ResourceManager (ResourceManager.java:run(691)) - Returning, interrupted : java.lang.InterruptedException 2015-11-22 19:26:50,148 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:serviceStop(141)) - AsyncDispatcher is draining to stop, igonring any new events. 2015-11-22 19:26:50,149 ERROR [Thread[Thread-2328,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(670)) - ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted 2015-11-22 19:26:50,150 INFO [Ping Checker] util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(135)) - AMLivelinessMonitor thread interrupted 2015-11-22 19:26:50,150 INFO [Ping Checker] util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(135)) - org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.ContainerAllocationExpirer thread interrupted 2015-11-22 19:26:50,150 INFO [Ping Checker] util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(135)) - AMLivelinessMonitor thread interrupted 2015-11-22 19:26:50,151 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stop(211)) - Stopping ApplicationHistoryServer metrics system... 2015-11-22 19:26:50,154 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stop(217)) - ApplicationHistoryServer metrics system stopped. 2015-11-22 19:26:50,155 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:shutdown(608)) - ApplicationHistoryServer metrics system shutdown complete. 2015-11-22 19:26:50,155 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:serviceStop(141)) - AsyncDispatcher is draining to stop, igonring any new events. 2015-11-22 19:26:50,155 INFO [main] resourcemanager.ResourceManager (ResourceManager.java:transitionToStandby(1058)) - Transitioned to standby state 2015-11-22 19:26:50,165 INFO [main] distributedshell.TestDistributedShell (TestDistributedShell.java:setupInternal(77)) - Starting up YARN cluster 2015-11-22 19:26:50,175 INFO [main] net.ServerSocketUtil (ServerSocketUtil.java:getPort(52)) - Using port 10021 2015-11-22 19:26:50,186 INFO [main] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2363)) - found resource core-site.xml at file:/home/ubuntu/hadoop-dev/hadoop-common-project/hadoop-common/target/test-classes/core-site.xml 2015-11-22 19:26:50,196 WARN [main] conf.Configuration (Configuration.java:loadProperty(2687)) - core-site.xml:an attempt to override final parameter: hadoop.tmp.dir; Ignoring. 2015-11-22 19:26:50,196 INFO [main] security.Groups (Groups.java:refresh(256)) - clearing userToGroupsMap cache 2015-11-22 19:26:50,197 INFO [main] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2363)) - found resource yarn-site.xml at file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/test-classes/yarn-site.xml 2015-11-22 19:26:50,211 WARN [main] conf.Configuration (Configuration.java:loadProperty(2687)) - yarn-site.xml:an attempt to override final parameter: hadoop.tmp.dir; Ignoring. 2015-11-22 19:26:50,212 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.RMFatalEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMFatalEventDispatcher 2015-11-22 19:26:50,212 INFO [main] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:(75)) - NMTokenKeyRollingInterval: 86400000ms and NMTokenKeyActivationDelay: 900000ms 2015-11-22 19:26:50,212 INFO [main] security.RMContainerTokenSecretManager (RMContainerTokenSecretManager.java:(78)) - ContainerTokenKeyRollingInterval: 86400000ms and ContainerTokenKeyActivationDelay: 900000ms 2015-11-22 19:26:50,213 INFO [main] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:(94)) - AMRMTokenKeyRollingInterval: 86400000ms and AMRMTokenKeyActivationDelay: 900000 ms 2015-11-22 19:26:50,213 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStoreEventType for class org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$ForwardingEventHandler 2015-11-22 19:26:50,213 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.NodesListManagerEventType for class org.apache.hadoop.yarn.server.resourcemanager.NodesListManager 2015-11-22 19:26:50,213 INFO [main] resourcemanager.ResourceManager (ResourceManager.java:createScheduler(295)) - Using Scheduler: org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler 2015-11-22 19:26:50,213 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.SchedulerEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher 2015-11-22 19:26:50,213 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationEventDispatcher 2015-11-22 19:26:50,213 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationAttemptEventDispatcher 2015-11-22 19:26:50,213 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$NodeEventDispatcher 2015-11-22 19:26:50,215 INFO [main] impl.MetricsConfig (MetricsConfig.java:loadFirst(115)) - loaded properties from hadoop-metrics2.properties 2015-11-22 19:26:50,216 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:startTimer(374)) - Scheduled snapshot period at 10 second(s). 2015-11-22 19:26:50,216 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:start(192)) - ResourceManager metrics system started 2015-11-22 19:26:50,217 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.RMAppManagerEventType for class org.apache.hadoop.yarn.server.resourcemanager.RMAppManager 2015-11-22 19:26:50,217 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncherEventType for class org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher 2015-11-22 19:26:50,217 WARN [main] util.MBeans (MBeans.java:register(73)) - Failed to register MBean "Hadoop:service=ResourceManager,name=RMNMInfo": Instance already exists. 2015-11-22 19:26:50,217 INFO [main] resourcemanager.RMNMInfo (RMNMInfo.java:(63)) - Registered RMNMInfo MBean 2015-11-22 19:26:50,218 INFO [main] util.HostsFileReader (HostsFileReader.java:refresh(132)) - Refreshing hosts (include/exclude) list 2015-11-22 19:26:50,218 INFO [main] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2363)) - found resource capacity-scheduler.xml at jar:file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/target/hadoop-yarn-server-tests-2.8.0-SNAPSHOT-tests.jar!/capacity-scheduler.xml 2015-11-22 19:26:50,230 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(647)) - max alloc mb per queue for root is undefined 2015-11-22 19:26:50,230 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(651)) - max alloc vcore per queue for root is undefined 2015-11-22 19:26:50,230 INFO [main] capacity.ParentQueue (ParentQueue.java:setupQueueConfigs(125)) - root, capacity=1.0, asboluteCapacity=1.0, maxCapacity=1.0, asboluteMaxCapacity=1.0, state=RUNNING, acls=ADMINISTER_QUEUE:*SUBMIT_APP:*, labels=*, , reservationsContinueLooking=true 2015-11-22 19:26:50,230 INFO [main] capacity.ParentQueue (ParentQueue.java:(104)) - Initialized parent-queue root name=root, fullname=root 2015-11-22 19:26:50,231 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(647)) - max alloc mb per queue for root.default is undefined 2015-11-22 19:26:50,231 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(651)) - max alloc vcore per queue for root.default is undefined 2015-11-22 19:26:50,232 INFO [main] capacity.LeafQueue (LeafQueue.java:setupQueueConfigs(217)) - Initializing default capacity = 1.0 [= (float) configuredCapacity / 100 ] asboluteCapacity = 1.0 [= parentAbsoluteCapacity * capacity ] maxCapacity = 1.0 [= configuredMaxCapacity ] absoluteMaxCapacity = 1.0 [= 1.0 maximumCapacity undefined, (parentAbsoluteMaxCapacity * maximumCapacity) / 100 otherwise ] userLimit = 100 [= configuredUserLimit ] userLimitFactor = 1.0 [= configuredUserLimitFactor ] maxApplications = 10000 [= configuredMaximumSystemApplicationsPerQueue or (int)(configuredMaximumSystemApplications * absoluteCapacity)] maxApplicationsPerUser = 10000 [= (int)(maxApplications * (userLimit / 100.0f) * userLimitFactor) ] usedCapacity = 0.0 [= usedResourcesMemory / (clusterResourceMemory * absoluteCapacity)] absoluteUsedCapacity = 0.0 [= usedResourcesMemory / clusterResourceMemory] maxAMResourcePerQueuePercent = 0.1 [= configuredMaximumAMResourcePercent ] minimumAllocationFactor = 0.984375 [= (float)(maximumAllocationMemory - minimumAllocationMemory) / maximumAllocationMemory ] maximumAllocation = [= configuredMaxAllocation ] numContainers = 0 [= currentNumContainers ] state = RUNNING [= configuredState ] acls = ADMINISTER_QUEUE:*SUBMIT_APP:* [= configuredAcls ] nodeLocalityDelay = -1 labels=*, nodeLocalityDelay = -1 reservationsContinueLooking = true preemptionDisabled = true defaultAppPriorityPerQueue = 0 2015-11-22 19:26:50,232 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:parseQueue(657)) - Initialized queue: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=0, numContainers=0 2015-11-22 19:26:50,232 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:parseQueue(657)) - Initialized queue: root: numChildQueue= 1, capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.0, numApps=0, numContainers=0 2015-11-22 19:26:50,232 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:initializeQueues(501)) - Initialized root queue root: numChildQueue= 1, capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.0, numApps=0, numContainers=0 2015-11-22 19:26:50,233 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:getUserGroupMappingPlacementRule(449)) - Initialized queue mappings, override: false 2015-11-22 19:26:50,233 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:initScheduler(314)) - Initialized CapacityScheduler with calculator=class org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator, minimumAllocation=<>, maximumAllocation=<>, asynchronousScheduling=false, asyncScheduleInterval=5ms 2015-11-22 19:26:50,233 INFO [main] metrics.SystemMetricsPublisher (SystemMetricsPublisher.java:serviceInit(95)) - YARN system metrics publishing service is not enabled 2015-11-22 19:26:50,233 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEventType for class org.apache.hadoop.yarn.server.MiniYARNCluster$1 2015-11-22 19:26:50,234 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:prepareDirs(558)) - Created localDir in /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0 2015-11-22 19:26:50,234 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:prepareDirs(558)) - Created logDir in /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-logDir-nm-0_0 2015-11-22 19:26:50,234 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceInit(540)) - Starting NM: 0 2015-11-22 19:26:50,243 INFO [main] nodemanager.NodeManager (NodeManager.java:getNodeHealthScriptRunner(255)) - Node Manager health check script is not available or doesn't have execute permission, so not starting the node health script runner. 2015-11-22 19:26:50,244 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ContainerEventDispatcher 2015-11-22 19:26:50,244 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ApplicationEventDispatcher 2015-11-22 19:26:50,244 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizationEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService 2015-11-22 19:26:50,244 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServicesEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices 2015-11-22 19:26:50,244 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl 2015-11-22 19:26:50,244 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncherEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncher 2015-11-22 19:26:50,244 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.ContainerManagerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl 2015-11-22 19:26:50,245 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.NodeManagerEventType for class org.apache.hadoop.yarn.server.MiniYARNCluster$ShortCircuitedNodeManager 2015-11-22 19:26:50,245 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:init(159)) - NodeManager metrics system started (again) 2015-11-22 19:26:50,246 INFO [main] nodemanager.NodeResourceMonitorImpl (NodeResourceMonitorImpl.java:serviceInit(76)) - Using ResourceCalculatorPlugin : org.apache.hadoop.yarn.util.ResourceCalculatorPlugin@41c2284a 2015-11-22 19:26:50,246 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.event.LogHandlerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.NonAggregatingLogHandler 2015-11-22 19:26:50,246 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.sharedcache.SharedCacheUploadEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.sharedcache.SharedCacheUploadService 2015-11-22 19:26:50,246 INFO [main] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:serviceInit(261)) - AMRMProxyService is disabled 2015-11-22 19:26:50,246 INFO [main] localizer.ResourceLocalizationService (ResourceLocalizationService.java:validateConf(224)) - per directory file limit = 8192 2015-11-22 19:26:50,257 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerTracker 2015-11-22 19:26:50,257 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(116)) - Using ResourceCalculatorPlugin : org.apache.hadoop.yarn.util.ResourceCalculatorPlugin@53aad5d5 2015-11-22 19:26:50,257 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(121)) - Using ResourceCalculatorProcessTree : null 2015-11-22 19:26:50,257 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(158)) - Physical memory check enabled: false 2015-11-22 19:26:50,257 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(159)) - Virtual memory check enabled: false 2015-11-22 19:26:50,258 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:isEnabled(206)) - Neither virutal-memory nor physical-memory monitoring is needed. Not running the monitor-thread 2015-11-22 19:26:50,258 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(162)) - ContainersMonitor enabled: false 2015-11-22 19:26:50,258 INFO [main] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:serviceInit(175)) - Nodemanager resources: memory set to 4096MB. 2015-11-22 19:26:50,258 INFO [main] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:serviceInit(176)) - Nodemanager resources: vcores set to 8. 2015-11-22 19:26:50,258 INFO [main] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:serviceInit(208)) - Initialized nodemanager with : physical-memory=4096 virtual-memory=8602 virtual-cores=8 2015-11-22 19:26:50,258 WARN [main] applicationhistoryservice.ApplicationHistoryServer (ApplicationHistoryServer.java:createApplicationHistoryManager(218)) - The filesystem based application history store is deprecated. 2015-11-22 19:26:50,258 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:init(159)) - ApplicationHistoryServer metrics system started (again) 2015-11-22 19:26:50,259 INFO [main] applicationhistoryservice.ApplicationHistoryManagerImpl (ApplicationHistoryManagerImpl.java:serviceInit(61)) - ApplicationHistory Init 2015-11-22 19:26:50,259 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:startResourceManager(326)) - Waiting for RM to start... 2015-11-22 19:26:50,259 INFO [RM-0] resourcemanager.ResourceManager (ResourceManager.java:transitionToActive(1024)) - Transitioning to active state 2015-11-22 19:26:50,260 INFO [RM-0] recovery.RMStateStore (RMStateStore.java:transition(482)) - Updating AMRMToken 2015-11-22 19:26:50,260 INFO [RM-0] security.RMContainerTokenSecretManager (RMContainerTokenSecretManager.java:rollMasterKey(106)) - Rolling master-key for container-tokens 2015-11-22 19:26:50,260 INFO [RM-0] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:rollMasterKey(95)) - Rolling master-key for nm-tokens 2015-11-22 19:26:50,261 INFO [RM-0] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:26:50,261 INFO [RM-0] security.RMDelegationTokenSecretManager (RMDelegationTokenSecretManager.java:storeNewMasterKey(87)) - storing master key with keyID 1 2015-11-22 19:26:50,261 INFO [RM-0] recovery.RMStateStore (RMStateStore.java:transition(433)) - Storing RMDTMasterKey. 2015-11-22 19:26:50,261 INFO [Thread[Thread-2705,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2015-11-22 19:26:50,261 INFO [Thread[Thread-2705,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:26:50,262 INFO [Thread[Thread-2705,5,main]] security.RMDelegationTokenSecretManager (RMDelegationTokenSecretManager.java:storeNewMasterKey(87)) - storing master key with keyID 2 2015-11-22 19:26:50,263 INFO [Thread[Thread-2705,5,main]] recovery.RMStateStore (RMStateStore.java:transition(433)) - Storing RMDTMasterKey. 2015-11-22 19:26:50,263 INFO [RM-0] nodelabels.CommonNodeLabelsManager (CommonNodeLabelsManager.java:addToCluserNodeLabels(309)) - Add labels: [] 2015-11-22 19:26:50,263 INFO [RM-0] nodelabels.CommonNodeLabelsManager (CommonNodeLabelsManager.java:internalUpdateLabelsOnNodes(638)) - REPLACE labels on nodes: 2015-11-22 19:26:50,263 INFO [RM-0] nodelabels.CommonNodeLabelsManager (CommonNodeLabelsManager.java:internalUpdateLabelsOnNodes(640)) - NM=ip-172-31-20-42.ap-northeast-1.compute.internal:58694, labels=[x] 2015-11-22 19:26:50,264 INFO [RM-0] nodelabels.FileSystemNodeLabelsStore (FileSystemNodeLabelsStore.java:recover(279)) - Finished write mirror at:file:/tmp/hadoop-yarn-ubuntu/node-labels/nodelabel.mirror 2015-11-22 19:26:50,264 INFO [RM-0] nodelabels.FileSystemNodeLabelsStore (FileSystemNodeLabelsStore.java:recover(280)) - Finished create editlog file at:file:/tmp/hadoop-yarn-ubuntu/node-labels/nodelabel.editlog 2015-11-22 19:26:50,264 INFO [RM-0] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.nodelabels.event.NodeLabelsStoreEventType for class org.apache.hadoop.yarn.nodelabels.CommonNodeLabelsManager$ForwardingEventHandler 2015-11-22 19:26:50,265 INFO [RM-0] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:26:50,266 INFO [Socket Reader #1 for port 57053] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 57053 2015-11-22 19:26:50,267 INFO [RM-0] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.server.api.ResourceTrackerPB to the server 2015-11-22 19:26:50,267 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:26:50,267 INFO [IPC Server listener on 57053] ipc.Server (Server.java:run(794)) - IPC Server listener on 57053: starting 2015-11-22 19:26:50,271 INFO [RM-0] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:26:50,272 INFO [Socket Reader #1 for port 43231] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 43231 2015-11-22 19:26:50,273 INFO [RM-0] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB to the server 2015-11-22 19:26:50,273 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:26:50,273 INFO [IPC Server listener on 43231] ipc.Server (Server.java:run(794)) - IPC Server listener on 43231: starting 2015-11-22 19:26:50,278 INFO [RM-0] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:26:50,278 INFO [Socket Reader #1 for port 54235] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 54235 2015-11-22 19:26:50,279 INFO [RM-0] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.api.ApplicationClientProtocolPB to the server 2015-11-22 19:26:50,280 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:26:50,285 INFO [IPC Server listener on 54235] ipc.Server (Server.java:run(794)) - IPC Server listener on 54235: starting 2015-11-22 19:26:50,289 INFO [RM-0] resourcemanager.ResourceManager (ResourceManager.java:transitionToActive(1040)) - Transitioned to active state 2015-11-22 19:26:50,291 INFO [RM-0] server.AuthenticationFilter (AuthenticationFilter.java:constructSecretProvider(304)) - Unable to initialize FileSignerSecretProvider, falling back to use random secrets. 2015-11-22 19:26:50,292 INFO [RM-0] http.HttpRequestLog (HttpRequestLog.java:getRequestLog(80)) - Http request log for http.requests.resourcemanager is not defined 2015-11-22 19:26:50,292 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter) 2015-11-22 19:26:50,293 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context cluster 2015-11-22 19:26:50,293 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context logs 2015-11-22 19:26:50,293 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context static 2015-11-22 19:26:50,293 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context cluster 2015-11-22 19:26:50,293 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2015-11-22 19:26:50,293 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2015-11-22 19:26:50,294 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'Timeline Authentication Filter' (class=org.apache.hadoop.yarn.server.timeline.security.TimelineAuthenticationFilter) 2015-11-22 19:26:50,294 INFO [RM-0] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /cluster/* 2015-11-22 19:26:50,294 INFO [RM-0] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /ws/* 2015-11-22 19:26:50,298 INFO [RM-0] webapp.WebApps (WebApps.java:build(288)) - Registered webapp guice modules 2015-11-22 19:26:50,298 INFO [RM-0] http.HttpServer2 (HttpServer2.java:openListeners(917)) - Jetty bound to port 43440 2015-11-22 19:26:50,298 INFO [RM-0] mortbay.log (Slf4jLog.java:info(67)) - jetty-6.1.26 2015-11-22 19:26:50,309 INFO [RM-0] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:26:50,309 INFO [Thread[Thread-2874,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2015-11-22 19:26:50,310 INFO [Thread[Thread-2874,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:26:50,311 INFO [RM-0] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:26:50,311 INFO [Thread[Thread-2876,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) Nov 22, 2015 7:26:50 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.resourcemanager.webapp.JAXBContextResolver as a provider class Nov 22, 2015 7:26:50 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices as a root resource class Nov 22, 2015 7:26:50 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a provider class Nov 22, 2015 7:26:50 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate INFO: Initiating Jersey application, version 'Jersey: 1.9 09/02/2011 11:17 AM' Nov 22, 2015 7:26:50 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.resourcemanager.webapp.JAXBContextResolver to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:26:50 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.webapp.GenericExceptionHandler to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:26:50 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices to GuiceManagedComponentProvider with the scope "Singleton" 2015-11-22 19:26:50,482 INFO [RM-0] mortbay.log (Slf4jLog.java:info(67)) - Started HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:43440 2015-11-22 19:26:50,484 INFO [RM-0] webapp.WebApps (WebApps.java:start(307)) - Web app cluster started at 43440 2015-11-22 19:26:50,485 INFO [RM-0] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:26:50,486 INFO [Socket Reader #1 for port 40856] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 40856 2015-11-22 19:26:50,487 INFO [RM-0] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.server.api.ResourceManagerAdministrationProtocolPB to the server 2015-11-22 19:26:50,487 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:26:50,493 INFO [IPC Server listener on 40856] ipc.Server (Server.java:run(794)) - IPC Server listener on 40856: starting 2015-11-22 19:26:51,760 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:startResourceManager(339)) - MiniYARN ResourceManager address: ip-172-31-20-42.ap-northeast-1.compute.internal:54235 2015-11-22 19:26:51,760 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:startResourceManager(341)) - MiniYARN ResourceManager web address: ip-172-31-20-42.ap-northeast-1.compute.internal:43440 2015-11-22 19:26:51,760 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(459)) - MiniYARN ResourceManager address: ip-172-31-20-42.ap-northeast-1.compute.internal:54235 2015-11-22 19:26:51,760 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(461)) - MiniYARN ResourceManager web address: ip-172-31-20-42.ap-northeast-1.compute.internal:43440 2015-11-22 19:26:51,761 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(575)) - Waiting for NM 0 to start... 2015-11-22 19:26:51,763 INFO [Thread-2883] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:26:51,764 INFO [Socket Reader #1 for port 57515] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 57515 2015-11-22 19:26:51,765 INFO [Thread-2883] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.api.ContainerManagementProtocolPB to the server 2015-11-22 19:26:51,765 INFO [Thread-2883] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:serviceStart(464)) - Blocking new container-requests as container manager rpc server is still starting. 2015-11-22 19:26:51,765 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:26:51,766 INFO [IPC Server listener on 57515] ipc.Server (Server.java:run(794)) - IPC Server listener on 57515: starting 2015-11-22 19:26:51,768 INFO [Thread-2883] security.NMContainerTokenSecretManager (NMContainerTokenSecretManager.java:setNodeId(260)) - Updating node address : ip-172-31-20-42.ap-northeast-1.compute.internal:57515 2015-11-22 19:26:51,768 INFO [Thread-2883] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:26:51,769 INFO [Socket Reader #1 for port 40894] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 40894 2015-11-22 19:26:51,770 INFO [Thread-2883] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB to the server 2015-11-22 19:26:51,770 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:26:51,770 INFO [IPC Server listener on 40894] ipc.Server (Server.java:run(794)) - IPC Server listener on 40894: starting 2015-11-22 19:26:51,770 INFO [Thread-2883] localizer.ResourceLocalizationService (ResourceLocalizationService.java:serviceStart(363)) - Localizer started on port 40894 2015-11-22 19:26:51,782 INFO [Thread-2883] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:serviceStart(509)) - ContainerManager started at /172.31.20.42:57515 2015-11-22 19:26:51,782 INFO [Thread-2883] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:serviceStart(510)) - ContainerManager bound to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:0 2015-11-22 19:26:51,782 INFO [Thread-2883] webapp.WebServer (WebServer.java:serviceStart(71)) - Instantiating NMWebApp at ip-172-31-20-42.ap-northeast-1.compute.internal:0 2015-11-22 19:26:51,782 INFO [Thread-2883] server.AuthenticationFilter (AuthenticationFilter.java:constructSecretProvider(304)) - Unable to initialize FileSignerSecretProvider, falling back to use random secrets. 2015-11-22 19:26:51,783 INFO [Thread-2883] http.HttpRequestLog (HttpRequestLog.java:getRequestLog(80)) - Http request log for http.requests.nodemanager is not defined 2015-11-22 19:26:51,783 INFO [Thread-2883] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter) 2015-11-22 19:26:51,784 INFO [Thread-2883] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context node 2015-11-22 19:26:51,784 INFO [Thread-2883] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context static 2015-11-22 19:26:51,784 INFO [Thread-2883] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context logs 2015-11-22 19:26:51,784 INFO [Thread-2883] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context node 2015-11-22 19:26:51,784 INFO [Thread-2883] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2015-11-22 19:26:51,784 INFO [Thread-2883] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2015-11-22 19:26:51,785 INFO [Thread-2883] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'Timeline Authentication Filter' (class=org.apache.hadoop.yarn.server.timeline.security.TimelineAuthenticationFilter) 2015-11-22 19:26:51,785 INFO [Thread-2883] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /node/* 2015-11-22 19:26:51,785 INFO [Thread-2883] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /ws/* Nov 22, 2015 7:26:51 PM com.google.inject.servlet.GuiceFilter setPipeline WARNING: Multiple Servlet injectors detected. This is a warning indicating that you have more than one GuiceFilter running in your web application. If this is deliberate, you may safely ignore this message. If this is NOT deliberate however, your application may not work as expected. 2015-11-22 19:26:51,789 INFO [Thread-2883] webapp.WebApps (WebApps.java:build(288)) - Registered webapp guice modules 2015-11-22 19:26:51,789 INFO [Thread-2883] http.HttpServer2 (HttpServer2.java:openListeners(917)) - Jetty bound to port 42079 2015-11-22 19:26:51,790 INFO [Thread-2883] mortbay.log (Slf4jLog.java:info(67)) - jetty-6.1.26 2015-11-22 19:26:51,800 INFO [Thread-2883] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:26:51,800 INFO [Thread[Thread-2924,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2015-11-22 19:26:51,801 INFO [Thread[Thread-2924,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:26:51,801 INFO [Thread-2883] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:26:51,802 INFO [Thread[Thread-2926,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) Nov 22, 2015 7:26:51 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.nodemanager.webapp.NMWebServices as a root resource class Nov 22, 2015 7:26:51 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a provider class Nov 22, 2015 7:26:51 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.nodemanager.webapp.JAXBContextResolver as a provider class Nov 22, 2015 7:26:51 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate INFO: Initiating Jersey application, version 'Jersey: 1.9 09/02/2011 11:17 AM' Nov 22, 2015 7:26:51 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.nodemanager.webapp.JAXBContextResolver to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:26:51 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.webapp.GenericExceptionHandler to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:26:51 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.nodemanager.webapp.NMWebServices to GuiceManagedComponentProvider with the scope "Singleton" 2015-11-22 19:26:51,893 INFO [Thread-2883] mortbay.log (Slf4jLog.java:info(67)) - Started HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:42079 2015-11-22 19:26:51,894 INFO [Thread-2883] webapp.WebApps (WebApps.java:start(307)) - Web app node started at 42079 2015-11-22 19:26:51,894 INFO [Thread-2883] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:serviceStart(222)) - Node ID assigned is : ip-172-31-20-42.ap-northeast-1.compute.internal:57515 2015-11-22 19:26:51,894 INFO [Thread-2883] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNMContainerStatuses(550)) - Sending out 0 NM container statuses: [] 2015-11-22 19:26:51,894 INFO [Thread-2883] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:registerWithRM(330)) - Registering with RM using containers :[] 2015-11-22 19:26:51,895 INFO [Thread-2883] resourcemanager.ResourceTrackerService (ResourceTrackerService.java:registerNodeManager(375)) - NodeManager from node ip-172-31-20-42.ap-northeast-1.compute.internal(cmPort: 57515 httpPort: 42079) registered with capability: , assigned nodeId ip-172-31-20-42.ap-northeast-1.compute.internal:57515 2015-11-22 19:26:51,895 INFO [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(566)) - ip-172-31-20-42.ap-northeast-1.compute.internal:57515 Node Transitioned from NEW to RUNNING 2015-11-22 19:26:51,895 INFO [Thread-2883] security.NMContainerTokenSecretManager (NMContainerTokenSecretManager.java:setMasterKey(138)) - Rolling master-key for container-tokens, got key with id 1764259912 2015-11-22 19:26:51,896 INFO [Thread-2883] security.NMTokenSecretManagerInNM (NMTokenSecretManagerInNM.java:setMasterKey(135)) - Rolling master-key for container-tokens, got key with id -34002137 2015-11-22 19:26:51,896 INFO [Thread-2883] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:registerWithRM(391)) - Registered with ResourceManager as ip-172-31-20-42.ap-northeast-1.compute.internal:57515 with total resource of 2015-11-22 19:26:51,896 INFO [Thread-2883] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:registerWithRM(392)) - Notifying ContainerManager to unblock new container-requests 2015-11-22 19:26:51,896 INFO [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:addNode(1427)) - Added node ip-172-31-20-42.ap-northeast-1.compute.internal:57515 clusterResource: 2015-11-22 19:26:52,761 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(728)) - Waiting for Timeline Server to start... 2015-11-22 19:26:52,762 INFO [Thread-2929] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:26:52,762 INFO [Thread[Thread-2931,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2015-11-22 19:26:52,762 INFO [Thread-2929] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:26:52,762 INFO [Thread[Thread-2931,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:26:52,763 INFO [Socket Reader #1 for port 38383] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 38383 2015-11-22 19:26:52,764 INFO [Thread-2929] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.api.ApplicationHistoryProtocolPB to the server 2015-11-22 19:26:52,764 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:26:52,765 INFO [IPC Server listener on 38383] ipc.Server (Server.java:run(794)) - IPC Server listener on 38383: starting 2015-11-22 19:26:52,767 INFO [Thread-2929] applicationhistoryservice.ApplicationHistoryClientService (ApplicationHistoryClientService.java:serviceStart(111)) - Instantiated ApplicationHistoryClientService at ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:38383 2015-11-22 19:26:52,767 INFO [Thread-2929] applicationhistoryservice.ApplicationHistoryManagerImpl (ApplicationHistoryManagerImpl.java:serviceStart(71)) - Starting ApplicationHistory 2015-11-22 19:26:52,768 INFO [Thread-2929] server.AuthenticationFilter (AuthenticationFilter.java:constructSecretProvider(304)) - Unable to initialize FileSignerSecretProvider, falling back to use random secrets. 2015-11-22 19:26:52,769 INFO [Thread-2929] http.HttpRequestLog (HttpRequestLog.java:getRequestLog(80)) - Http request log for http.requests.applicationhistory is not defined 2015-11-22 19:26:52,769 INFO [Thread-2929] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter) 2015-11-22 19:26:52,770 INFO [Thread-2929] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context applicationhistory 2015-11-22 19:26:52,770 INFO [Thread-2929] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context static 2015-11-22 19:26:52,770 INFO [Thread-2929] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context logs 2015-11-22 19:26:52,770 INFO [Thread-2929] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context applicationhistory 2015-11-22 19:26:52,770 INFO [Thread-2929] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2015-11-22 19:26:52,770 INFO [Thread-2929] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2015-11-22 19:26:52,771 INFO [Thread-2929] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'Timeline Authentication Filter' (class=org.apache.hadoop.yarn.server.timeline.security.TimelineAuthenticationFilter) 2015-11-22 19:26:52,772 INFO [Thread-2929] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /applicationhistory/* 2015-11-22 19:26:52,772 INFO [Thread-2929] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /ws/* Nov 22, 2015 7:26:52 PM com.google.inject.servlet.GuiceFilter setPipeline WARNING: Multiple Servlet injectors detected. This is a warning indicating that you have more than one GuiceFilter running in your web application. If this is deliberate, you may safely ignore this message. If this is NOT deliberate however, your application may not work as expected. 2015-11-22 19:26:52,778 INFO [Thread-2929] webapp.WebApps (WebApps.java:build(288)) - Registered webapp guice modules 2015-11-22 19:26:52,778 INFO [Thread-2929] http.HttpServer2 (HttpServer2.java:openListeners(917)) - Jetty bound to port 52032 2015-11-22 19:26:52,778 INFO [Thread-2929] mortbay.log (Slf4jLog.java:info(67)) - jetty-6.1.26 2015-11-22 19:26:52,791 INFO [Thread-2929] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:26:52,792 INFO [Thread[Thread-2949,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2015-11-22 19:26:52,793 INFO [Thread[Thread-2949,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:26:52,794 INFO [Thread-2929] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:26:52,794 INFO [Thread[Thread-2951,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) Nov 22, 2015 7:26:52 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider as a provider class Nov 22, 2015 7:26:52 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.AHSWebServices as a root resource class Nov 22, 2015 7:26:52 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.timeline.webapp.TimelineWebServices as a root resource class Nov 22, 2015 7:26:52 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a provider class Nov 22, 2015 7:26:52 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate INFO: Initiating Jersey application, version 'Jersey: 1.9 09/02/2011 11:17 AM' Nov 22, 2015 7:26:52 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.webapp.GenericExceptionHandler to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:26:52 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:26:52 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.AHSWebServices to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:26:52 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.timeline.webapp.TimelineWebServices to GuiceManagedComponentProvider with the scope "Singleton" 2015-11-22 19:26:52,865 INFO [Thread-2929] mortbay.log (Slf4jLog.java:info(67)) - Started HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:52032 2015-11-22 19:26:52,866 INFO [Thread-2929] applicationhistoryservice.ApplicationHistoryServer (ApplicationHistoryServer.java:startWebApp(342)) - Instantiating AHSWebApp at 52032 2015-11-22 19:26:54,261 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(739)) - MiniYARN ApplicationHistoryServer address: ip-172-31-20-42.ap-northeast-1.compute.internal:38383 2015-11-22 19:26:54,261 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(741)) - MiniYARN ApplicationHistoryServer web address: ip-172-31-20-42.ap-northeast-1.compute.internal:52032 /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/org.apache.hadoop.yarn.applications.distributedshell.TestDistributedShell/tmpDir/custom_script.sh 2015-11-22 19:26:56,291 INFO [Thread-2953] distributedshell.TestDistributedShell (TestDistributedShell.java:testDSShellWithShellScript(613)) - Initializing DS Client 2015-11-22 19:26:56,341 INFO [Thread-2953] impl.TimelineClientImpl (TimelineClientImpl.java:serviceInit(296)) - Timeline service address: http://ip-172-31-20-42.ap-northeast-1.compute.internal:52032/ws/v1/timeline/ 2015-11-22 19:26:56,341 INFO [Thread-2953] distributedshell.TestDistributedShell (TestDistributedShell.java:testDSShellWithShellScript(618)) - Running DS Client 2015-11-22 19:26:56,341 INFO [Thread-2953] distributedshell.Client (Client.java:run(443)) - Running Client 2015-11-22 19:26:56,354 INFO [Thread-2953] client.RMProxy (RMProxy.java:createRMProxy(98)) - Connecting to ResourceManager at ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:54235 2015-11-22 19:26:56,358 INFO [Thread-2953] distributedshell.Client (Client.java:run(447)) - Got Cluster metric info from ASM, numNodeManagers=1 2015-11-22 19:26:56,360 INFO [Thread-2953] distributedshell.Client (Client.java:run(452)) - Got Cluster node info from ASM 2015-11-22 19:26:56,360 INFO [Thread-2953] distributedshell.Client (Client.java:run(454)) - Got node report from ASM for, nodeId=ip-172-31-20-42.ap-northeast-1.compute.internal:57515, nodeAddress=ip-172-31-20-42.ap-northeast-1.compute.internal:42079, nodeRackName=/default-rack, nodeNumContainers=0 2015-11-22 19:26:56,361 INFO [Thread-2953] distributedshell.Client (Client.java:run(462)) - Queue info, queueName=default, queueCurrentCapacity=0.0, queueMaxCapacity=1.0, queueApplicationCount=0, queueChildQueueCount=0 2015-11-22 19:26:56,362 INFO [Thread-2953] distributedshell.Client (Client.java:run(472)) - User ACL Info for Queue, queueName=root, userAcl=SUBMIT_APPLICATIONS 2015-11-22 19:26:56,362 INFO [Thread-2953] distributedshell.Client (Client.java:run(472)) - User ACL Info for Queue, queueName=root, userAcl=ADMINISTER_QUEUE 2015-11-22 19:26:56,362 INFO [Thread-2953] distributedshell.Client (Client.java:run(472)) - User ACL Info for Queue, queueName=default, userAcl=SUBMIT_APPLICATIONS 2015-11-22 19:26:56,362 INFO [Thread-2953] distributedshell.Client (Client.java:run(472)) - User ACL Info for Queue, queueName=default, userAcl=ADMINISTER_QUEUE 2015-11-22 19:26:56,362 INFO [IPC Server handler 5 on 54235] resourcemanager.ClientRMService (ClientRMService.java:getNewApplicationId(305)) - Allocated new applicationId: 1 2015-11-22 19:26:56,363 INFO [Thread-2953] distributedshell.Client (Client.java:run(491)) - Max mem capability of resources in this cluster 4096 2015-11-22 19:26:56,363 INFO [Thread-2953] distributedshell.Client (Client.java:run(502)) - Max virtual cores capability of resources in this cluster 4 2015-11-22 19:26:56,363 INFO [Thread-2953] distributedshell.Client (Client.java:run(528)) - Copy App Master jar from local filesystem and add to local environment 2015-11-22 19:26:56,367 INFO [Thread-2953] distributedshell.Client (Client.java:run(577)) - Set the environment for the application master 2015-11-22 19:26:56,367 INFO [Thread-2953] distributedshell.Client (Client.java:run(619)) - Setting up app master command 2015-11-22 19:26:56,368 INFO [Thread-2953] distributedshell.Client (Client.java:run(650)) - Completed setting up app master command {{JAVA_HOME}}/bin/java -Xmx512m org.apache.hadoop.yarn.applications.distributedshell.ApplicationMaster --container_memory 128 --container_vcores 1 --num_containers 1 --priority 0 1>/AppMaster.stdout 2>/AppMaster.stderr 2015-11-22 19:26:56,368 INFO [Thread-2953] distributedshell.Client (Client.java:run(706)) - Submitting application to ASM 2015-11-22 19:26:56,372 INFO [IPC Server handler 4 on 54235] capacity.CapacityScheduler (CapacityScheduler.java:checkAndGetApplicationPriority(1941)) - Priority '0' is acceptable in queue : default for application: application_1448220410260_0001 for the user: ubuntu 2015-11-22 19:26:56,372 WARN [IPC Server handler 4 on 54235] rmapp.RMAppImpl (RMAppImpl.java:(420)) - The specific max attempts: 0 for application: 1 is invalid, because it is out of the range [1, 2]. Use the global max attempts instead. 2015-11-22 19:26:56,372 INFO [IPC Server handler 4 on 54235] resourcemanager.ClientRMService (ClientRMService.java:submitApplication(602)) - Application with id 1 submitted by user ubuntu 2015-11-22 19:26:56,372 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:transition(1079)) - Storing application with id application_1448220410260_0001 2015-11-22 19:26:56,372 INFO [IPC Server handler 4 on 54235] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(154)) - USER=ubuntu IP=172.31.20.42 OPERATION=Submit Application Request TARGET=ClientRMService RESULT=SUCCESS APPID=application_1448220410260_0001 2015-11-22 19:26:56,373 INFO [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:transition(208)) - Storing info for app: application_1448220410260_0001 2015-11-22 19:26:56,372 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220410260_0001 State change from NEW to NEW_SAVING 2015-11-22 19:26:56,373 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220410260_0001 State change from NEW_SAVING to SUBMITTED 2015-11-22 19:26:56,373 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:addApplication(349)) - Application added - appId: application_1448220410260_0001 user: ubuntu leaf-queue of parent: root #applications: 1 2015-11-22 19:26:56,373 INFO [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:addApplication(765)) - Accepted application application_1448220410260_0001 from user: ubuntu, in queue: default 2015-11-22 19:26:56,373 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220410260_0001 State change from SUBMITTED to ACCEPTED 2015-11-22 19:26:56,374 INFO [AsyncDispatcher event handler] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:registerAppAttempt(661)) - Registering app attempt : appattempt_1448220410260_0001_000001 2015-11-22 19:26:56,374 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220410260_0001_000001 State change from NEW to SUBMITTED 2015-11-22 19:26:56,374 INFO [Thread-2953] impl.YarnClientImpl (YarnClientImpl.java:submitApplication(279)) - Submitted application application_1448220410260_0001 2015-11-22 19:26:56,374 INFO [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:activateApplications(705)) - Application application_1448220410260_0001 from user: ubuntu activated in queue: default 2015-11-22 19:26:56,374 INFO [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:addApplicationAttempt(720)) - Application added - appId: application_1448220410260_0001 user: ubuntu, leaf-queue: default #user-pending-applications: 0 #user-active-applications: 1 #queue-pending-applications: 0 #queue-active-applications: 1 2015-11-22 19:26:56,375 INFO [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:addApplicationAttempt(805)) - Added Application Attempt appattempt_1448220410260_0001_000001 to scheduler from user ubuntu in queue default 2015-11-22 19:26:56,376 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220410260_0001_000001 State change from SUBMITTED to SCHEDULED 2015-11-22 19:26:56,902 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220410260_0001_01_000001 Container Transitioned from NEW to ALLOCATED 2015-11-22 19:26:56,902 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220410260_0001 CONTAINERID=container_1448220410260_0001_01_000001 2015-11-22 19:26:56,902 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(154)) - Assigned container container_1448220410260_0001_01_000001 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:57515, which has 1 containers, used and available after allocation 2015-11-22 19:26:56,902 INFO [ResourceManager Event Processor] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(89)) - assignedContainer application attempt=appattempt_1448220410260_0001_000001 container=container_1448220410260_0001_01_000001 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@7f595d61 clusterResource= type=OFF_SWITCH 2015-11-22 19:26:56,903 INFO [AsyncDispatcher event handler] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:createAndGetNMToken(200)) - Sending NMToken for nodeId : ip-172-31-20-42.ap-northeast-1.compute.internal:57515 for container : container_1448220410260_0001_01_000001 2015-11-22 19:26:56,905 INFO [AsyncDispatcher event handler] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220410260_0001_01_000001 Container Transitioned from ALLOCATED to ACQUIRED 2015-11-22 19:26:56,905 INFO [AsyncDispatcher event handler] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:clearNodeSetForAttempt(146)) - Clear node set for appattempt_1448220410260_0001_000001 2015-11-22 19:26:56,905 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(589)) - Re-sorting assigned queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.125, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 2015-11-22 19:26:56,905 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:storeAttempt(2026)) - Storing attempt: AppId: application_1448220410260_0001 AttemptId: appattempt_1448220410260_0001_000001 MasterContainer: Container: [ContainerId: container_1448220410260_0001_01_000001, NodeId: ip-172-31-20-42.ap-northeast-1.compute.internal:57515, NodeHttpAddress: ip-172-31-20-42.ap-northeast-1.compute.internal:42079, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.31.20.42:57515 }, ] 2015-11-22 19:26:56,905 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(461)) - assignedContainer queue=root usedCapacity=0.125 absoluteUsedCapacity=0.125 used= cluster= 2015-11-22 19:26:56,905 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220410260_0001_000001 State change from SCHEDULED to ALLOCATED_SAVING 2015-11-22 19:26:56,906 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220410260_0001_000001 State change from ALLOCATED_SAVING to ALLOCATED 2015-11-22 19:26:56,906 INFO [ApplicationMasterLauncher #0] amlauncher.AMLauncher (AMLauncher.java:run(254)) - Launching masterappattempt_1448220410260_0001_000001 2015-11-22 19:26:56,909 INFO [ApplicationMasterLauncher #0] amlauncher.AMLauncher (AMLauncher.java:launch(106)) - Setting up container Container: [ContainerId: container_1448220410260_0001_01_000001, NodeId: ip-172-31-20-42.ap-northeast-1.compute.internal:57515, NodeHttpAddress: ip-172-31-20-42.ap-northeast-1.compute.internal:42079, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.31.20.42:57515 }, ] for AM appattempt_1448220410260_0001_000001 2015-11-22 19:26:56,909 INFO [ApplicationMasterLauncher #0] amlauncher.AMLauncher (AMLauncher.java:createAMContainerLaunchContext(188)) - Command to launch container container_1448220410260_0001_01_000001 : {{JAVA_HOME}}/bin/java -Xmx512m org.apache.hadoop.yarn.applications.distributedshell.ApplicationMaster --container_memory 128 --container_vcores 1 --num_containers 1 --priority 0 1>/AppMaster.stdout 2>/AppMaster.stderr 2015-11-22 19:26:56,909 INFO [ApplicationMasterLauncher #0] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:createAndGetAMRMToken(195)) - Create AMRMToken for ApplicationAttempt: appattempt_1448220410260_0001_000001 2015-11-22 19:26:56,909 INFO [ApplicationMasterLauncher #0] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:createPassword(307)) - Creating password for appattempt_1448220410260_0001_000001 2015-11-22 19:26:56,913 INFO [Socket Reader #1 for port 57515] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220410260_0001_000001 (auth:SIMPLE) 2015-11-22 19:26:56,917 INFO [IPC Server handler 3 on 57515] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:startContainerInternal(889)) - Start request for container_1448220410260_0001_01_000001 by user ubuntu 2015-11-22 19:26:56,917 INFO [IPC Server handler 3 on 57515] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:startContainerInternal(930)) - Creating a new application reference for app application_1448220410260_0001 2015-11-22 19:26:56,917 INFO [IPC Server handler 3 on 57515] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu IP=172.31.20.42 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1448220410260_0001 CONTAINERID=container_1448220410260_0001_01_000001 2015-11-22 19:26:56,917 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(463)) - Application application_1448220410260_0001 transitioned from NEW to INITING 2015-11-22 19:26:56,918 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(303)) - Adding container_1448220410260_0001_01_000001 to application application_1448220410260_0001 2015-11-22 19:26:56,918 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(463)) - Application application_1448220410260_0001 transitioned from INITING to RUNNING 2015-11-22 19:26:56,918 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220410260_0001_01_000001 transitioned from NEW to LOCALIZING 2015-11-22 19:26:56,918 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_INIT for appId application_1448220410260_0001 2015-11-22 19:26:56,919 INFO [AsyncDispatcher event handler] localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource file:/home/ubuntu/DistributedShell/application_1448220410260_0001/AppMaster.jar transitioned from INIT to DOWNLOADING 2015-11-22 19:26:56,919 INFO [AsyncDispatcher event handler] localizer.ResourceLocalizationService (ResourceLocalizationService.java:handle(742)) - Created localizer for container_1448220410260_0001_01_000001 2015-11-22 19:26:56,921 INFO [ApplicationMasterLauncher #0] amlauncher.AMLauncher (AMLauncher.java:launch(127)) - Done launching container Container: [ContainerId: container_1448220410260_0001_01_000001, NodeId: ip-172-31-20-42.ap-northeast-1.compute.internal:57515, NodeHttpAddress: ip-172-31-20-42.ap-northeast-1.compute.internal:42079, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.31.20.42:57515 }, ] for AM appattempt_1448220410260_0001_000001 2015-11-22 19:26:56,921 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220410260_0001_000001 State change from ALLOCATED to LAUNCHED 2015-11-22 19:26:56,924 INFO [LocalizerRunner for container_1448220410260_0001_01_000001] localizer.ResourceLocalizationService (ResourceLocalizationService.java:writeCredentials(1221)) - Writing credentials to the nmPrivate file /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/container_1448220410260_0001_01_000001.tokens. Credentials list: 2015-11-22 19:26:56,925 INFO [LocalizerRunner for container_1448220410260_0001_01_000001] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:createUserCacheDirs(632)) - Initializing user ubuntu 2015-11-22 19:26:56,933 INFO [LocalizerRunner for container_1448220410260_0001_01_000001] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:startLocalizer(126)) - Copying from /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/container_1448220410260_0001_01_000001.tokens to /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220410260_0001/container_1448220410260_0001_01_000001.tokens 2015-11-22 19:26:56,933 INFO [LocalizerRunner for container_1448220410260_0001_01_000001] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:startLocalizer(133)) - Localizer CWD set to /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220410260_0001 = file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220410260_0001 2015-11-22 19:26:56,970 INFO [IPC Server handler 2 on 40894] localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource file:/home/ubuntu/DistributedShell/application_1448220410260_0001/AppMaster.jar(->/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220410260_0001/filecache/10/AppMaster.jar) transitioned from DOWNLOADING to LOCALIZED 2015-11-22 19:26:56,970 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220410260_0001_01_000001 transitioned from LOCALIZING to LOCALIZED 2015-11-22 19:26:56,992 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220410260_0001_01_000001 transitioned from LOCALIZED to RUNNING 2015-11-22 19:26:57,009 INFO [ContainersLauncher #0] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:buildCommandExecutor(281)) - launchContainer: [nice, -n, 0, bash, /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220410260_0001/container_1448220410260_0001_01_000001/default_container_executor.sh] 2015-11-22 19:26:57,377 INFO [Thread-2953] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=N/A, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220416372, yarnAppState=ACCEPTED, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:43440/proxy/application_1448220410260_0001/, appUser=ubuntu 2015-11-22 19:26:57,900 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220410260_0001_01_000001 Container Transitioned from ACQUIRED to RUNNING 2015-11-22 19:26:58,378 INFO [Thread-2953] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=N/A, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220416372, yarnAppState=ACCEPTED, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:43440/proxy/application_1448220410260_0001/, appUser=ubuntu 2015-11-22 19:26:59,380 INFO [Thread-2953] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=N/A, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220416372, yarnAppState=ACCEPTED, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:43440/proxy/application_1448220410260_0001/, appUser=ubuntu 2015-11-22 19:26:59,394 INFO [Socket Reader #1 for port 43231] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220410260_0001_000001 (auth:SIMPLE) 2015-11-22 19:26:59,401 INFO [IPC Server handler 1 on 43231] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:registerApplicationMaster(220)) - AM registration appattempt_1448220410260_0001_000001 2015-11-22 19:26:59,401 INFO [IPC Server handler 1 on 43231] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(133)) - USER=ubuntu IP=172.31.20.42 OPERATION=Register App Master TARGET=ApplicationMasterService RESULT=SUCCESS APPID=application_1448220410260_0001 APPATTEMPTID=appattempt_1448220410260_0001_000001 2015-11-22 19:26:59,401 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220410260_0001_000001 State change from LAUNCHED to RUNNING 2015-11-22 19:26:59,402 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220410260_0001 State change from ACCEPTED to RUNNING 2015-11-22 19:27:00,382 INFO [Thread-2953] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220416372, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:43440/proxy/application_1448220410260_0001/, appUser=ubuntu 2015-11-22 19:27:00,902 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220410260_0001_01_000002 Container Transitioned from NEW to ALLOCATED 2015-11-22 19:27:00,903 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220410260_0001 CONTAINERID=container_1448220410260_0001_01_000002 2015-11-22 19:27:00,903 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(154)) - Assigned container container_1448220410260_0001_01_000002 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:57515, which has 2 containers, used and available after allocation 2015-11-22 19:27:00,903 INFO [ResourceManager Event Processor] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(89)) - assignedContainer application attempt=appattempt_1448220410260_0001_000001 container=container_1448220410260_0001_01_000002 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@7f595d61 clusterResource= type=OFF_SWITCH 2015-11-22 19:27:00,903 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(589)) - Re-sorting assigned queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.15625, absoluteUsedCapacity=0.15625, numApps=1, numContainers=2 2015-11-22 19:27:00,903 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(461)) - assignedContainer queue=root usedCapacity=0.15625 absoluteUsedCapacity=0.15625 used= cluster= 2015-11-22 19:27:01,383 INFO [Thread-2953] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220416372, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:43440/proxy/application_1448220410260_0001/, appUser=ubuntu 2015-11-22 19:27:01,446 INFO [IPC Server handler 7 on 43231] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:createAndGetNMToken(200)) - Sending NMToken for nodeId : ip-172-31-20-42.ap-northeast-1.compute.internal:57515 for container : container_1448220410260_0001_01_000002 2015-11-22 19:27:01,447 INFO [IPC Server handler 7 on 43231] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220410260_0001_01_000002 Container Transitioned from ALLOCATED to ACQUIRED 2015-11-22 19:27:01,571 INFO [Socket Reader #1 for port 57515] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220410260_0001_000001 (auth:SIMPLE) 2015-11-22 19:27:01,581 INFO [IPC Server handler 6 on 57515] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:startContainerInternal(889)) - Start request for container_1448220410260_0001_01_000002 by user ubuntu 2015-11-22 19:27:01,581 INFO [IPC Server handler 6 on 57515] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu IP=172.31.20.42 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1448220410260_0001 CONTAINERID=container_1448220410260_0001_01_000002 2015-11-22 19:27:01,581 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(303)) - Adding container_1448220410260_0001_01_000002 to application application_1448220410260_0001 2015-11-22 19:27:01,583 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220410260_0001_01_000002 transitioned from NEW to LOCALIZING 2015-11-22 19:27:01,583 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_INIT for appId application_1448220410260_0001 2015-11-22 19:27:01,583 INFO [AsyncDispatcher event handler] localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource file:/home/ubuntu/DistributedShell/application_1448220410260_0001/ExecScript.sh transitioned from INIT to DOWNLOADING 2015-11-22 19:27:01,583 INFO [AsyncDispatcher event handler] localizer.ResourceLocalizationService (ResourceLocalizationService.java:handle(742)) - Created localizer for container_1448220410260_0001_01_000002 2015-11-22 19:27:01,598 INFO [LocalizerRunner for container_1448220410260_0001_01_000002] localizer.ResourceLocalizationService (ResourceLocalizationService.java:writeCredentials(1221)) - Writing credentials to the nmPrivate file /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/container_1448220410260_0001_01_000002.tokens. Credentials list: 2015-11-22 19:27:01,598 INFO [LocalizerRunner for container_1448220410260_0001_01_000002] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:createUserCacheDirs(632)) - Initializing user ubuntu 2015-11-22 19:27:01,620 INFO [Socket Reader #1 for port 57515] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220410260_0001_000001 (auth:SIMPLE) 2015-11-22 19:27:01,625 INFO [LocalizerRunner for container_1448220410260_0001_01_000002] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:startLocalizer(126)) - Copying from /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/container_1448220410260_0001_01_000002.tokens to /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220410260_0001/container_1448220410260_0001_01_000002.tokens 2015-11-22 19:27:01,625 INFO [LocalizerRunner for container_1448220410260_0001_01_000002] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:startLocalizer(133)) - Localizer CWD set to /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220410260_0001 = file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220410260_0001 2015-11-22 19:27:01,636 INFO [IPC Server handler 1 on 57515] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:getContainerStatusInternal(1221)) - Getting container-status for container_1448220410260_0001_01_000002 2015-11-22 19:27:01,644 INFO [IPC Server handler 1 on 57515] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:getContainerStatusInternal(1235)) - Returning ContainerStatus: [ContainerId: container_1448220410260_0001_01_000002, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ] 2015-11-22 19:27:01,691 INFO [IPC Server handler 0 on 40894] localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource file:/home/ubuntu/DistributedShell/application_1448220410260_0001/ExecScript.sh(->/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220410260_0001/filecache/11/ExecScript.sh) transitioned from DOWNLOADING to LOCALIZED 2015-11-22 19:27:01,692 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220410260_0001_01_000002 transitioned from LOCALIZING to LOCALIZED 2015-11-22 19:27:01,711 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220410260_0001_01_000002 transitioned from LOCALIZED to RUNNING 2015-11-22 19:27:01,727 INFO [ContainersLauncher #1] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:buildCommandExecutor(281)) - launchContainer: [nice, -n, 0, bash, /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220410260_0001/container_1448220410260_0001_01_000002/default_container_executor.sh] 2015-11-22 19:27:01,738 INFO [ContainersLauncher #1] launcher.ContainerLaunch (ContainerLaunch.java:call(356)) - Container container_1448220410260_0001_01_000002 succeeded 2015-11-22 19:27:01,738 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220410260_0001_01_000002 transitioned from RUNNING to EXITED_WITH_SUCCESS 2015-11-22 19:27:01,738 INFO [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(380)) - Cleaning up container container_1448220410260_0001_01_000002 2015-11-22 19:27:01,753 INFO [DeletionService #1] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(480)) - Deleting absolute path : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220410260_0001/container_1448220410260_0001_01_000002 2015-11-22 19:27:01,754 INFO [AsyncDispatcher event handler] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu OPERATION=Container Finished - Succeeded TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1448220410260_0001 CONTAINERID=container_1448220410260_0001_01_000002 2015-11-22 19:27:01,754 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220410260_0001_01_000002 transitioned from EXITED_WITH_SUCCESS to DONE 2015-11-22 19:27:01,754 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(346)) - Removing container_1448220410260_0001_01_000002 from application application_1448220410260_0001 2015-11-22 19:27:01,754 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_STOP for appId application_1448220410260_0001 2015-11-22 19:27:01,902 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220410260_0001_01_000002 Container Transitioned from ACQUIRED to COMPLETED 2015-11-22 19:27:01,902 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220410260_0001 CONTAINERID=container_1448220410260_0001_01_000002 2015-11-22 19:27:01,902 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:releaseContainer(248)) - Released container container_1448220410260_0001_01_000002 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:57515, which currently has 1 containers, used and available, release resources=true 2015-11-22 19:27:02,385 INFO [Thread-2953] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220416372, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:43440/proxy/application_1448220410260_0001/, appUser=ubuntu 2015-11-22 19:27:02,677 INFO [Socket Reader #1 for port 57515] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220410260_0001_000001 (auth:SIMPLE) 2015-11-22 19:27:02,679 INFO [IPC Server handler 7 on 57515] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:stopContainerInternal(1163)) - Stopping container with container Id: container_1448220410260_0001_01_000002 2015-11-22 19:27:02,680 INFO [IPC Server handler 7 on 57515] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu IP=172.31.20.42 OPERATION=Stop Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1448220410260_0001 CONTAINERID=container_1448220410260_0001_01_000002 2015-11-22 19:27:02,686 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:rememberTargetTransitionsAndStoreState(1258)) - Updating application attempt appattempt_1448220410260_0001_000001 with final state: FINISHING, and exit status: -1000 2015-11-22 19:27:02,686 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220410260_0001_000001 State change from RUNNING to FINAL_SAVING 2015-11-22 19:27:02,686 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:rememberTargetTransitionsAndStoreState(1098)) - Updating application application_1448220410260_0001 with final state: FINISHING 2015-11-22 19:27:02,686 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220410260_0001 State change from RUNNING to FINAL_SAVING 2015-11-22 19:27:02,687 INFO [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:transition(237)) - Updating info for app: application_1448220410260_0001 2015-11-22 19:27:02,687 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220410260_0001_000001 State change from FINAL_SAVING to FINISHING 2015-11-22 19:27:02,687 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220410260_0001 State change from FINAL_SAVING to FINISHING 2015-11-22 19:27:02,789 INFO [IPC Server handler 3 on 43231] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:finishApplicationMaster(300)) - application_1448220410260_0001 unregistered successfully. 2015-11-22 19:27:02,808 INFO [ContainersLauncher #0] launcher.ContainerLaunch (ContainerLaunch.java:call(356)) - Container container_1448220410260_0001_01_000001 succeeded 2015-11-22 19:27:02,809 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220410260_0001_01_000001 transitioned from RUNNING to EXITED_WITH_SUCCESS 2015-11-22 19:27:02,809 INFO [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(380)) - Cleaning up container container_1448220410260_0001_01_000001 2015-11-22 19:27:02,849 INFO [DeletionService #2] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(480)) - Deleting absolute path : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220410260_0001/container_1448220410260_0001_01_000001 2015-11-22 19:27:02,849 INFO [AsyncDispatcher event handler] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu OPERATION=Container Finished - Succeeded TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1448220410260_0001 CONTAINERID=container_1448220410260_0001_01_000001 2015-11-22 19:27:02,850 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220410260_0001_01_000001 transitioned from EXITED_WITH_SUCCESS to DONE 2015-11-22 19:27:02,850 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(346)) - Removing container_1448220410260_0001_01_000001 from application application_1448220410260_0001 2015-11-22 19:27:02,850 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_STOP for appId application_1448220410260_0001 2015-11-22 19:27:03,386 INFO [Thread-2953] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220416372, yarnAppState=FINISHED, distributedFinalState=SUCCEEDED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:43440/proxy/application_1448220410260_0001/, appUser=ubuntu 2015-11-22 19:27:03,386 INFO [Thread-2953] distributedshell.Client (Client.java:monitorApplication(759)) - Application has completed successfully. Breaking monitoring loop 2015-11-22 19:27:03,386 INFO [Thread-2953] distributedshell.TestDistributedShell (TestDistributedShell.java:testDSShellWithShellScript(620)) - Client run completed. Result=true 2015-11-22 19:27:03,391 ERROR [Thread[Thread-2949,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(670)) - ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted 2015-11-22 19:27:03,394 INFO [main] mortbay.log (Slf4jLog.java:info(67)) - Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:0 2015-11-22 19:27:03,496 INFO [main] applicationhistoryservice.ApplicationHistoryManagerImpl (ApplicationHistoryManagerImpl.java:serviceStop(78)) - Stopping ApplicationHistory 2015-11-22 19:27:03,497 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 38383 2015-11-22 19:27:03,498 INFO [IPC Server listener on 38383] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 38383 2015-11-22 19:27:03,499 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:27:03,499 ERROR [Thread[Thread-2931,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(670)) - ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted 2015-11-22 19:27:03,500 INFO [main] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:unRegisterNM(268)) - Successfully Unregistered the Node ip-172-31-20-42.ap-northeast-1.compute.internal:57515 with ResourceManager. 2015-11-22 19:27:03,504 ERROR [Thread[Thread-2924,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(670)) - ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted 2015-11-22 19:27:03,505 INFO [main] mortbay.log (Slf4jLog.java:info(67)) - Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:0 2015-11-22 19:27:03,607 INFO [main] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:cleanUpApplicationsOnNMShutDown(556)) - Applications still running : [application_1448220410260_0001] 2015-11-22 19:27:03,607 INFO [main] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:cleanUpApplicationsOnNMShutDown(572)) - Waiting for Applications to be Finished 2015-11-22 19:27:03,607 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(463)) - Application application_1448220410260_0001 transitioned from RUNNING to APPLICATION_RESOURCES_CLEANINGUP 2015-11-22 19:27:03,607 INFO [DeletionService #0] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(480)) - Deleting absolute path : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220410260_0001 2015-11-22 19:27:03,608 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event APPLICATION_STOP for appId application_1448220410260_0001 2015-11-22 19:27:03,608 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(463)) - Application application_1448220410260_0001 transitioned from APPLICATION_RESOURCES_CLEANINGUP to FINISHED 2015-11-22 19:27:03,608 INFO [AsyncDispatcher event handler] loghandler.NonAggregatingLogHandler (NonAggregatingLogHandler.java:handle(167)) - Scheduling Log Deletion for application: application_1448220410260_0001, with delay of 10800 seconds 2015-11-22 19:27:07,608 INFO [main] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:cleanUpApplicationsOnNMShutDown(589)) - Done waiting for Applications to be Finished. Still alive: [application_1448220410260_0001] 2015-11-22 19:27:07,608 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 57515 2015-11-22 19:27:07,611 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:27:07,611 INFO [IPC Server listener on 57515] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 57515 2015-11-22 19:27:17,613 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 40894 2015-11-22 19:27:17,615 INFO [IPC Server listener on 40894] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 40894 2015-11-22 19:27:17,615 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:27:17,615 WARN [Node Resource Monitor] nodemanager.NodeResourceMonitorImpl (NodeResourceMonitorImpl.java:run(154)) - org.apache.hadoop.yarn.server.nodemanager.NodeResourceMonitorImpl is interrupted. Exiting. 2015-11-22 19:27:17,615 INFO [Public Localizer] localizer.ResourceLocalizationService (ResourceLocalizationService.java:run(908)) - Public cache exiting 2015-11-22 19:27:17,625 ERROR [Thread[Thread-2874,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(670)) - ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted 2015-11-22 19:27:17,626 INFO [main] mortbay.log (Slf4jLog.java:info(67)) - Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:0 2015-11-22 19:27:17,728 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 54235 2015-11-22 19:27:17,730 INFO [IPC Server listener on 54235] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 54235 2015-11-22 19:27:17,730 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:27:17,730 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 40856 2015-11-22 19:27:17,732 INFO [IPC Server listener on 40856] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 40856 2015-11-22 19:27:17,733 INFO [main] resourcemanager.ResourceManager (ResourceManager.java:transitionToStandby(1051)) - Transitioning to standby state 2015-11-22 19:27:17,733 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:27:17,733 WARN [ApplicationMaster Launcher] amlauncher.ApplicationMasterLauncher (ApplicationMasterLauncher.java:run(122)) - org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher$LauncherThread interrupted. Returning. 2015-11-22 19:27:17,734 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 43231 2015-11-22 19:27:17,737 INFO [IPC Server listener on 43231] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 43231 2015-11-22 19:27:17,737 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:27:17,737 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 57053 2015-11-22 19:27:17,740 INFO [IPC Server listener on 57053] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 57053 2015-11-22 19:27:17,740 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:27:17,741 ERROR [ResourceManager Event Processor] resourcemanager.ResourceManager (ResourceManager.java:run(691)) - Returning, interrupted : java.lang.InterruptedException 2015-11-22 19:27:17,741 INFO [Ping Checker] util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(135)) - NMLivelinessMonitor thread interrupted 2015-11-22 19:27:17,741 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:serviceStop(141)) - AsyncDispatcher is draining to stop, igonring any new events. 2015-11-22 19:27:17,742 INFO [Ping Checker] util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(135)) - AMLivelinessMonitor thread interrupted 2015-11-22 19:27:17,743 INFO [Ping Checker] util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(135)) - AMLivelinessMonitor thread interrupted 2015-11-22 19:27:17,743 INFO [Ping Checker] util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(135)) - org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.ContainerAllocationExpirer thread interrupted 2015-11-22 19:27:17,743 ERROR [Thread[Thread-2705,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(670)) - ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted 2015-11-22 19:27:17,745 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stop(211)) - Stopping ApplicationHistoryServer metrics system... 2015-11-22 19:27:17,750 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stop(217)) - ApplicationHistoryServer metrics system stopped. 2015-11-22 19:27:17,750 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:shutdown(608)) - ApplicationHistoryServer metrics system shutdown complete. 2015-11-22 19:27:17,750 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:serviceStop(141)) - AsyncDispatcher is draining to stop, igonring any new events. 2015-11-22 19:27:17,751 INFO [main] resourcemanager.ResourceManager (ResourceManager.java:transitionToStandby(1058)) - Transitioned to standby state 2015-11-22 19:27:17,761 INFO [main] distributedshell.TestDistributedShell (TestDistributedShell.java:setupInternal(77)) - Starting up YARN cluster 2015-11-22 19:27:17,770 INFO [main] net.ServerSocketUtil (ServerSocketUtil.java:getPort(52)) - Using port 10021 2015-11-22 19:27:17,783 INFO [main] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2363)) - found resource core-site.xml at file:/home/ubuntu/hadoop-dev/hadoop-common-project/hadoop-common/target/test-classes/core-site.xml 2015-11-22 19:27:17,792 WARN [main] conf.Configuration (Configuration.java:loadProperty(2687)) - core-site.xml:an attempt to override final parameter: hadoop.tmp.dir; Ignoring. 2015-11-22 19:27:17,793 INFO [main] security.Groups (Groups.java:refresh(256)) - clearing userToGroupsMap cache 2015-11-22 19:27:17,794 INFO [main] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2363)) - found resource yarn-site.xml at file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/test-classes/yarn-site.xml 2015-11-22 19:27:17,809 WARN [main] conf.Configuration (Configuration.java:loadProperty(2687)) - yarn-site.xml:an attempt to override final parameter: hadoop.tmp.dir; Ignoring. 2015-11-22 19:27:17,810 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.RMFatalEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMFatalEventDispatcher 2015-11-22 19:27:17,810 INFO [main] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:(75)) - NMTokenKeyRollingInterval: 86400000ms and NMTokenKeyActivationDelay: 900000ms 2015-11-22 19:27:17,810 INFO [main] security.RMContainerTokenSecretManager (RMContainerTokenSecretManager.java:(78)) - ContainerTokenKeyRollingInterval: 86400000ms and ContainerTokenKeyActivationDelay: 900000ms 2015-11-22 19:27:17,811 INFO [main] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:(94)) - AMRMTokenKeyRollingInterval: 86400000ms and AMRMTokenKeyActivationDelay: 900000 ms 2015-11-22 19:27:17,811 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStoreEventType for class org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$ForwardingEventHandler 2015-11-22 19:27:17,811 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.NodesListManagerEventType for class org.apache.hadoop.yarn.server.resourcemanager.NodesListManager 2015-11-22 19:27:17,811 INFO [main] resourcemanager.ResourceManager (ResourceManager.java:createScheduler(295)) - Using Scheduler: org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler 2015-11-22 19:27:17,811 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.SchedulerEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher 2015-11-22 19:27:17,811 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationEventDispatcher 2015-11-22 19:27:17,811 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationAttemptEventDispatcher 2015-11-22 19:27:17,811 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$NodeEventDispatcher 2015-11-22 19:27:17,813 INFO [main] impl.MetricsConfig (MetricsConfig.java:loadFirst(115)) - loaded properties from hadoop-metrics2.properties 2015-11-22 19:27:17,814 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:startTimer(374)) - Scheduled snapshot period at 10 second(s). 2015-11-22 19:27:17,814 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:start(192)) - ResourceManager metrics system started 2015-11-22 19:27:17,815 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.RMAppManagerEventType for class org.apache.hadoop.yarn.server.resourcemanager.RMAppManager 2015-11-22 19:27:17,815 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncherEventType for class org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher 2015-11-22 19:27:17,815 WARN [main] util.MBeans (MBeans.java:register(73)) - Failed to register MBean "Hadoop:service=ResourceManager,name=RMNMInfo": Instance already exists. 2015-11-22 19:27:17,815 INFO [main] resourcemanager.RMNMInfo (RMNMInfo.java:(63)) - Registered RMNMInfo MBean 2015-11-22 19:27:17,815 INFO [main] util.HostsFileReader (HostsFileReader.java:refresh(132)) - Refreshing hosts (include/exclude) list 2015-11-22 19:27:17,816 INFO [main] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2363)) - found resource capacity-scheduler.xml at jar:file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/target/hadoop-yarn-server-tests-2.8.0-SNAPSHOT-tests.jar!/capacity-scheduler.xml 2015-11-22 19:27:17,829 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(647)) - max alloc mb per queue for root is undefined 2015-11-22 19:27:17,829 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(651)) - max alloc vcore per queue for root is undefined 2015-11-22 19:27:17,829 INFO [main] capacity.ParentQueue (ParentQueue.java:setupQueueConfigs(125)) - root, capacity=1.0, asboluteCapacity=1.0, maxCapacity=1.0, asboluteMaxCapacity=1.0, state=RUNNING, acls=ADMINISTER_QUEUE:*SUBMIT_APP:*, labels=*, , reservationsContinueLooking=true 2015-11-22 19:27:17,829 INFO [main] capacity.ParentQueue (ParentQueue.java:(104)) - Initialized parent-queue root name=root, fullname=root 2015-11-22 19:27:17,831 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(647)) - max alloc mb per queue for root.default is undefined 2015-11-22 19:27:17,831 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(651)) - max alloc vcore per queue for root.default is undefined 2015-11-22 19:27:17,832 INFO [main] capacity.LeafQueue (LeafQueue.java:setupQueueConfigs(217)) - Initializing default capacity = 1.0 [= (float) configuredCapacity / 100 ] asboluteCapacity = 1.0 [= parentAbsoluteCapacity * capacity ] maxCapacity = 1.0 [= configuredMaxCapacity ] absoluteMaxCapacity = 1.0 [= 1.0 maximumCapacity undefined, (parentAbsoluteMaxCapacity * maximumCapacity) / 100 otherwise ] userLimit = 100 [= configuredUserLimit ] userLimitFactor = 1.0 [= configuredUserLimitFactor ] maxApplications = 10000 [= configuredMaximumSystemApplicationsPerQueue or (int)(configuredMaximumSystemApplications * absoluteCapacity)] maxApplicationsPerUser = 10000 [= (int)(maxApplications * (userLimit / 100.0f) * userLimitFactor) ] usedCapacity = 0.0 [= usedResourcesMemory / (clusterResourceMemory * absoluteCapacity)] absoluteUsedCapacity = 0.0 [= usedResourcesMemory / clusterResourceMemory] maxAMResourcePerQueuePercent = 0.1 [= configuredMaximumAMResourcePercent ] minimumAllocationFactor = 0.984375 [= (float)(maximumAllocationMemory - minimumAllocationMemory) / maximumAllocationMemory ] maximumAllocation = [= configuredMaxAllocation ] numContainers = 0 [= currentNumContainers ] state = RUNNING [= configuredState ] acls = ADMINISTER_QUEUE:*SUBMIT_APP:* [= configuredAcls ] nodeLocalityDelay = -1 labels=*, nodeLocalityDelay = -1 reservationsContinueLooking = true preemptionDisabled = true defaultAppPriorityPerQueue = 0 2015-11-22 19:27:17,832 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:parseQueue(657)) - Initialized queue: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=0, numContainers=0 2015-11-22 19:27:17,832 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:parseQueue(657)) - Initialized queue: root: numChildQueue= 1, capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.0, numApps=0, numContainers=0 2015-11-22 19:27:17,832 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:initializeQueues(501)) - Initialized root queue root: numChildQueue= 1, capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.0, numApps=0, numContainers=0 2015-11-22 19:27:17,832 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:getUserGroupMappingPlacementRule(449)) - Initialized queue mappings, override: false 2015-11-22 19:27:17,832 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:initScheduler(314)) - Initialized CapacityScheduler with calculator=class org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator, minimumAllocation=<>, maximumAllocation=<>, asynchronousScheduling=false, asyncScheduleInterval=5ms 2015-11-22 19:27:17,833 INFO [main] metrics.SystemMetricsPublisher (SystemMetricsPublisher.java:serviceInit(95)) - YARN system metrics publishing service is not enabled 2015-11-22 19:27:17,833 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEventType for class org.apache.hadoop.yarn.server.MiniYARNCluster$1 2015-11-22 19:27:17,833 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:prepareDirs(558)) - Created localDir in /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0 2015-11-22 19:27:17,834 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:prepareDirs(558)) - Created logDir in /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-logDir-nm-0_0 2015-11-22 19:27:17,834 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceInit(540)) - Starting NM: 0 2015-11-22 19:27:17,843 INFO [main] nodemanager.NodeManager (NodeManager.java:getNodeHealthScriptRunner(255)) - Node Manager health check script is not available or doesn't have execute permission, so not starting the node health script runner. 2015-11-22 19:27:17,844 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ContainerEventDispatcher 2015-11-22 19:27:17,844 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ApplicationEventDispatcher 2015-11-22 19:27:17,844 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizationEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService 2015-11-22 19:27:17,844 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServicesEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices 2015-11-22 19:27:17,844 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl 2015-11-22 19:27:17,844 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncherEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncher 2015-11-22 19:27:17,844 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.ContainerManagerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl 2015-11-22 19:27:17,845 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.NodeManagerEventType for class org.apache.hadoop.yarn.server.MiniYARNCluster$ShortCircuitedNodeManager 2015-11-22 19:27:17,845 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:init(159)) - NodeManager metrics system started (again) 2015-11-22 19:27:17,846 INFO [main] nodemanager.NodeResourceMonitorImpl (NodeResourceMonitorImpl.java:serviceInit(76)) - Using ResourceCalculatorPlugin : org.apache.hadoop.yarn.util.ResourceCalculatorPlugin@62e7f11d 2015-11-22 19:27:17,846 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.event.LogHandlerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.NonAggregatingLogHandler 2015-11-22 19:27:17,846 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.sharedcache.SharedCacheUploadEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.sharedcache.SharedCacheUploadService 2015-11-22 19:27:17,846 INFO [main] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:serviceInit(261)) - AMRMProxyService is disabled 2015-11-22 19:27:17,846 INFO [main] localizer.ResourceLocalizationService (ResourceLocalizationService.java:validateConf(224)) - per directory file limit = 8192 2015-11-22 19:27:17,857 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerTracker 2015-11-22 19:27:17,857 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(116)) - Using ResourceCalculatorPlugin : org.apache.hadoop.yarn.util.ResourceCalculatorPlugin@503d687a 2015-11-22 19:27:17,857 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(121)) - Using ResourceCalculatorProcessTree : null 2015-11-22 19:27:17,857 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(158)) - Physical memory check enabled: false 2015-11-22 19:27:17,858 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(159)) - Virtual memory check enabled: false 2015-11-22 19:27:17,858 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:isEnabled(206)) - Neither virutal-memory nor physical-memory monitoring is needed. Not running the monitor-thread 2015-11-22 19:27:17,858 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(162)) - ContainersMonitor enabled: false 2015-11-22 19:27:17,858 INFO [main] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:serviceInit(175)) - Nodemanager resources: memory set to 4096MB. 2015-11-22 19:27:17,858 INFO [main] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:serviceInit(176)) - Nodemanager resources: vcores set to 8. 2015-11-22 19:27:17,858 INFO [main] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:serviceInit(208)) - Initialized nodemanager with : physical-memory=4096 virtual-memory=8602 virtual-cores=8 2015-11-22 19:27:17,858 WARN [main] applicationhistoryservice.ApplicationHistoryServer (ApplicationHistoryServer.java:createApplicationHistoryManager(218)) - The filesystem based application history store is deprecated. 2015-11-22 19:27:17,859 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:init(159)) - ApplicationHistoryServer metrics system started (again) 2015-11-22 19:27:17,859 INFO [main] applicationhistoryservice.ApplicationHistoryManagerImpl (ApplicationHistoryManagerImpl.java:serviceInit(61)) - ApplicationHistory Init 2015-11-22 19:27:17,859 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:startResourceManager(326)) - Waiting for RM to start... 2015-11-22 19:27:17,859 INFO [RM-0] resourcemanager.ResourceManager (ResourceManager.java:transitionToActive(1024)) - Transitioning to active state 2015-11-22 19:27:17,860 INFO [RM-0] recovery.RMStateStore (RMStateStore.java:transition(482)) - Updating AMRMToken 2015-11-22 19:27:17,860 INFO [RM-0] security.RMContainerTokenSecretManager (RMContainerTokenSecretManager.java:rollMasterKey(106)) - Rolling master-key for container-tokens 2015-11-22 19:27:17,860 INFO [RM-0] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:rollMasterKey(95)) - Rolling master-key for nm-tokens 2015-11-22 19:27:17,861 INFO [RM-0] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:27:17,861 INFO [RM-0] security.RMDelegationTokenSecretManager (RMDelegationTokenSecretManager.java:storeNewMasterKey(87)) - storing master key with keyID 1 2015-11-22 19:27:17,861 INFO [RM-0] recovery.RMStateStore (RMStateStore.java:transition(433)) - Storing RMDTMasterKey. 2015-11-22 19:27:17,861 INFO [Thread[Thread-3060,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2015-11-22 19:27:17,862 INFO [Thread[Thread-3060,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:27:17,862 INFO [Thread[Thread-3060,5,main]] security.RMDelegationTokenSecretManager (RMDelegationTokenSecretManager.java:storeNewMasterKey(87)) - storing master key with keyID 2 2015-11-22 19:27:17,862 INFO [Thread[Thread-3060,5,main]] recovery.RMStateStore (RMStateStore.java:transition(433)) - Storing RMDTMasterKey. 2015-11-22 19:27:17,863 INFO [RM-0] nodelabels.CommonNodeLabelsManager (CommonNodeLabelsManager.java:addToCluserNodeLabels(309)) - Add labels: [] 2015-11-22 19:27:17,863 INFO [RM-0] nodelabels.CommonNodeLabelsManager (CommonNodeLabelsManager.java:internalUpdateLabelsOnNodes(638)) - REPLACE labels on nodes: 2015-11-22 19:27:17,863 INFO [RM-0] nodelabels.CommonNodeLabelsManager (CommonNodeLabelsManager.java:internalUpdateLabelsOnNodes(640)) - NM=ip-172-31-20-42.ap-northeast-1.compute.internal:58694, labels=[x] 2015-11-22 19:27:17,864 INFO [RM-0] nodelabels.FileSystemNodeLabelsStore (FileSystemNodeLabelsStore.java:recover(279)) - Finished write mirror at:file:/tmp/hadoop-yarn-ubuntu/node-labels/nodelabel.mirror 2015-11-22 19:27:17,864 INFO [RM-0] nodelabels.FileSystemNodeLabelsStore (FileSystemNodeLabelsStore.java:recover(280)) - Finished create editlog file at:file:/tmp/hadoop-yarn-ubuntu/node-labels/nodelabel.editlog 2015-11-22 19:27:17,864 INFO [RM-0] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.nodelabels.event.NodeLabelsStoreEventType for class org.apache.hadoop.yarn.nodelabels.CommonNodeLabelsManager$ForwardingEventHandler 2015-11-22 19:27:17,865 INFO [RM-0] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:27:17,865 INFO [Socket Reader #1 for port 49964] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 49964 2015-11-22 19:27:17,866 INFO [RM-0] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.server.api.ResourceTrackerPB to the server 2015-11-22 19:27:17,866 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:27:17,866 INFO [IPC Server listener on 49964] ipc.Server (Server.java:run(794)) - IPC Server listener on 49964: starting 2015-11-22 19:27:17,871 INFO [RM-0] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:27:17,871 INFO [Socket Reader #1 for port 60164] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 60164 2015-11-22 19:27:17,872 INFO [RM-0] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB to the server 2015-11-22 19:27:17,873 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:27:17,873 INFO [IPC Server listener on 60164] ipc.Server (Server.java:run(794)) - IPC Server listener on 60164: starting 2015-11-22 19:27:17,878 INFO [RM-0] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:27:17,878 INFO [Socket Reader #1 for port 34580] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 34580 2015-11-22 19:27:17,879 INFO [RM-0] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.api.ApplicationClientProtocolPB to the server 2015-11-22 19:27:17,879 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:27:17,879 INFO [IPC Server listener on 34580] ipc.Server (Server.java:run(794)) - IPC Server listener on 34580: starting 2015-11-22 19:27:17,884 INFO [RM-0] resourcemanager.ResourceManager (ResourceManager.java:transitionToActive(1040)) - Transitioned to active state 2015-11-22 19:27:17,885 INFO [RM-0] server.AuthenticationFilter (AuthenticationFilter.java:constructSecretProvider(304)) - Unable to initialize FileSignerSecretProvider, falling back to use random secrets. 2015-11-22 19:27:17,885 INFO [RM-0] http.HttpRequestLog (HttpRequestLog.java:getRequestLog(80)) - Http request log for http.requests.resourcemanager is not defined 2015-11-22 19:27:17,885 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter) 2015-11-22 19:27:17,886 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context cluster 2015-11-22 19:27:17,886 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context logs 2015-11-22 19:27:17,886 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context static 2015-11-22 19:27:17,886 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context cluster 2015-11-22 19:27:17,886 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2015-11-22 19:27:17,886 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2015-11-22 19:27:17,887 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'Timeline Authentication Filter' (class=org.apache.hadoop.yarn.server.timeline.security.TimelineAuthenticationFilter) 2015-11-22 19:27:17,887 INFO [RM-0] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /cluster/* 2015-11-22 19:27:17,887 INFO [RM-0] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /ws/* 2015-11-22 19:27:17,891 INFO [RM-0] webapp.WebApps (WebApps.java:build(288)) - Registered webapp guice modules 2015-11-22 19:27:17,891 INFO [RM-0] http.HttpServer2 (HttpServer2.java:openListeners(917)) - Jetty bound to port 44490 2015-11-22 19:27:17,891 INFO [RM-0] mortbay.log (Slf4jLog.java:info(67)) - jetty-6.1.26 2015-11-22 19:27:17,902 INFO [RM-0] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:27:17,902 INFO [Thread[Thread-3229,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2015-11-22 19:27:17,903 INFO [Thread[Thread-3229,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:27:17,903 INFO [RM-0] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:27:17,905 INFO [Thread[Thread-3231,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) Nov 22, 2015 7:27:17 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.resourcemanager.webapp.JAXBContextResolver as a provider class Nov 22, 2015 7:27:17 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices as a root resource class Nov 22, 2015 7:27:17 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a provider class Nov 22, 2015 7:27:17 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate INFO: Initiating Jersey application, version 'Jersey: 1.9 09/02/2011 11:17 AM' Nov 22, 2015 7:27:17 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.resourcemanager.webapp.JAXBContextResolver to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:27:17 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.webapp.GenericExceptionHandler to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:27:18 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices to GuiceManagedComponentProvider with the scope "Singleton" 2015-11-22 19:27:18,044 INFO [RM-0] mortbay.log (Slf4jLog.java:info(67)) - Started HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:44490 2015-11-22 19:27:18,045 INFO [RM-0] webapp.WebApps (WebApps.java:start(307)) - Web app cluster started at 44490 2015-11-22 19:27:18,046 INFO [RM-0] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:27:18,046 INFO [Socket Reader #1 for port 42921] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 42921 2015-11-22 19:27:18,047 INFO [RM-0] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.server.api.ResourceManagerAdministrationProtocolPB to the server 2015-11-22 19:27:18,048 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:27:18,048 INFO [IPC Server listener on 42921] ipc.Server (Server.java:run(794)) - IPC Server listener on 42921: starting 2015-11-22 19:27:19,360 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:startResourceManager(339)) - MiniYARN ResourceManager address: ip-172-31-20-42.ap-northeast-1.compute.internal:34580 2015-11-22 19:27:19,360 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:startResourceManager(341)) - MiniYARN ResourceManager web address: ip-172-31-20-42.ap-northeast-1.compute.internal:44490 2015-11-22 19:27:19,360 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(459)) - MiniYARN ResourceManager address: ip-172-31-20-42.ap-northeast-1.compute.internal:34580 2015-11-22 19:27:19,360 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(461)) - MiniYARN ResourceManager web address: ip-172-31-20-42.ap-northeast-1.compute.internal:44490 2015-11-22 19:27:19,361 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(575)) - Waiting for NM 0 to start... 2015-11-22 19:27:19,362 INFO [Thread-3238] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:27:19,363 INFO [Socket Reader #1 for port 36674] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 36674 2015-11-22 19:27:19,364 INFO [Thread-3238] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.api.ContainerManagementProtocolPB to the server 2015-11-22 19:27:19,364 INFO [Thread-3238] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:serviceStart(464)) - Blocking new container-requests as container manager rpc server is still starting. 2015-11-22 19:27:19,365 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:27:19,365 INFO [IPC Server listener on 36674] ipc.Server (Server.java:run(794)) - IPC Server listener on 36674: starting 2015-11-22 19:27:19,367 INFO [Thread-3238] security.NMContainerTokenSecretManager (NMContainerTokenSecretManager.java:setNodeId(260)) - Updating node address : ip-172-31-20-42.ap-northeast-1.compute.internal:36674 2015-11-22 19:27:19,367 INFO [Thread-3238] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:27:19,368 INFO [Socket Reader #1 for port 47867] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 47867 2015-11-22 19:27:19,369 INFO [Thread-3238] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB to the server 2015-11-22 19:27:19,369 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:27:19,369 INFO [IPC Server listener on 47867] ipc.Server (Server.java:run(794)) - IPC Server listener on 47867: starting 2015-11-22 19:27:19,370 INFO [Thread-3238] localizer.ResourceLocalizationService (ResourceLocalizationService.java:serviceStart(363)) - Localizer started on port 47867 2015-11-22 19:27:19,376 INFO [Thread-3238] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:serviceStart(509)) - ContainerManager started at /172.31.20.42:36674 2015-11-22 19:27:19,376 INFO [Thread-3238] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:serviceStart(510)) - ContainerManager bound to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:0 2015-11-22 19:27:19,376 INFO [Thread-3238] webapp.WebServer (WebServer.java:serviceStart(71)) - Instantiating NMWebApp at ip-172-31-20-42.ap-northeast-1.compute.internal:0 2015-11-22 19:27:19,377 INFO [Thread-3238] server.AuthenticationFilter (AuthenticationFilter.java:constructSecretProvider(304)) - Unable to initialize FileSignerSecretProvider, falling back to use random secrets. 2015-11-22 19:27:19,377 INFO [Thread-3238] http.HttpRequestLog (HttpRequestLog.java:getRequestLog(80)) - Http request log for http.requests.nodemanager is not defined 2015-11-22 19:27:19,378 INFO [Thread-3238] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter) 2015-11-22 19:27:19,378 INFO [Thread-3238] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context node 2015-11-22 19:27:19,378 INFO [Thread-3238] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context static 2015-11-22 19:27:19,378 INFO [Thread-3238] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context logs 2015-11-22 19:27:19,379 INFO [Thread-3238] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context node 2015-11-22 19:27:19,379 INFO [Thread-3238] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2015-11-22 19:27:19,379 INFO [Thread-3238] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2015-11-22 19:27:19,379 INFO [Thread-3238] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'Timeline Authentication Filter' (class=org.apache.hadoop.yarn.server.timeline.security.TimelineAuthenticationFilter) 2015-11-22 19:27:19,379 INFO [Thread-3238] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /node/* 2015-11-22 19:27:19,379 INFO [Thread-3238] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /ws/* Nov 22, 2015 7:27:19 PM com.google.inject.servlet.GuiceFilter setPipeline WARNING: Multiple Servlet injectors detected. This is a warning indicating that you have more than one GuiceFilter running in your web application. If this is deliberate, you may safely ignore this message. If this is NOT deliberate however, your application may not work as expected. 2015-11-22 19:27:19,384 INFO [Thread-3238] webapp.WebApps (WebApps.java:build(288)) - Registered webapp guice modules 2015-11-22 19:27:19,385 INFO [Thread-3238] http.HttpServer2 (HttpServer2.java:openListeners(917)) - Jetty bound to port 47331 2015-11-22 19:27:19,385 INFO [Thread-3238] mortbay.log (Slf4jLog.java:info(67)) - jetty-6.1.26 2015-11-22 19:27:19,396 INFO [Thread-3238] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:27:19,396 INFO [Thread[Thread-3279,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2015-11-22 19:27:19,397 INFO [Thread[Thread-3279,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:27:19,398 INFO [Thread-3238] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:27:19,398 INFO [Thread[Thread-3281,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) Nov 22, 2015 7:27:19 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.nodemanager.webapp.NMWebServices as a root resource class Nov 22, 2015 7:27:19 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a provider class Nov 22, 2015 7:27:19 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.nodemanager.webapp.JAXBContextResolver as a provider class Nov 22, 2015 7:27:19 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate INFO: Initiating Jersey application, version 'Jersey: 1.9 09/02/2011 11:17 AM' Nov 22, 2015 7:27:19 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.nodemanager.webapp.JAXBContextResolver to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:27:19 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.webapp.GenericExceptionHandler to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:27:19 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.nodemanager.webapp.NMWebServices to GuiceManagedComponentProvider with the scope "Singleton" 2015-11-22 19:27:19,480 INFO [Thread-3238] mortbay.log (Slf4jLog.java:info(67)) - Started HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:47331 2015-11-22 19:27:19,484 INFO [Thread-3238] webapp.WebApps (WebApps.java:start(307)) - Web app node started at 47331 2015-11-22 19:27:19,485 INFO [Thread-3238] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:serviceStart(222)) - Node ID assigned is : ip-172-31-20-42.ap-northeast-1.compute.internal:36674 2015-11-22 19:27:19,485 INFO [Thread-3238] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNMContainerStatuses(550)) - Sending out 0 NM container statuses: [] 2015-11-22 19:27:19,485 INFO [Thread-3238] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:registerWithRM(330)) - Registering with RM using containers :[] 2015-11-22 19:27:19,488 INFO [Thread-3238] resourcemanager.ResourceTrackerService (ResourceTrackerService.java:registerNodeManager(375)) - NodeManager from node ip-172-31-20-42.ap-northeast-1.compute.internal(cmPort: 36674 httpPort: 47331) registered with capability: , assigned nodeId ip-172-31-20-42.ap-northeast-1.compute.internal:36674 2015-11-22 19:27:19,488 INFO [Thread-3238] security.NMContainerTokenSecretManager (NMContainerTokenSecretManager.java:setMasterKey(138)) - Rolling master-key for container-tokens, got key with id -2050068055 2015-11-22 19:27:19,488 INFO [Thread-3238] security.NMTokenSecretManagerInNM (NMTokenSecretManagerInNM.java:setMasterKey(135)) - Rolling master-key for container-tokens, got key with id 216388716 2015-11-22 19:27:19,488 INFO [Thread-3238] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:registerWithRM(391)) - Registered with ResourceManager as ip-172-31-20-42.ap-northeast-1.compute.internal:36674 with total resource of 2015-11-22 19:27:19,488 INFO [Thread-3238] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:registerWithRM(392)) - Notifying ContainerManager to unblock new container-requests 2015-11-22 19:27:19,489 INFO [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(566)) - ip-172-31-20-42.ap-northeast-1.compute.internal:36674 Node Transitioned from NEW to RUNNING 2015-11-22 19:27:19,491 INFO [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:addNode(1427)) - Added node ip-172-31-20-42.ap-northeast-1.compute.internal:36674 clusterResource: 2015-11-22 19:27:20,361 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(728)) - Waiting for Timeline Server to start... 2015-11-22 19:27:20,361 INFO [Thread-3284] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:27:20,363 INFO [Thread[Thread-3286,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2015-11-22 19:27:20,364 INFO [Thread-3284] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:27:20,364 INFO [Thread[Thread-3286,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:27:20,364 INFO [Socket Reader #1 for port 37547] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 37547 2015-11-22 19:27:20,369 INFO [Thread-3284] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.api.ApplicationHistoryProtocolPB to the server 2015-11-22 19:27:20,369 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:27:20,370 INFO [IPC Server listener on 37547] ipc.Server (Server.java:run(794)) - IPC Server listener on 37547: starting 2015-11-22 19:27:20,373 INFO [Thread-3284] applicationhistoryservice.ApplicationHistoryClientService (ApplicationHistoryClientService.java:serviceStart(111)) - Instantiated ApplicationHistoryClientService at ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:37547 2015-11-22 19:27:20,373 INFO [Thread-3284] applicationhistoryservice.ApplicationHistoryManagerImpl (ApplicationHistoryManagerImpl.java:serviceStart(71)) - Starting ApplicationHistory 2015-11-22 19:27:20,374 INFO [Thread-3284] server.AuthenticationFilter (AuthenticationFilter.java:constructSecretProvider(304)) - Unable to initialize FileSignerSecretProvider, falling back to use random secrets. 2015-11-22 19:27:20,374 INFO [Thread-3284] http.HttpRequestLog (HttpRequestLog.java:getRequestLog(80)) - Http request log for http.requests.applicationhistory is not defined 2015-11-22 19:27:20,374 INFO [Thread-3284] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter) 2015-11-22 19:27:20,375 INFO [Thread-3284] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context applicationhistory 2015-11-22 19:27:20,375 INFO [Thread-3284] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context static 2015-11-22 19:27:20,375 INFO [Thread-3284] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context logs 2015-11-22 19:27:20,375 INFO [Thread-3284] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context applicationhistory 2015-11-22 19:27:20,375 INFO [Thread-3284] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2015-11-22 19:27:20,375 INFO [Thread-3284] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2015-11-22 19:27:20,376 INFO [Thread-3284] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'Timeline Authentication Filter' (class=org.apache.hadoop.yarn.server.timeline.security.TimelineAuthenticationFilter) 2015-11-22 19:27:20,376 INFO [Thread-3284] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /applicationhistory/* 2015-11-22 19:27:20,376 INFO [Thread-3284] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /ws/* Nov 22, 2015 7:27:20 PM com.google.inject.servlet.GuiceFilter setPipeline WARNING: Multiple Servlet injectors detected. This is a warning indicating that you have more than one GuiceFilter running in your web application. If this is deliberate, you may safely ignore this message. If this is NOT deliberate however, your application may not work as expected. 2015-11-22 19:27:20,380 INFO [Thread-3284] webapp.WebApps (WebApps.java:build(288)) - Registered webapp guice modules 2015-11-22 19:27:20,382 INFO [Thread-3284] http.HttpServer2 (HttpServer2.java:openListeners(917)) - Jetty bound to port 48234 2015-11-22 19:27:20,382 INFO [Thread-3284] mortbay.log (Slf4jLog.java:info(67)) - jetty-6.1.26 2015-11-22 19:27:20,392 INFO [Thread-3284] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:27:20,392 INFO [Thread[Thread-3304,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2015-11-22 19:27:20,393 INFO [Thread[Thread-3304,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:27:20,394 INFO [Thread-3284] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:27:20,394 INFO [Thread[Thread-3306,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) Nov 22, 2015 7:27:20 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider as a provider class Nov 22, 2015 7:27:20 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.AHSWebServices as a root resource class Nov 22, 2015 7:27:20 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.timeline.webapp.TimelineWebServices as a root resource class Nov 22, 2015 7:27:20 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a provider class Nov 22, 2015 7:27:20 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate INFO: Initiating Jersey application, version 'Jersey: 1.9 09/02/2011 11:17 AM' Nov 22, 2015 7:27:20 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.webapp.GenericExceptionHandler to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:27:20 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:27:20 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.AHSWebServices to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:27:20 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.timeline.webapp.TimelineWebServices to GuiceManagedComponentProvider with the scope "Singleton" 2015-11-22 19:27:20,471 INFO [Thread-3284] mortbay.log (Slf4jLog.java:info(67)) - Started HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:48234 2015-11-22 19:27:20,472 INFO [Thread-3284] applicationhistoryservice.ApplicationHistoryServer (ApplicationHistoryServer.java:startWebApp(342)) - Instantiating AHSWebApp at 48234 2015-11-22 19:27:21,861 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(739)) - MiniYARN ApplicationHistoryServer address: ip-172-31-20-42.ap-northeast-1.compute.internal:37547 2015-11-22 19:27:21,862 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(741)) - MiniYARN ApplicationHistoryServer web address: ip-172-31-20-42.ap-northeast-1.compute.internal:48234 2015-11-22 19:27:23,897 INFO [Thread-3308] distributedshell.TestDistributedShell (TestDistributedShell.java:testDSRestartWithPreviousRunningContainers(360)) - Initializing DS Client 2015-11-22 19:27:23,950 INFO [Thread-3308] impl.TimelineClientImpl (TimelineClientImpl.java:serviceInit(296)) - Timeline service address: http://ip-172-31-20-42.ap-northeast-1.compute.internal:48234/ws/v1/timeline/ 2015-11-22 19:27:23,951 INFO [Thread-3308] distributedshell.Client (Client.java:init(339)) - keep_containers_across_application_attempts 2015-11-22 19:27:23,951 INFO [Thread-3308] distributedshell.TestDistributedShell (TestDistributedShell.java:testDSRestartWithPreviousRunningContainers(365)) - Running DS Client 2015-11-22 19:27:23,951 INFO [Thread-3308] distributedshell.Client (Client.java:run(443)) - Running Client 2015-11-22 19:27:23,962 INFO [Thread-3308] client.RMProxy (RMProxy.java:createRMProxy(98)) - Connecting to ResourceManager at ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:34580 2015-11-22 19:27:23,965 INFO [Thread-3308] distributedshell.Client (Client.java:run(447)) - Got Cluster metric info from ASM, numNodeManagers=1 2015-11-22 19:27:23,967 INFO [Thread-3308] distributedshell.Client (Client.java:run(452)) - Got Cluster node info from ASM 2015-11-22 19:27:23,967 INFO [Thread-3308] distributedshell.Client (Client.java:run(454)) - Got node report from ASM for, nodeId=ip-172-31-20-42.ap-northeast-1.compute.internal:36674, nodeAddress=ip-172-31-20-42.ap-northeast-1.compute.internal:47331, nodeRackName=/default-rack, nodeNumContainers=0 2015-11-22 19:27:23,968 INFO [Thread-3308] distributedshell.Client (Client.java:run(462)) - Queue info, queueName=default, queueCurrentCapacity=0.0, queueMaxCapacity=1.0, queueApplicationCount=0, queueChildQueueCount=0 2015-11-22 19:27:23,969 INFO [Thread-3308] distributedshell.Client (Client.java:run(472)) - User ACL Info for Queue, queueName=root, userAcl=SUBMIT_APPLICATIONS 2015-11-22 19:27:23,969 INFO [Thread-3308] distributedshell.Client (Client.java:run(472)) - User ACL Info for Queue, queueName=root, userAcl=ADMINISTER_QUEUE 2015-11-22 19:27:23,969 INFO [Thread-3308] distributedshell.Client (Client.java:run(472)) - User ACL Info for Queue, queueName=default, userAcl=SUBMIT_APPLICATIONS 2015-11-22 19:27:23,969 INFO [Thread-3308] distributedshell.Client (Client.java:run(472)) - User ACL Info for Queue, queueName=default, userAcl=ADMINISTER_QUEUE 2015-11-22 19:27:23,970 INFO [IPC Server handler 5 on 34580] resourcemanager.ClientRMService (ClientRMService.java:getNewApplicationId(305)) - Allocated new applicationId: 1 2015-11-22 19:27:23,970 INFO [Thread-3308] distributedshell.Client (Client.java:run(491)) - Max mem capability of resources in this cluster 4096 2015-11-22 19:27:23,970 INFO [Thread-3308] distributedshell.Client (Client.java:run(502)) - Max virtual cores capability of resources in this cluster 4 2015-11-22 19:27:23,970 INFO [Thread-3308] distributedshell.Client (Client.java:run(528)) - Copy App Master jar from local filesystem and add to local environment 2015-11-22 19:27:23,978 INFO [Thread-3308] distributedshell.Client (Client.java:run(577)) - Set the environment for the application master 2015-11-22 19:27:23,978 INFO [Thread-3308] distributedshell.Client (Client.java:run(619)) - Setting up app master command 2015-11-22 19:27:23,978 INFO [Thread-3308] distributedshell.Client (Client.java:run(650)) - Completed setting up app master command {{JAVA_HOME}}/bin/java -Xmx512m org.apache.hadoop.yarn.applications.distributedshell.TestDSFailedAppMaster --container_memory 128 --container_vcores 1 --num_containers 1 --priority 0 1>/AppMaster.stdout 2>/AppMaster.stderr 2015-11-22 19:27:23,978 INFO [Thread-3308] distributedshell.Client (Client.java:run(706)) - Submitting application to ASM 2015-11-22 19:27:23,979 INFO [IPC Server handler 4 on 34580] capacity.CapacityScheduler (CapacityScheduler.java:checkAndGetApplicationPriority(1941)) - Priority '0' is acceptable in queue : default for application: application_1448220437860_0001 for the user: ubuntu 2015-11-22 19:27:23,979 WARN [IPC Server handler 4 on 34580] rmapp.RMAppImpl (RMAppImpl.java:(420)) - The specific max attempts: 0 for application: 1 is invalid, because it is out of the range [1, 2]. Use the global max attempts instead. 2015-11-22 19:27:23,979 INFO [IPC Server handler 4 on 34580] resourcemanager.ClientRMService (ClientRMService.java:submitApplication(602)) - Application with id 1 submitted by user ubuntu 2015-11-22 19:27:23,979 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:transition(1079)) - Storing application with id application_1448220437860_0001 2015-11-22 19:27:23,979 INFO [IPC Server handler 4 on 34580] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(154)) - USER=ubuntu IP=172.31.20.42 OPERATION=Submit Application Request TARGET=ClientRMService RESULT=SUCCESS APPID=application_1448220437860_0001 2015-11-22 19:27:23,979 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220437860_0001 State change from NEW to NEW_SAVING 2015-11-22 19:27:23,980 INFO [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:transition(208)) - Storing info for app: application_1448220437860_0001 2015-11-22 19:27:23,980 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220437860_0001 State change from NEW_SAVING to SUBMITTED 2015-11-22 19:27:23,980 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:addApplication(349)) - Application added - appId: application_1448220437860_0001 user: ubuntu leaf-queue of parent: root #applications: 1 2015-11-22 19:27:23,980 INFO [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:addApplication(765)) - Accepted application application_1448220437860_0001 from user: ubuntu, in queue: default 2015-11-22 19:27:23,980 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220437860_0001 State change from SUBMITTED to ACCEPTED 2015-11-22 19:27:23,981 INFO [AsyncDispatcher event handler] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:registerAppAttempt(661)) - Registering app attempt : appattempt_1448220437860_0001_000001 2015-11-22 19:27:23,981 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220437860_0001_000001 State change from NEW to SUBMITTED 2015-11-22 19:27:23,981 INFO [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:activateApplications(705)) - Application application_1448220437860_0001 from user: ubuntu activated in queue: default 2015-11-22 19:27:23,981 INFO [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:addApplicationAttempt(720)) - Application added - appId: application_1448220437860_0001 user: ubuntu, leaf-queue: default #user-pending-applications: 0 #user-active-applications: 1 #queue-pending-applications: 0 #queue-active-applications: 1 2015-11-22 19:27:23,981 INFO [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:addApplicationAttempt(805)) - Added Application Attempt appattempt_1448220437860_0001_000001 to scheduler from user ubuntu in queue default 2015-11-22 19:27:23,983 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220437860_0001_000001 State change from SUBMITTED to SCHEDULED 2015-11-22 19:27:24,182 INFO [Thread-3308] impl.YarnClientImpl (YarnClientImpl.java:submitApplication(279)) - Submitted application application_1448220437860_0001 2015-11-22 19:27:24,495 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220437860_0001_01_000001 Container Transitioned from NEW to ALLOCATED 2015-11-22 19:27:24,495 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220437860_0001 CONTAINERID=container_1448220437860_0001_01_000001 2015-11-22 19:27:24,495 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(154)) - Assigned container container_1448220437860_0001_01_000001 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:36674, which has 1 containers, used and available after allocation 2015-11-22 19:27:24,496 INFO [ResourceManager Event Processor] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(89)) - assignedContainer application attempt=appattempt_1448220437860_0001_000001 container=container_1448220437860_0001_01_000001 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@6fc89cc2 clusterResource= type=OFF_SWITCH 2015-11-22 19:27:24,497 INFO [AsyncDispatcher event handler] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:createAndGetNMToken(200)) - Sending NMToken for nodeId : ip-172-31-20-42.ap-northeast-1.compute.internal:36674 for container : container_1448220437860_0001_01_000001 2015-11-22 19:27:24,498 INFO [AsyncDispatcher event handler] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220437860_0001_01_000001 Container Transitioned from ALLOCATED to ACQUIRED 2015-11-22 19:27:24,498 INFO [AsyncDispatcher event handler] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:clearNodeSetForAttempt(146)) - Clear node set for appattempt_1448220437860_0001_000001 2015-11-22 19:27:24,498 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(589)) - Re-sorting assigned queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.125, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 2015-11-22 19:27:24,498 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(461)) - assignedContainer queue=root usedCapacity=0.125 absoluteUsedCapacity=0.125 used= cluster= 2015-11-22 19:27:24,498 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:storeAttempt(2026)) - Storing attempt: AppId: application_1448220437860_0001 AttemptId: appattempt_1448220437860_0001_000001 MasterContainer: Container: [ContainerId: container_1448220437860_0001_01_000001, NodeId: ip-172-31-20-42.ap-northeast-1.compute.internal:36674, NodeHttpAddress: ip-172-31-20-42.ap-northeast-1.compute.internal:47331, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.31.20.42:36674 }, ] 2015-11-22 19:27:24,498 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220437860_0001_000001 State change from SCHEDULED to ALLOCATED_SAVING 2015-11-22 19:27:24,499 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220437860_0001_000001 State change from ALLOCATED_SAVING to ALLOCATED 2015-11-22 19:27:24,499 INFO [ApplicationMasterLauncher #0] amlauncher.AMLauncher (AMLauncher.java:run(254)) - Launching masterappattempt_1448220437860_0001_000001 2015-11-22 19:27:24,501 INFO [ApplicationMasterLauncher #0] amlauncher.AMLauncher (AMLauncher.java:launch(106)) - Setting up container Container: [ContainerId: container_1448220437860_0001_01_000001, NodeId: ip-172-31-20-42.ap-northeast-1.compute.internal:36674, NodeHttpAddress: ip-172-31-20-42.ap-northeast-1.compute.internal:47331, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.31.20.42:36674 }, ] for AM appattempt_1448220437860_0001_000001 2015-11-22 19:27:24,501 INFO [ApplicationMasterLauncher #0] amlauncher.AMLauncher (AMLauncher.java:createAMContainerLaunchContext(188)) - Command to launch container container_1448220437860_0001_01_000001 : {{JAVA_HOME}}/bin/java -Xmx512m org.apache.hadoop.yarn.applications.distributedshell.TestDSFailedAppMaster --container_memory 128 --container_vcores 1 --num_containers 1 --priority 0 1>/AppMaster.stdout 2>/AppMaster.stderr 2015-11-22 19:27:24,501 INFO [ApplicationMasterLauncher #0] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:createAndGetAMRMToken(195)) - Create AMRMToken for ApplicationAttempt: appattempt_1448220437860_0001_000001 2015-11-22 19:27:24,502 INFO [ApplicationMasterLauncher #0] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:createPassword(307)) - Creating password for appattempt_1448220437860_0001_000001 2015-11-22 19:27:24,504 INFO [Socket Reader #1 for port 36674] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220437860_0001_000001 (auth:SIMPLE) 2015-11-22 19:27:24,507 INFO [IPC Server handler 17 on 36674] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:startContainerInternal(889)) - Start request for container_1448220437860_0001_01_000001 by user ubuntu 2015-11-22 19:27:24,508 INFO [IPC Server handler 17 on 36674] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:startContainerInternal(930)) - Creating a new application reference for app application_1448220437860_0001 2015-11-22 19:27:24,508 INFO [IPC Server handler 17 on 36674] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu IP=172.31.20.42 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1448220437860_0001 CONTAINERID=container_1448220437860_0001_01_000001 2015-11-22 19:27:24,508 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(463)) - Application application_1448220437860_0001 transitioned from NEW to INITING 2015-11-22 19:27:24,508 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(303)) - Adding container_1448220437860_0001_01_000001 to application application_1448220437860_0001 2015-11-22 19:27:24,508 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(463)) - Application application_1448220437860_0001 transitioned from INITING to RUNNING 2015-11-22 19:27:24,509 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220437860_0001_01_000001 transitioned from NEW to LOCALIZING 2015-11-22 19:27:24,509 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_INIT for appId application_1448220437860_0001 2015-11-22 19:27:24,509 INFO [AsyncDispatcher event handler] localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource file:/home/ubuntu/DistributedShell/application_1448220437860_0001/AppMaster.jar transitioned from INIT to DOWNLOADING 2015-11-22 19:27:24,509 INFO [AsyncDispatcher event handler] localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource file:/home/ubuntu/DistributedShell/application_1448220437860_0001/shellCommands transitioned from INIT to DOWNLOADING 2015-11-22 19:27:24,509 INFO [AsyncDispatcher event handler] localizer.ResourceLocalizationService (ResourceLocalizationService.java:handle(742)) - Created localizer for container_1448220437860_0001_01_000001 2015-11-22 19:27:24,511 INFO [ApplicationMasterLauncher #0] amlauncher.AMLauncher (AMLauncher.java:launch(127)) - Done launching container Container: [ContainerId: container_1448220437860_0001_01_000001, NodeId: ip-172-31-20-42.ap-northeast-1.compute.internal:36674, NodeHttpAddress: ip-172-31-20-42.ap-northeast-1.compute.internal:47331, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.31.20.42:36674 }, ] for AM appattempt_1448220437860_0001_000001 2015-11-22 19:27:24,511 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220437860_0001_000001 State change from ALLOCATED to LAUNCHED 2015-11-22 19:27:24,514 INFO [LocalizerRunner for container_1448220437860_0001_01_000001] localizer.ResourceLocalizationService (ResourceLocalizationService.java:writeCredentials(1221)) - Writing credentials to the nmPrivate file /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/container_1448220437860_0001_01_000001.tokens. Credentials list: 2015-11-22 19:27:24,515 INFO [LocalizerRunner for container_1448220437860_0001_01_000001] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:createUserCacheDirs(632)) - Initializing user ubuntu 2015-11-22 19:27:24,523 INFO [LocalizerRunner for container_1448220437860_0001_01_000001] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:startLocalizer(126)) - Copying from /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/container_1448220437860_0001_01_000001.tokens to /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220437860_0001/container_1448220437860_0001_01_000001.tokens 2015-11-22 19:27:24,523 INFO [LocalizerRunner for container_1448220437860_0001_01_000001] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:startLocalizer(133)) - Localizer CWD set to /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220437860_0001 = file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220437860_0001 2015-11-22 19:27:24,559 INFO [IPC Server handler 0 on 47867] localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource file:/home/ubuntu/DistributedShell/application_1448220437860_0001/AppMaster.jar(->/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220437860_0001/filecache/10/AppMaster.jar) transitioned from DOWNLOADING to LOCALIZED 2015-11-22 19:27:24,575 INFO [IPC Server handler 3 on 47867] localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource file:/home/ubuntu/DistributedShell/application_1448220437860_0001/shellCommands(->/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220437860_0001/filecache/11/shellCommands) transitioned from DOWNLOADING to LOCALIZED 2015-11-22 19:27:24,575 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220437860_0001_01_000001 transitioned from LOCALIZING to LOCALIZED 2015-11-22 19:27:24,593 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220437860_0001_01_000001 transitioned from LOCALIZED to RUNNING 2015-11-22 19:27:24,609 INFO [ContainersLauncher #0] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:buildCommandExecutor(281)) - launchContainer: [nice, -n, 0, bash, /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220437860_0001/container_1448220437860_0001_01_000001/default_container_executor.sh] 2015-11-22 19:27:25,184 INFO [Thread-3308] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=N/A, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220443979, yarnAppState=ACCEPTED, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:44490/proxy/application_1448220437860_0001/, appUser=ubuntu 2015-11-22 19:27:25,494 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220437860_0001_01_000001 Container Transitioned from ACQUIRED to RUNNING 2015-11-22 19:27:26,185 INFO [Thread-3308] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=N/A, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220443979, yarnAppState=ACCEPTED, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:44490/proxy/application_1448220437860_0001/, appUser=ubuntu 2015-11-22 19:27:27,059 INFO [Socket Reader #1 for port 60164] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220437860_0001_000001 (auth:SIMPLE) 2015-11-22 19:27:27,067 INFO [IPC Server handler 6 on 60164] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:registerApplicationMaster(220)) - AM registration appattempt_1448220437860_0001_000001 2015-11-22 19:27:27,067 INFO [IPC Server handler 6 on 60164] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(133)) - USER=ubuntu IP=172.31.20.42 OPERATION=Register App Master TARGET=ApplicationMasterService RESULT=SUCCESS APPID=application_1448220437860_0001 APPATTEMPTID=appattempt_1448220437860_0001_000001 2015-11-22 19:27:27,068 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220437860_0001_000001 State change from LAUNCHED to RUNNING 2015-11-22 19:27:27,069 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220437860_0001 State change from ACCEPTED to RUNNING 2015-11-22 19:27:27,187 INFO [Thread-3308] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220443979, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:44490/proxy/application_1448220437860_0001/, appUser=ubuntu 2015-11-22 19:27:28,188 INFO [Thread-3308] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220443979, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:44490/proxy/application_1448220437860_0001/, appUser=ubuntu 2015-11-22 19:27:28,496 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220437860_0001_01_000002 Container Transitioned from NEW to ALLOCATED 2015-11-22 19:27:28,496 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220437860_0001 CONTAINERID=container_1448220437860_0001_01_000002 2015-11-22 19:27:28,496 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(154)) - Assigned container container_1448220437860_0001_01_000002 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:36674, which has 2 containers, used and available after allocation 2015-11-22 19:27:28,497 INFO [ResourceManager Event Processor] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(89)) - assignedContainer application attempt=appattempt_1448220437860_0001_000001 container=container_1448220437860_0001_01_000002 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@6fc89cc2 clusterResource= type=OFF_SWITCH 2015-11-22 19:27:28,497 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(589)) - Re-sorting assigned queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.15625, absoluteUsedCapacity=0.15625, numApps=1, numContainers=2 2015-11-22 19:27:28,497 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(461)) - assignedContainer queue=root usedCapacity=0.15625 absoluteUsedCapacity=0.15625 used= cluster= 2015-11-22 19:27:29,112 INFO [IPC Server handler 2 on 60164] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:createAndGetNMToken(200)) - Sending NMToken for nodeId : ip-172-31-20-42.ap-northeast-1.compute.internal:36674 for container : container_1448220437860_0001_01_000002 2015-11-22 19:27:29,112 INFO [IPC Server handler 2 on 60164] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220437860_0001_01_000002 Container Transitioned from ALLOCATED to ACQUIRED 2015-11-22 19:27:29,185 INFO [Socket Reader #1 for port 36674] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220437860_0001_000001 (auth:SIMPLE) 2015-11-22 19:27:29,189 INFO [IPC Server handler 12 on 36674] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:startContainerInternal(889)) - Start request for container_1448220437860_0001_01_000002 by user ubuntu 2015-11-22 19:27:29,190 INFO [IPC Server handler 12 on 36674] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu IP=172.31.20.42 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1448220437860_0001 CONTAINERID=container_1448220437860_0001_01_000002 2015-11-22 19:27:29,190 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(303)) - Adding container_1448220437860_0001_01_000002 to application application_1448220437860_0001 2015-11-22 19:27:29,190 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220437860_0001_01_000002 transitioned from NEW to LOCALIZED 2015-11-22 19:27:29,190 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_INIT for appId application_1448220437860_0001 2015-11-22 19:27:29,191 INFO [Thread-3308] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220443979, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:44490/proxy/application_1448220437860_0001/, appUser=ubuntu 2015-11-22 19:27:29,222 INFO [Socket Reader #1 for port 36674] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220437860_0001_000001 (auth:SIMPLE) 2015-11-22 19:27:29,231 INFO [IPC Server handler 7 on 36674] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:getContainerStatusInternal(1221)) - Getting container-status for container_1448220437860_0001_01_000002 2015-11-22 19:27:29,231 INFO [IPC Server handler 7 on 36674] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:getContainerStatusInternal(1235)) - Returning ContainerStatus: [ContainerId: container_1448220437860_0001_01_000002, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ] 2015-11-22 19:27:29,250 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220437860_0001_01_000002 transitioned from LOCALIZED to RUNNING 2015-11-22 19:27:29,288 INFO [ContainersLauncher #1] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:buildCommandExecutor(281)) - launchContainer: [nice, -n, 0, bash, /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220437860_0001/container_1448220437860_0001_01_000002/default_container_executor.sh] 2015-11-22 19:27:29,496 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220437860_0001_01_000002 Container Transitioned from ACQUIRED to RUNNING 2015-11-22 19:27:30,106 WARN [ContainersLauncher #0] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(237)) - Exit code from container container_1448220437860_0001_01_000001 is : 100 2015-11-22 19:27:30,106 WARN [ContainersLauncher #0] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(243)) - Exception from container-launch with container ID: container_1448220437860_0001_01_000001 and exit code: 100 ExitCodeException exitCode=100: at org.apache.hadoop.util.Shell.runCommand(Shell.java:909) at org.apache.hadoop.util.Shell.run(Shell.java:820) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1099) at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:225) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:304) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:84) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 2015-11-22 19:27:30,106 INFO [ContainersLauncher #0] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - Exception from container-launch. 2015-11-22 19:27:30,106 INFO [ContainersLauncher #0] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - Container id: container_1448220437860_0001_01_000001 2015-11-22 19:27:30,106 INFO [ContainersLauncher #0] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - Exit code: 100 2015-11-22 19:27:30,106 INFO [ContainersLauncher #0] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - Stack trace: ExitCodeException exitCode=100: 2015-11-22 19:27:30,106 INFO [ContainersLauncher #0] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - at org.apache.hadoop.util.Shell.runCommand(Shell.java:909) 2015-11-22 19:27:30,107 INFO [ContainersLauncher #0] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - at org.apache.hadoop.util.Shell.run(Shell.java:820) 2015-11-22 19:27:30,107 INFO [ContainersLauncher #0] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1099) 2015-11-22 19:27:30,107 INFO [ContainersLauncher #0] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:225) 2015-11-22 19:27:30,107 INFO [ContainersLauncher #0] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:304) 2015-11-22 19:27:30,107 INFO [ContainersLauncher #0] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:84) 2015-11-22 19:27:30,107 INFO [ContainersLauncher #0] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - at java.util.concurrent.FutureTask.run(FutureTask.java:266) 2015-11-22 19:27:30,107 INFO [ContainersLauncher #0] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 2015-11-22 19:27:30,107 INFO [ContainersLauncher #0] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 2015-11-22 19:27:30,107 INFO [ContainersLauncher #0] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - at java.lang.Thread.run(Thread.java:745) 2015-11-22 19:27:30,107 WARN [ContainersLauncher #0] launcher.ContainerLaunch (ContainerLaunch.java:call(348)) - Container exited with a non-zero exit code 100 2015-11-22 19:27:30,107 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220437860_0001_01_000001 transitioned from RUNNING to EXITED_WITH_FAILURE 2015-11-22 19:27:30,108 INFO [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(380)) - Cleaning up container container_1448220437860_0001_01_000001 2015-11-22 19:27:30,121 INFO [DeletionService #0] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(480)) - Deleting absolute path : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220437860_0001/container_1448220437860_0001_01_000001 2015-11-22 19:27:30,121 WARN [AsyncDispatcher event handler] nodemanager.NMAuditLogger (NMAuditLogger.java:logFailure(150)) - USER=ubuntu OPERATION=Container Finished - Failed TARGET=ContainerImpl RESULT=FAILURE DESCRIPTION=Container failed with state: EXITED_WITH_FAILURE APPID=application_1448220437860_0001 CONTAINERID=container_1448220437860_0001_01_000001 2015-11-22 19:27:30,121 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220437860_0001_01_000001 transitioned from EXITED_WITH_FAILURE to DONE 2015-11-22 19:27:30,121 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(346)) - Removing container_1448220437860_0001_01_000001 from application application_1448220437860_0001 2015-11-22 19:27:30,121 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_STOP for appId application_1448220437860_0001 2015-11-22 19:27:30,192 INFO [Thread-3308] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220443979, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:44490/proxy/application_1448220437860_0001/, appUser=ubuntu 2015-11-22 19:27:30,496 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220437860_0001_01_000001 Container Transitioned from RUNNING to COMPLETED 2015-11-22 19:27:30,496 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220437860_0001 CONTAINERID=container_1448220437860_0001_01_000001 2015-11-22 19:27:30,496 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:releaseContainer(248)) - Released container container_1448220437860_0001_01_000001 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:36674, which currently has 1 containers, used and available, release resources=true 2015-11-22 19:27:30,497 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:rememberTargetTransitionsAndStoreState(1258)) - Updating application attempt appattempt_1448220437860_0001_000001 with final state: FAILED, and exit status: 100 2015-11-22 19:27:30,497 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220437860_0001_000001 State change from RUNNING to FINAL_SAVING 2015-11-22 19:27:30,497 INFO [AsyncDispatcher event handler] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:unregisterAttempt(667)) - Unregistering app attempt : appattempt_1448220437860_0001_000001 2015-11-22 19:27:30,497 INFO [AsyncDispatcher event handler] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:applicationMasterFinished(124)) - Application finished, removing password for appattempt_1448220437860_0001_000001 2015-11-22 19:27:30,497 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220437860_0001_000001 State change from FINAL_SAVING to FAILED 2015-11-22 19:27:30,497 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:transition(1302)) - The number of failed attempts is 1. The max attempts is 2 2015-11-22 19:27:30,497 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220437860_0001 State change from RUNNING to ACCEPTED 2015-11-22 19:27:30,498 INFO [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:doneApplicationAttempt(844)) - Application Attempt appattempt_1448220437860_0001_000001 is done. finalState=FAILED 2015-11-22 19:27:30,498 INFO [AsyncDispatcher event handler] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:registerAppAttempt(661)) - Registering app attempt : appattempt_1448220437860_0001_000002 2015-11-22 19:27:30,498 INFO [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:doneApplicationAttempt(862)) - Skip killing container_1448220437860_0001_01_000002 2015-11-22 19:27:30,498 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220437860_0001_000002 State change from NEW to SUBMITTED 2015-11-22 19:27:30,498 INFO [ResourceManager Event Processor] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:clearRequests(125)) - Application application_1448220437860_0001 requests cleared 2015-11-22 19:27:30,498 INFO [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:removeApplicationAttempt(772)) - Application removed - appId: application_1448220437860_0001 user: ubuntu queue: default #user-pending-applications: 0 #user-active-applications: 0 #queue-pending-applications: 0 #queue-active-applications: 0 2015-11-22 19:27:30,498 INFO [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:activateApplications(705)) - Application application_1448220437860_0001 from user: ubuntu activated in queue: default 2015-11-22 19:27:30,498 INFO [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:addApplicationAttempt(720)) - Application added - appId: application_1448220437860_0001 user: ubuntu, leaf-queue: default #user-pending-applications: 0 #user-active-applications: 1 #queue-pending-applications: 0 #queue-active-applications: 1 2015-11-22 19:27:30,498 INFO [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:addApplicationAttempt(805)) - Added Application Attempt appattempt_1448220437860_0001_000002 to scheduler from user ubuntu in queue default 2015-11-22 19:27:30,499 INFO [ApplicationMasterLauncher #1] amlauncher.AMLauncher (AMLauncher.java:run(268)) - Cleaning master appattempt_1448220437860_0001_000001 2015-11-22 19:27:30,499 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220437860_0001_000002 State change from SUBMITTED to SCHEDULED 2015-11-22 19:27:30,503 INFO [Socket Reader #1 for port 36674] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220437860_0001_000001 (auth:SIMPLE) 2015-11-22 19:27:30,506 INFO [IPC Server handler 17 on 36674] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:stopContainerInternal(1163)) - Stopping container with container Id: container_1448220437860_0001_01_000001 2015-11-22 19:27:30,506 INFO [IPC Server handler 17 on 36674] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu IP=172.31.20.42 OPERATION=Stop Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1448220437860_0001 CONTAINERID=container_1448220437860_0001_01_000001 2015-11-22 19:27:30,507 INFO [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:removeOrTrackCompletedContainersFromContext(605)) - Removed completed containers from NM context: [container_1448220437860_0001_01_000001] 2015-11-22 19:27:30,507 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220437860_0001_02_000001 Container Transitioned from NEW to ALLOCATED 2015-11-22 19:27:30,507 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220437860_0001 CONTAINERID=container_1448220437860_0001_02_000001 2015-11-22 19:27:30,508 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(154)) - Assigned container container_1448220437860_0001_02_000001 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:36674, which has 2 containers, used and available after allocation 2015-11-22 19:27:30,508 INFO [ResourceManager Event Processor] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(89)) - assignedContainer application attempt=appattempt_1448220437860_0001_000002 container=container_1448220437860_0001_02_000001 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@610f6f4b clusterResource= type=OFF_SWITCH 2015-11-22 19:27:30,508 INFO [AsyncDispatcher event handler] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:createAndGetNMToken(200)) - Sending NMToken for nodeId : ip-172-31-20-42.ap-northeast-1.compute.internal:36674 for container : container_1448220437860_0001_02_000001 2015-11-22 19:27:30,509 INFO [AsyncDispatcher event handler] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220437860_0001_02_000001 Container Transitioned from ALLOCATED to ACQUIRED 2015-11-22 19:27:30,509 INFO [AsyncDispatcher event handler] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:clearNodeSetForAttempt(146)) - Clear node set for appattempt_1448220437860_0001_000002 2015-11-22 19:27:30,509 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:storeAttempt(2026)) - Storing attempt: AppId: application_1448220437860_0001 AttemptId: appattempt_1448220437860_0001_000002 MasterContainer: Container: [ContainerId: container_1448220437860_0001_02_000001, NodeId: ip-172-31-20-42.ap-northeast-1.compute.internal:36674, NodeHttpAddress: ip-172-31-20-42.ap-northeast-1.compute.internal:47331, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.31.20.42:36674 }, ] 2015-11-22 19:27:30,509 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220437860_0001_000002 State change from SCHEDULED to ALLOCATED_SAVING 2015-11-22 19:27:30,510 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220437860_0001_000002 State change from ALLOCATED_SAVING to ALLOCATED 2015-11-22 19:27:30,510 INFO [ApplicationMasterLauncher #2] amlauncher.AMLauncher (AMLauncher.java:run(254)) - Launching masterappattempt_1448220437860_0001_000002 2015-11-22 19:27:30,511 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(589)) - Re-sorting assigned queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.15625, absoluteUsedCapacity=0.15625, numApps=1, numContainers=2 2015-11-22 19:27:30,512 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(461)) - assignedContainer queue=root usedCapacity=0.15625 absoluteUsedCapacity=0.15625 used= cluster= 2015-11-22 19:27:30,513 INFO [ApplicationMasterLauncher #2] amlauncher.AMLauncher (AMLauncher.java:launch(106)) - Setting up container Container: [ContainerId: container_1448220437860_0001_02_000001, NodeId: ip-172-31-20-42.ap-northeast-1.compute.internal:36674, NodeHttpAddress: ip-172-31-20-42.ap-northeast-1.compute.internal:47331, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.31.20.42:36674 }, ] for AM appattempt_1448220437860_0001_000002 2015-11-22 19:27:30,513 INFO [ApplicationMasterLauncher #2] amlauncher.AMLauncher (AMLauncher.java:createAMContainerLaunchContext(188)) - Command to launch container container_1448220437860_0001_02_000001 : {{JAVA_HOME}}/bin/java -Xmx512m org.apache.hadoop.yarn.applications.distributedshell.TestDSFailedAppMaster --container_memory 128 --container_vcores 1 --num_containers 1 --priority 0 1>/AppMaster.stdout 2>/AppMaster.stderr 2015-11-22 19:27:30,513 INFO [ApplicationMasterLauncher #2] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:createAndGetAMRMToken(195)) - Create AMRMToken for ApplicationAttempt: appattempt_1448220437860_0001_000002 2015-11-22 19:27:30,513 INFO [ApplicationMasterLauncher #2] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:createPassword(307)) - Creating password for appattempt_1448220437860_0001_000002 2015-11-22 19:27:30,515 INFO [Socket Reader #1 for port 36674] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220437860_0001_000002 (auth:SIMPLE) 2015-11-22 19:27:30,518 INFO [IPC Server handler 12 on 36674] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:startContainerInternal(889)) - Start request for container_1448220437860_0001_02_000001 by user ubuntu 2015-11-22 19:27:30,518 INFO [IPC Server handler 12 on 36674] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu IP=172.31.20.42 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1448220437860_0001 CONTAINERID=container_1448220437860_0001_02_000001 2015-11-22 19:27:30,518 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(303)) - Adding container_1448220437860_0001_02_000001 to application application_1448220437860_0001 2015-11-22 19:27:30,518 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220437860_0001_02_000001 transitioned from NEW to LOCALIZING 2015-11-22 19:27:30,518 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_INIT for appId application_1448220437860_0001 2015-11-22 19:27:30,519 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220437860_0001_02_000001 transitioned from LOCALIZING to LOCALIZED 2015-11-22 19:27:30,520 INFO [ApplicationMasterLauncher #2] amlauncher.AMLauncher (AMLauncher.java:launch(127)) - Done launching container Container: [ContainerId: container_1448220437860_0001_02_000001, NodeId: ip-172-31-20-42.ap-northeast-1.compute.internal:36674, NodeHttpAddress: ip-172-31-20-42.ap-northeast-1.compute.internal:47331, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.31.20.42:36674 }, ] for AM appattempt_1448220437860_0001_000002 2015-11-22 19:27:30,520 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220437860_0001_000002 State change from ALLOCATED to LAUNCHED 2015-11-22 19:27:30,536 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220437860_0001_02_000001 transitioned from LOCALIZED to RUNNING 2015-11-22 19:27:30,552 INFO [ContainersLauncher #0] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:buildCommandExecutor(281)) - launchContainer: [nice, -n, 0, bash, /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220437860_0001/container_1448220437860_0001_02_000001/default_container_executor.sh] 2015-11-22 19:27:31,194 INFO [Thread-3308] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=N/A, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220443979, yarnAppState=ACCEPTED, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:44490/proxy/application_1448220437860_0001/, appUser=ubuntu 2015-11-22 19:27:31,507 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220437860_0001_02_000001 Container Transitioned from ACQUIRED to RUNNING 2015-11-22 19:27:32,196 INFO [Thread-3308] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=N/A, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220443979, yarnAppState=ACCEPTED, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:44490/proxy/application_1448220437860_0001/, appUser=ubuntu 2015-11-22 19:27:33,032 INFO [Socket Reader #1 for port 60164] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220437860_0001_000002 (auth:SIMPLE) 2015-11-22 19:27:33,041 INFO [IPC Server handler 6 on 60164] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:registerApplicationMaster(220)) - AM registration appattempt_1448220437860_0001_000002 2015-11-22 19:27:33,041 INFO [IPC Server handler 6 on 60164] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(133)) - USER=ubuntu IP=172.31.20.42 OPERATION=Register App Master TARGET=ApplicationMasterService RESULT=SUCCESS APPID=application_1448220437860_0001 APPATTEMPTID=appattempt_1448220437860_0001_000002 2015-11-22 19:27:33,041 INFO [IPC Server handler 6 on 60164] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:createAndGetNMToken(200)) - Sending NMToken for nodeId : ip-172-31-20-42.ap-northeast-1.compute.internal:36674 for container : container_1448220437860_0001_01_000002 2015-11-22 19:27:33,041 INFO [IPC Server handler 6 on 60164] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:registerApplicationMaster(272)) - Application application_1448220437860_0001 retrieved 1 containers from previous attempts and 1 NM tokens. 2015-11-22 19:27:33,042 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220437860_0001_000002 State change from LAUNCHED to RUNNING 2015-11-22 19:27:33,042 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220437860_0001 State change from ACCEPTED to RUNNING 2015-11-22 19:27:33,198 INFO [Thread-3308] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220443979, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:44490/proxy/application_1448220437860_0001/, appUser=ubuntu 2015-11-22 19:27:34,199 INFO [Thread-3308] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220443979, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:44490/proxy/application_1448220437860_0001/, appUser=ubuntu 2015-11-22 19:27:35,200 INFO [Thread-3308] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220443979, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:44490/proxy/application_1448220437860_0001/, appUser=ubuntu 2015-11-22 19:27:36,202 INFO [Thread-3308] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220443979, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:44490/proxy/application_1448220437860_0001/, appUser=ubuntu 2015-11-22 19:27:37,203 INFO [Thread-3308] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220443979, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:44490/proxy/application_1448220437860_0001/, appUser=ubuntu 2015-11-22 19:27:37,298 INFO [ContainersLauncher #1] launcher.ContainerLaunch (ContainerLaunch.java:call(356)) - Container container_1448220437860_0001_01_000002 succeeded 2015-11-22 19:27:37,299 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220437860_0001_01_000002 transitioned from RUNNING to EXITED_WITH_SUCCESS 2015-11-22 19:27:37,299 INFO [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(380)) - Cleaning up container container_1448220437860_0001_01_000002 2015-11-22 19:27:37,312 INFO [DeletionService #0] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(480)) - Deleting absolute path : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220437860_0001/container_1448220437860_0001_01_000002 2015-11-22 19:27:37,312 INFO [AsyncDispatcher event handler] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu OPERATION=Container Finished - Succeeded TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1448220437860_0001 CONTAINERID=container_1448220437860_0001_01_000002 2015-11-22 19:27:37,312 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220437860_0001_01_000002 transitioned from EXITED_WITH_SUCCESS to DONE 2015-11-22 19:27:37,312 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(346)) - Removing container_1448220437860_0001_01_000002 from application application_1448220437860_0001 2015-11-22 19:27:37,313 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_STOP for appId application_1448220437860_0001 2015-11-22 19:27:37,510 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220437860_0001_01_000002 Container Transitioned from RUNNING to COMPLETED 2015-11-22 19:27:37,510 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220437860_0001 CONTAINERID=container_1448220437860_0001_01_000002 2015-11-22 19:27:37,510 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:releaseContainer(248)) - Released container container_1448220437860_0001_01_000002 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:36674, which currently has 1 containers, used and available, release resources=true 2015-11-22 19:27:38,205 INFO [Thread-3308] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220443979, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:44490/proxy/application_1448220437860_0001/, appUser=ubuntu 2015-11-22 19:27:38,300 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:rememberTargetTransitionsAndStoreState(1258)) - Updating application attempt appattempt_1448220437860_0001_000002 with final state: FINISHING, and exit status: -1000 2015-11-22 19:27:38,300 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220437860_0001_000002 State change from RUNNING to FINAL_SAVING 2015-11-22 19:27:38,300 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:rememberTargetTransitionsAndStoreState(1098)) - Updating application application_1448220437860_0001 with final state: FINISHING 2015-11-22 19:27:38,300 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220437860_0001 State change from RUNNING to FINAL_SAVING 2015-11-22 19:27:38,300 INFO [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:transition(237)) - Updating info for app: application_1448220437860_0001 2015-11-22 19:27:38,301 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220437860_0001_000002 State change from FINAL_SAVING to FINISHING 2015-11-22 19:27:38,301 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220437860_0001 State change from FINAL_SAVING to FINISHING 2015-11-22 19:27:38,402 INFO [IPC Server handler 13 on 60164] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:finishApplicationMaster(300)) - application_1448220437860_0001 unregistered successfully. 2015-11-22 19:27:38,519 INFO [ContainersLauncher #0] launcher.ContainerLaunch (ContainerLaunch.java:call(356)) - Container container_1448220437860_0001_02_000001 succeeded 2015-11-22 19:27:38,519 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220437860_0001_02_000001 transitioned from RUNNING to EXITED_WITH_SUCCESS 2015-11-22 19:27:38,519 INFO [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(380)) - Cleaning up container container_1448220437860_0001_02_000001 2015-11-22 19:27:38,532 INFO [DeletionService #1] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(480)) - Deleting absolute path : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220437860_0001/container_1448220437860_0001_02_000001 2015-11-22 19:27:38,533 INFO [AsyncDispatcher event handler] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu OPERATION=Container Finished - Succeeded TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1448220437860_0001 CONTAINERID=container_1448220437860_0001_02_000001 2015-11-22 19:27:38,533 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220437860_0001_02_000001 transitioned from EXITED_WITH_SUCCESS to DONE 2015-11-22 19:27:38,533 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(346)) - Removing container_1448220437860_0001_02_000001 from application application_1448220437860_0001 2015-11-22 19:27:38,533 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_STOP for appId application_1448220437860_0001 2015-11-22 19:27:39,206 INFO [Thread-3308] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220443979, yarnAppState=FINISHED, distributedFinalState=SUCCEEDED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:44490/proxy/application_1448220437860_0001/, appUser=ubuntu 2015-11-22 19:27:39,206 INFO [Thread-3308] distributedshell.Client (Client.java:monitorApplication(759)) - Application has completed successfully. Breaking monitoring loop 2015-11-22 19:27:39,206 INFO [Thread-3308] distributedshell.TestDistributedShell (TestDistributedShell.java:testDSRestartWithPreviousRunningContainers(368)) - Client run completed. Result=true 2015-11-22 19:27:39,209 ERROR [Thread[Thread-3304,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(670)) - ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted 2015-11-22 19:27:39,211 INFO [main] mortbay.log (Slf4jLog.java:info(67)) - Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:0 2015-11-22 19:27:39,313 INFO [main] applicationhistoryservice.ApplicationHistoryManagerImpl (ApplicationHistoryManagerImpl.java:serviceStop(78)) - Stopping ApplicationHistory 2015-11-22 19:27:39,313 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 37547 2015-11-22 19:27:39,314 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:27:39,314 ERROR [Thread[Thread-3286,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(670)) - ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted 2015-11-22 19:27:39,315 INFO [IPC Server listener on 37547] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 37547 2015-11-22 19:27:39,316 INFO [main] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:unRegisterNM(268)) - Successfully Unregistered the Node ip-172-31-20-42.ap-northeast-1.compute.internal:36674 with ResourceManager. 2015-11-22 19:27:39,320 ERROR [Thread[Thread-3279,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(670)) - ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted 2015-11-22 19:27:39,322 INFO [main] mortbay.log (Slf4jLog.java:info(67)) - Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:0 2015-11-22 19:27:39,424 INFO [main] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:cleanUpApplicationsOnNMShutDown(556)) - Applications still running : [application_1448220437860_0001] 2015-11-22 19:27:39,424 INFO [main] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:cleanUpApplicationsOnNMShutDown(572)) - Waiting for Applications to be Finished 2015-11-22 19:27:39,424 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(463)) - Application application_1448220437860_0001 transitioned from RUNNING to APPLICATION_RESOURCES_CLEANINGUP 2015-11-22 19:27:39,425 INFO [DeletionService #2] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(480)) - Deleting absolute path : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220437860_0001 2015-11-22 19:27:39,425 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event APPLICATION_STOP for appId application_1448220437860_0001 2015-11-22 19:27:39,425 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(463)) - Application application_1448220437860_0001 transitioned from APPLICATION_RESOURCES_CLEANINGUP to FINISHED 2015-11-22 19:27:39,425 INFO [AsyncDispatcher event handler] loghandler.NonAggregatingLogHandler (NonAggregatingLogHandler.java:handle(167)) - Scheduling Log Deletion for application: application_1448220437860_0001, with delay of 10800 seconds 2015-11-22 19:27:43,425 INFO [main] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:cleanUpApplicationsOnNMShutDown(589)) - Done waiting for Applications to be Finished. Still alive: [application_1448220437860_0001] 2015-11-22 19:27:43,426 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 36674 2015-11-22 19:27:43,429 INFO [IPC Server listener on 36674] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 36674 2015-11-22 19:27:43,430 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:27:53,432 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 47867 2015-11-22 19:27:53,433 INFO [IPC Server listener on 47867] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 47867 2015-11-22 19:27:53,433 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:27:53,434 INFO [Public Localizer] localizer.ResourceLocalizationService (ResourceLocalizationService.java:run(908)) - Public cache exiting 2015-11-22 19:27:53,434 WARN [Node Resource Monitor] nodemanager.NodeResourceMonitorImpl (NodeResourceMonitorImpl.java:run(154)) - org.apache.hadoop.yarn.server.nodemanager.NodeResourceMonitorImpl is interrupted. Exiting. 2015-11-22 19:27:58,438 WARN [main] server.MiniYARNCluster (MiniYARNCluster.java:waitForAppMastersToFinish(474)) - Stopping RM while some app masters are still alive 2015-11-22 19:27:58,441 ERROR [Thread[Thread-3229,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(670)) - ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted 2015-11-22 19:27:58,442 INFO [main] mortbay.log (Slf4jLog.java:info(67)) - Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:0 2015-11-22 19:27:58,545 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 34580 2015-11-22 19:27:58,546 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:27:58,547 INFO [IPC Server listener on 34580] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 34580 2015-11-22 19:27:58,547 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 42921 2015-11-22 19:27:58,549 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:27:58,549 INFO [IPC Server listener on 42921] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 42921 2015-11-22 19:27:58,550 INFO [main] resourcemanager.ResourceManager (ResourceManager.java:transitionToStandby(1051)) - Transitioning to standby state 2015-11-22 19:27:58,551 WARN [ApplicationMaster Launcher] amlauncher.ApplicationMasterLauncher (ApplicationMasterLauncher.java:run(122)) - org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher$LauncherThread interrupted. Returning. 2015-11-22 19:27:58,554 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 60164 2015-11-22 19:27:58,556 INFO [IPC Server listener on 60164] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 60164 2015-11-22 19:27:58,556 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:27:58,556 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 49964 2015-11-22 19:27:58,558 INFO [IPC Server listener on 49964] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 49964 2015-11-22 19:27:58,558 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:27:58,558 ERROR [ResourceManager Event Processor] resourcemanager.ResourceManager (ResourceManager.java:run(691)) - Returning, interrupted : java.lang.InterruptedException 2015-11-22 19:27:58,559 INFO [Ping Checker] util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(135)) - NMLivelinessMonitor thread interrupted 2015-11-22 19:27:58,560 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:serviceStop(141)) - AsyncDispatcher is draining to stop, igonring any new events. 2015-11-22 19:27:58,561 INFO [Ping Checker] util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(135)) - AMLivelinessMonitor thread interrupted 2015-11-22 19:27:58,562 ERROR [Thread[Thread-3060,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(670)) - ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted 2015-11-22 19:27:58,562 INFO [Ping Checker] util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(135)) - org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.ContainerAllocationExpirer thread interrupted 2015-11-22 19:27:58,562 INFO [Ping Checker] util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(135)) - AMLivelinessMonitor thread interrupted 2015-11-22 19:27:58,563 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stop(211)) - Stopping ApplicationHistoryServer metrics system... 2015-11-22 19:27:58,566 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stop(217)) - ApplicationHistoryServer metrics system stopped. 2015-11-22 19:27:58,567 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:shutdown(608)) - ApplicationHistoryServer metrics system shutdown complete. 2015-11-22 19:27:58,567 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:serviceStop(141)) - AsyncDispatcher is draining to stop, igonring any new events. 2015-11-22 19:27:58,567 INFO [main] resourcemanager.ResourceManager (ResourceManager.java:transitionToStandby(1058)) - Transitioned to standby state 2015-11-22 19:27:58,577 INFO [main] distributedshell.TestDistributedShell (TestDistributedShell.java:setupInternal(77)) - Starting up YARN cluster 2015-11-22 19:27:58,587 INFO [main] net.ServerSocketUtil (ServerSocketUtil.java:getPort(52)) - Using port 10021 2015-11-22 19:27:58,598 INFO [main] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2363)) - found resource core-site.xml at file:/home/ubuntu/hadoop-dev/hadoop-common-project/hadoop-common/target/test-classes/core-site.xml 2015-11-22 19:27:58,608 WARN [main] conf.Configuration (Configuration.java:loadProperty(2687)) - core-site.xml:an attempt to override final parameter: hadoop.tmp.dir; Ignoring. 2015-11-22 19:27:58,608 INFO [main] security.Groups (Groups.java:refresh(256)) - clearing userToGroupsMap cache 2015-11-22 19:27:58,609 INFO [main] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2363)) - found resource yarn-site.xml at file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/test-classes/yarn-site.xml 2015-11-22 19:27:58,624 WARN [main] conf.Configuration (Configuration.java:loadProperty(2687)) - yarn-site.xml:an attempt to override final parameter: hadoop.tmp.dir; Ignoring. 2015-11-22 19:27:58,624 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.RMFatalEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMFatalEventDispatcher 2015-11-22 19:27:58,625 INFO [main] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:(75)) - NMTokenKeyRollingInterval: 86400000ms and NMTokenKeyActivationDelay: 900000ms 2015-11-22 19:27:58,625 INFO [main] security.RMContainerTokenSecretManager (RMContainerTokenSecretManager.java:(78)) - ContainerTokenKeyRollingInterval: 86400000ms and ContainerTokenKeyActivationDelay: 900000ms 2015-11-22 19:27:58,625 INFO [main] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:(94)) - AMRMTokenKeyRollingInterval: 86400000ms and AMRMTokenKeyActivationDelay: 900000 ms 2015-11-22 19:27:58,625 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStoreEventType for class org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$ForwardingEventHandler 2015-11-22 19:27:58,625 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.NodesListManagerEventType for class org.apache.hadoop.yarn.server.resourcemanager.NodesListManager 2015-11-22 19:27:58,626 INFO [main] resourcemanager.ResourceManager (ResourceManager.java:createScheduler(295)) - Using Scheduler: org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler 2015-11-22 19:27:58,626 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.SchedulerEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher 2015-11-22 19:27:58,626 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationEventDispatcher 2015-11-22 19:27:58,626 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationAttemptEventDispatcher 2015-11-22 19:27:58,626 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$NodeEventDispatcher 2015-11-22 19:27:58,627 INFO [main] impl.MetricsConfig (MetricsConfig.java:loadFirst(115)) - loaded properties from hadoop-metrics2.properties 2015-11-22 19:27:58,628 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:startTimer(374)) - Scheduled snapshot period at 10 second(s). 2015-11-22 19:27:58,628 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:start(192)) - ResourceManager metrics system started 2015-11-22 19:27:58,629 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.RMAppManagerEventType for class org.apache.hadoop.yarn.server.resourcemanager.RMAppManager 2015-11-22 19:27:58,629 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncherEventType for class org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher 2015-11-22 19:27:58,629 WARN [main] util.MBeans (MBeans.java:register(73)) - Failed to register MBean "Hadoop:service=ResourceManager,name=RMNMInfo": Instance already exists. 2015-11-22 19:27:58,629 INFO [main] resourcemanager.RMNMInfo (RMNMInfo.java:(63)) - Registered RMNMInfo MBean 2015-11-22 19:27:58,629 INFO [main] util.HostsFileReader (HostsFileReader.java:refresh(132)) - Refreshing hosts (include/exclude) list 2015-11-22 19:27:58,630 INFO [main] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2363)) - found resource capacity-scheduler.xml at jar:file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/target/hadoop-yarn-server-tests-2.8.0-SNAPSHOT-tests.jar!/capacity-scheduler.xml 2015-11-22 19:27:58,642 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(647)) - max alloc mb per queue for root is undefined 2015-11-22 19:27:58,642 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(651)) - max alloc vcore per queue for root is undefined 2015-11-22 19:27:58,642 INFO [main] capacity.ParentQueue (ParentQueue.java:setupQueueConfigs(125)) - root, capacity=1.0, asboluteCapacity=1.0, maxCapacity=1.0, asboluteMaxCapacity=1.0, state=RUNNING, acls=ADMINISTER_QUEUE:*SUBMIT_APP:*, labels=*, , reservationsContinueLooking=true 2015-11-22 19:27:58,642 INFO [main] capacity.ParentQueue (ParentQueue.java:(104)) - Initialized parent-queue root name=root, fullname=root 2015-11-22 19:27:58,643 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(647)) - max alloc mb per queue for root.default is undefined 2015-11-22 19:27:58,643 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(651)) - max alloc vcore per queue for root.default is undefined 2015-11-22 19:27:58,644 INFO [main] capacity.LeafQueue (LeafQueue.java:setupQueueConfigs(217)) - Initializing default capacity = 1.0 [= (float) configuredCapacity / 100 ] asboluteCapacity = 1.0 [= parentAbsoluteCapacity * capacity ] maxCapacity = 1.0 [= configuredMaxCapacity ] absoluteMaxCapacity = 1.0 [= 1.0 maximumCapacity undefined, (parentAbsoluteMaxCapacity * maximumCapacity) / 100 otherwise ] userLimit = 100 [= configuredUserLimit ] userLimitFactor = 1.0 [= configuredUserLimitFactor ] maxApplications = 10000 [= configuredMaximumSystemApplicationsPerQueue or (int)(configuredMaximumSystemApplications * absoluteCapacity)] maxApplicationsPerUser = 10000 [= (int)(maxApplications * (userLimit / 100.0f) * userLimitFactor) ] usedCapacity = 0.0 [= usedResourcesMemory / (clusterResourceMemory * absoluteCapacity)] absoluteUsedCapacity = 0.0 [= usedResourcesMemory / clusterResourceMemory] maxAMResourcePerQueuePercent = 0.1 [= configuredMaximumAMResourcePercent ] minimumAllocationFactor = 0.984375 [= (float)(maximumAllocationMemory - minimumAllocationMemory) / maximumAllocationMemory ] maximumAllocation = [= configuredMaxAllocation ] numContainers = 0 [= currentNumContainers ] state = RUNNING [= configuredState ] acls = ADMINISTER_QUEUE:*SUBMIT_APP:* [= configuredAcls ] nodeLocalityDelay = -1 labels=*, nodeLocalityDelay = -1 reservationsContinueLooking = true preemptionDisabled = true defaultAppPriorityPerQueue = 0 2015-11-22 19:27:58,644 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:parseQueue(657)) - Initialized queue: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=0, numContainers=0 2015-11-22 19:27:58,644 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:parseQueue(657)) - Initialized queue: root: numChildQueue= 1, capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.0, numApps=0, numContainers=0 2015-11-22 19:27:58,644 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:initializeQueues(501)) - Initialized root queue root: numChildQueue= 1, capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.0, numApps=0, numContainers=0 2015-11-22 19:27:58,644 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:getUserGroupMappingPlacementRule(449)) - Initialized queue mappings, override: false 2015-11-22 19:27:58,644 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:initScheduler(314)) - Initialized CapacityScheduler with calculator=class org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator, minimumAllocation=<>, maximumAllocation=<>, asynchronousScheduling=false, asyncScheduleInterval=5ms 2015-11-22 19:27:58,645 INFO [main] metrics.SystemMetricsPublisher (SystemMetricsPublisher.java:serviceInit(95)) - YARN system metrics publishing service is not enabled 2015-11-22 19:27:58,645 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEventType for class org.apache.hadoop.yarn.server.MiniYARNCluster$1 2015-11-22 19:27:58,645 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:prepareDirs(558)) - Created localDir in /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0 2015-11-22 19:27:58,645 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:prepareDirs(558)) - Created logDir in /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-logDir-nm-0_0 2015-11-22 19:27:58,646 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceInit(540)) - Starting NM: 0 2015-11-22 19:27:58,655 INFO [main] nodemanager.NodeManager (NodeManager.java:getNodeHealthScriptRunner(255)) - Node Manager health check script is not available or doesn't have execute permission, so not starting the node health script runner. 2015-11-22 19:27:58,656 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ContainerEventDispatcher 2015-11-22 19:27:58,656 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ApplicationEventDispatcher 2015-11-22 19:27:58,656 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizationEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService 2015-11-22 19:27:58,656 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServicesEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices 2015-11-22 19:27:58,656 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl 2015-11-22 19:27:58,656 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncherEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncher 2015-11-22 19:27:58,656 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.ContainerManagerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl 2015-11-22 19:27:58,656 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.NodeManagerEventType for class org.apache.hadoop.yarn.server.MiniYARNCluster$ShortCircuitedNodeManager 2015-11-22 19:27:58,656 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:init(159)) - NodeManager metrics system started (again) 2015-11-22 19:27:58,657 INFO [main] nodemanager.NodeResourceMonitorImpl (NodeResourceMonitorImpl.java:serviceInit(76)) - Using ResourceCalculatorPlugin : org.apache.hadoop.yarn.util.ResourceCalculatorPlugin@2f8dad04 2015-11-22 19:27:58,657 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.event.LogHandlerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.NonAggregatingLogHandler 2015-11-22 19:27:58,658 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.sharedcache.SharedCacheUploadEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.sharedcache.SharedCacheUploadService 2015-11-22 19:27:58,658 INFO [main] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:serviceInit(261)) - AMRMProxyService is disabled 2015-11-22 19:27:58,658 INFO [main] localizer.ResourceLocalizationService (ResourceLocalizationService.java:validateConf(224)) - per directory file limit = 8192 2015-11-22 19:27:58,668 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerTracker 2015-11-22 19:27:58,668 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(116)) - Using ResourceCalculatorPlugin : org.apache.hadoop.yarn.util.ResourceCalculatorPlugin@29e495ff 2015-11-22 19:27:58,669 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(121)) - Using ResourceCalculatorProcessTree : null 2015-11-22 19:27:58,669 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(158)) - Physical memory check enabled: false 2015-11-22 19:27:58,669 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(159)) - Virtual memory check enabled: false 2015-11-22 19:27:58,669 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:isEnabled(206)) - Neither virutal-memory nor physical-memory monitoring is needed. Not running the monitor-thread 2015-11-22 19:27:58,669 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(162)) - ContainersMonitor enabled: false 2015-11-22 19:27:58,669 INFO [main] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:serviceInit(175)) - Nodemanager resources: memory set to 4096MB. 2015-11-22 19:27:58,669 INFO [main] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:serviceInit(176)) - Nodemanager resources: vcores set to 8. 2015-11-22 19:27:58,669 INFO [main] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:serviceInit(208)) - Initialized nodemanager with : physical-memory=4096 virtual-memory=8602 virtual-cores=8 2015-11-22 19:27:58,669 WARN [main] applicationhistoryservice.ApplicationHistoryServer (ApplicationHistoryServer.java:createApplicationHistoryManager(218)) - The filesystem based application history store is deprecated. 2015-11-22 19:27:58,670 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:init(159)) - ApplicationHistoryServer metrics system started (again) 2015-11-22 19:27:58,670 INFO [main] applicationhistoryservice.ApplicationHistoryManagerImpl (ApplicationHistoryManagerImpl.java:serviceInit(61)) - ApplicationHistory Init 2015-11-22 19:27:58,670 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:startResourceManager(326)) - Waiting for RM to start... 2015-11-22 19:27:58,670 INFO [RM-0] resourcemanager.ResourceManager (ResourceManager.java:transitionToActive(1024)) - Transitioning to active state 2015-11-22 19:27:58,671 INFO [RM-0] recovery.RMStateStore (RMStateStore.java:transition(482)) - Updating AMRMToken 2015-11-22 19:27:58,671 INFO [RM-0] security.RMContainerTokenSecretManager (RMContainerTokenSecretManager.java:rollMasterKey(106)) - Rolling master-key for container-tokens 2015-11-22 19:27:58,671 INFO [RM-0] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:rollMasterKey(95)) - Rolling master-key for nm-tokens 2015-11-22 19:27:58,672 INFO [RM-0] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:27:58,672 INFO [RM-0] security.RMDelegationTokenSecretManager (RMDelegationTokenSecretManager.java:storeNewMasterKey(87)) - storing master key with keyID 1 2015-11-22 19:27:58,672 INFO [RM-0] recovery.RMStateStore (RMStateStore.java:transition(433)) - Storing RMDTMasterKey. 2015-11-22 19:27:58,672 INFO [Thread[Thread-3436,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2015-11-22 19:27:58,672 INFO [Thread[Thread-3436,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:27:58,673 INFO [Thread[Thread-3436,5,main]] security.RMDelegationTokenSecretManager (RMDelegationTokenSecretManager.java:storeNewMasterKey(87)) - storing master key with keyID 2 2015-11-22 19:27:58,673 INFO [Thread[Thread-3436,5,main]] recovery.RMStateStore (RMStateStore.java:transition(433)) - Storing RMDTMasterKey. 2015-11-22 19:27:58,674 INFO [RM-0] nodelabels.CommonNodeLabelsManager (CommonNodeLabelsManager.java:addToCluserNodeLabels(309)) - Add labels: [] 2015-11-22 19:27:58,674 INFO [RM-0] nodelabels.CommonNodeLabelsManager (CommonNodeLabelsManager.java:internalUpdateLabelsOnNodes(638)) - REPLACE labels on nodes: 2015-11-22 19:27:58,674 INFO [RM-0] nodelabels.CommonNodeLabelsManager (CommonNodeLabelsManager.java:internalUpdateLabelsOnNodes(640)) - NM=ip-172-31-20-42.ap-northeast-1.compute.internal:58694, labels=[x] 2015-11-22 19:27:58,675 INFO [RM-0] nodelabels.FileSystemNodeLabelsStore (FileSystemNodeLabelsStore.java:recover(279)) - Finished write mirror at:file:/tmp/hadoop-yarn-ubuntu/node-labels/nodelabel.mirror 2015-11-22 19:27:58,675 INFO [RM-0] nodelabels.FileSystemNodeLabelsStore (FileSystemNodeLabelsStore.java:recover(280)) - Finished create editlog file at:file:/tmp/hadoop-yarn-ubuntu/node-labels/nodelabel.editlog 2015-11-22 19:27:58,675 INFO [RM-0] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.nodelabels.event.NodeLabelsStoreEventType for class org.apache.hadoop.yarn.nodelabels.CommonNodeLabelsManager$ForwardingEventHandler 2015-11-22 19:27:58,676 INFO [RM-0] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:27:58,676 INFO [Socket Reader #1 for port 48629] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 48629 2015-11-22 19:27:58,677 INFO [RM-0] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.server.api.ResourceTrackerPB to the server 2015-11-22 19:27:58,678 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:27:58,678 INFO [IPC Server listener on 48629] ipc.Server (Server.java:run(794)) - IPC Server listener on 48629: starting 2015-11-22 19:27:58,682 INFO [RM-0] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:27:58,683 INFO [Socket Reader #1 for port 46284] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 46284 2015-11-22 19:27:58,684 INFO [RM-0] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB to the server 2015-11-22 19:27:58,684 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:27:58,684 INFO [IPC Server listener on 46284] ipc.Server (Server.java:run(794)) - IPC Server listener on 46284: starting 2015-11-22 19:27:58,689 INFO [RM-0] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:27:58,689 INFO [Socket Reader #1 for port 52028] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 52028 2015-11-22 19:27:58,690 INFO [RM-0] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.api.ApplicationClientProtocolPB to the server 2015-11-22 19:27:58,691 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:27:58,691 INFO [IPC Server listener on 52028] ipc.Server (Server.java:run(794)) - IPC Server listener on 52028: starting 2015-11-22 19:27:58,701 INFO [RM-0] resourcemanager.ResourceManager (ResourceManager.java:transitionToActive(1040)) - Transitioned to active state 2015-11-22 19:27:58,702 INFO [RM-0] server.AuthenticationFilter (AuthenticationFilter.java:constructSecretProvider(304)) - Unable to initialize FileSignerSecretProvider, falling back to use random secrets. 2015-11-22 19:27:58,702 INFO [RM-0] http.HttpRequestLog (HttpRequestLog.java:getRequestLog(80)) - Http request log for http.requests.resourcemanager is not defined 2015-11-22 19:27:58,702 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter) 2015-11-22 19:27:58,703 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context cluster 2015-11-22 19:27:58,703 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context logs 2015-11-22 19:27:58,703 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context static 2015-11-22 19:27:58,703 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context cluster 2015-11-22 19:27:58,703 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2015-11-22 19:27:58,703 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2015-11-22 19:27:58,704 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'Timeline Authentication Filter' (class=org.apache.hadoop.yarn.server.timeline.security.TimelineAuthenticationFilter) 2015-11-22 19:27:58,704 INFO [RM-0] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /cluster/* 2015-11-22 19:27:58,704 INFO [RM-0] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /ws/* 2015-11-22 19:27:58,710 INFO [RM-0] webapp.WebApps (WebApps.java:build(288)) - Registered webapp guice modules 2015-11-22 19:27:58,711 INFO [RM-0] http.HttpServer2 (HttpServer2.java:openListeners(917)) - Jetty bound to port 40307 2015-11-22 19:27:58,711 INFO [RM-0] mortbay.log (Slf4jLog.java:info(67)) - jetty-6.1.26 2015-11-22 19:27:58,728 INFO [RM-0] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:27:58,728 INFO [Thread[Thread-3605,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2015-11-22 19:27:58,728 INFO [Thread[Thread-3605,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:27:58,729 INFO [RM-0] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:27:58,729 INFO [Thread[Thread-3607,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) Nov 22, 2015 7:27:58 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.resourcemanager.webapp.JAXBContextResolver as a provider class Nov 22, 2015 7:27:58 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices as a root resource class Nov 22, 2015 7:27:58 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a provider class Nov 22, 2015 7:27:58 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate INFO: Initiating Jersey application, version 'Jersey: 1.9 09/02/2011 11:17 AM' Nov 22, 2015 7:27:58 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.resourcemanager.webapp.JAXBContextResolver to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:27:58 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.webapp.GenericExceptionHandler to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:27:58 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices to GuiceManagedComponentProvider with the scope "Singleton" 2015-11-22 19:27:58,937 INFO [RM-0] mortbay.log (Slf4jLog.java:info(67)) - Started HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:40307 2015-11-22 19:27:58,938 INFO [RM-0] webapp.WebApps (WebApps.java:start(307)) - Web app cluster started at 40307 2015-11-22 19:27:58,939 INFO [RM-0] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:27:58,939 INFO [Socket Reader #1 for port 53669] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 53669 2015-11-22 19:27:58,941 INFO [RM-0] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.server.api.ResourceManagerAdministrationProtocolPB to the server 2015-11-22 19:27:58,946 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:27:58,946 INFO [IPC Server listener on 53669] ipc.Server (Server.java:run(794)) - IPC Server listener on 53669: starting 2015-11-22 19:28:00,171 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:startResourceManager(339)) - MiniYARN ResourceManager address: ip-172-31-20-42.ap-northeast-1.compute.internal:52028 2015-11-22 19:28:00,171 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:startResourceManager(341)) - MiniYARN ResourceManager web address: ip-172-31-20-42.ap-northeast-1.compute.internal:40307 2015-11-22 19:28:00,171 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(459)) - MiniYARN ResourceManager address: ip-172-31-20-42.ap-northeast-1.compute.internal:52028 2015-11-22 19:28:00,171 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(461)) - MiniYARN ResourceManager web address: ip-172-31-20-42.ap-northeast-1.compute.internal:40307 2015-11-22 19:28:00,172 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(575)) - Waiting for NM 0 to start... 2015-11-22 19:28:00,173 INFO [Thread-3614] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:28:00,174 INFO [Socket Reader #1 for port 50663] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 50663 2015-11-22 19:28:00,175 INFO [Thread-3614] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.api.ContainerManagementProtocolPB to the server 2015-11-22 19:28:00,175 INFO [Thread-3614] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:serviceStart(464)) - Blocking new container-requests as container manager rpc server is still starting. 2015-11-22 19:28:00,176 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:28:00,176 INFO [IPC Server listener on 50663] ipc.Server (Server.java:run(794)) - IPC Server listener on 50663: starting 2015-11-22 19:28:00,179 INFO [Thread-3614] security.NMContainerTokenSecretManager (NMContainerTokenSecretManager.java:setNodeId(260)) - Updating node address : ip-172-31-20-42.ap-northeast-1.compute.internal:50663 2015-11-22 19:28:00,180 INFO [Thread-3614] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:28:00,180 INFO [Socket Reader #1 for port 47651] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 47651 2015-11-22 19:28:00,182 INFO [Thread-3614] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB to the server 2015-11-22 19:28:00,182 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:28:00,183 INFO [IPC Server listener on 47651] ipc.Server (Server.java:run(794)) - IPC Server listener on 47651: starting 2015-11-22 19:28:00,183 INFO [Thread-3614] localizer.ResourceLocalizationService (ResourceLocalizationService.java:serviceStart(363)) - Localizer started on port 47651 2015-11-22 19:28:00,191 INFO [Thread-3614] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:serviceStart(509)) - ContainerManager started at /172.31.20.42:50663 2015-11-22 19:28:00,191 INFO [Thread-3614] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:serviceStart(510)) - ContainerManager bound to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:0 2015-11-22 19:28:00,191 INFO [Thread-3614] webapp.WebServer (WebServer.java:serviceStart(71)) - Instantiating NMWebApp at ip-172-31-20-42.ap-northeast-1.compute.internal:0 2015-11-22 19:28:00,192 INFO [Thread-3614] server.AuthenticationFilter (AuthenticationFilter.java:constructSecretProvider(304)) - Unable to initialize FileSignerSecretProvider, falling back to use random secrets. 2015-11-22 19:28:00,192 INFO [Thread-3614] http.HttpRequestLog (HttpRequestLog.java:getRequestLog(80)) - Http request log for http.requests.nodemanager is not defined 2015-11-22 19:28:00,193 INFO [Thread-3614] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter) 2015-11-22 19:28:00,193 INFO [Thread-3614] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context node 2015-11-22 19:28:00,193 INFO [Thread-3614] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context static 2015-11-22 19:28:00,193 INFO [Thread-3614] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context logs 2015-11-22 19:28:00,193 INFO [Thread-3614] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context node 2015-11-22 19:28:00,194 INFO [Thread-3614] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2015-11-22 19:28:00,194 INFO [Thread-3614] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2015-11-22 19:28:00,194 INFO [Thread-3614] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'Timeline Authentication Filter' (class=org.apache.hadoop.yarn.server.timeline.security.TimelineAuthenticationFilter) 2015-11-22 19:28:00,194 INFO [Thread-3614] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /node/* 2015-11-22 19:28:00,194 INFO [Thread-3614] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /ws/* Nov 22, 2015 7:28:00 PM com.google.inject.servlet.GuiceFilter setPipeline WARNING: Multiple Servlet injectors detected. This is a warning indicating that you have more than one GuiceFilter running in your web application. If this is deliberate, you may safely ignore this message. If this is NOT deliberate however, your application may not work as expected. 2015-11-22 19:28:00,198 INFO [Thread-3614] webapp.WebApps (WebApps.java:build(288)) - Registered webapp guice modules 2015-11-22 19:28:00,199 INFO [Thread-3614] http.HttpServer2 (HttpServer2.java:openListeners(917)) - Jetty bound to port 49669 2015-11-22 19:28:00,199 INFO [Thread-3614] mortbay.log (Slf4jLog.java:info(67)) - jetty-6.1.26 2015-11-22 19:28:00,209 INFO [Thread-3614] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:28:00,209 INFO [Thread[Thread-3655,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2015-11-22 19:28:00,210 INFO [Thread[Thread-3655,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:28:00,210 INFO [Thread-3614] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:28:00,211 INFO [Thread[Thread-3657,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) Nov 22, 2015 7:28:00 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.nodemanager.webapp.NMWebServices as a root resource class Nov 22, 2015 7:28:00 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a provider class Nov 22, 2015 7:28:00 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.nodemanager.webapp.JAXBContextResolver as a provider class Nov 22, 2015 7:28:00 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate INFO: Initiating Jersey application, version 'Jersey: 1.9 09/02/2011 11:17 AM' Nov 22, 2015 7:28:00 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.nodemanager.webapp.JAXBContextResolver to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:28:00 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.webapp.GenericExceptionHandler to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:28:00 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.nodemanager.webapp.NMWebServices to GuiceManagedComponentProvider with the scope "Singleton" 2015-11-22 19:28:00,294 INFO [Thread-3614] mortbay.log (Slf4jLog.java:info(67)) - Started HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:49669 2015-11-22 19:28:00,295 INFO [Thread-3614] webapp.WebApps (WebApps.java:start(307)) - Web app node started at 49669 2015-11-22 19:28:00,296 INFO [Thread-3614] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:serviceStart(222)) - Node ID assigned is : ip-172-31-20-42.ap-northeast-1.compute.internal:50663 2015-11-22 19:28:00,296 INFO [Thread-3614] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNMContainerStatuses(550)) - Sending out 0 NM container statuses: [] 2015-11-22 19:28:00,296 INFO [Thread-3614] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:registerWithRM(330)) - Registering with RM using containers :[] 2015-11-22 19:28:00,296 INFO [Thread-3614] resourcemanager.ResourceTrackerService (ResourceTrackerService.java:registerNodeManager(375)) - NodeManager from node ip-172-31-20-42.ap-northeast-1.compute.internal(cmPort: 50663 httpPort: 49669) registered with capability: , assigned nodeId ip-172-31-20-42.ap-northeast-1.compute.internal:50663 2015-11-22 19:28:00,296 INFO [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(566)) - ip-172-31-20-42.ap-northeast-1.compute.internal:50663 Node Transitioned from NEW to RUNNING 2015-11-22 19:28:00,296 INFO [Thread-3614] security.NMContainerTokenSecretManager (NMContainerTokenSecretManager.java:setMasterKey(138)) - Rolling master-key for container-tokens, got key with id -550760804 2015-11-22 19:28:00,297 INFO [Thread-3614] security.NMTokenSecretManagerInNM (NMTokenSecretManagerInNM.java:setMasterKey(135)) - Rolling master-key for container-tokens, got key with id 744666633 2015-11-22 19:28:00,297 INFO [Thread-3614] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:registerWithRM(391)) - Registered with ResourceManager as ip-172-31-20-42.ap-northeast-1.compute.internal:50663 with total resource of 2015-11-22 19:28:00,298 INFO [Thread-3614] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:registerWithRM(392)) - Notifying ContainerManager to unblock new container-requests 2015-11-22 19:28:00,298 INFO [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:addNode(1427)) - Added node ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResource: 2015-11-22 19:28:01,172 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(728)) - Waiting for Timeline Server to start... 2015-11-22 19:28:01,173 INFO [Thread-3660] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:28:01,173 INFO [Thread[Thread-3662,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2015-11-22 19:28:01,174 INFO [Thread[Thread-3662,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:28:01,174 INFO [Thread-3660] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:28:01,175 INFO [Socket Reader #1 for port 32936] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 32936 2015-11-22 19:28:01,176 INFO [Thread-3660] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.api.ApplicationHistoryProtocolPB to the server 2015-11-22 19:28:01,176 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:28:01,177 INFO [IPC Server listener on 32936] ipc.Server (Server.java:run(794)) - IPC Server listener on 32936: starting 2015-11-22 19:28:01,178 INFO [Thread-3660] applicationhistoryservice.ApplicationHistoryClientService (ApplicationHistoryClientService.java:serviceStart(111)) - Instantiated ApplicationHistoryClientService at ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:32936 2015-11-22 19:28:01,178 INFO [Thread-3660] applicationhistoryservice.ApplicationHistoryManagerImpl (ApplicationHistoryManagerImpl.java:serviceStart(71)) - Starting ApplicationHistory 2015-11-22 19:28:01,179 INFO [Thread-3660] server.AuthenticationFilter (AuthenticationFilter.java:constructSecretProvider(304)) - Unable to initialize FileSignerSecretProvider, falling back to use random secrets. 2015-11-22 19:28:01,180 INFO [Thread-3660] http.HttpRequestLog (HttpRequestLog.java:getRequestLog(80)) - Http request log for http.requests.applicationhistory is not defined 2015-11-22 19:28:01,180 INFO [Thread-3660] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter) 2015-11-22 19:28:01,180 INFO [Thread-3660] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context applicationhistory 2015-11-22 19:28:01,181 INFO [Thread-3660] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context logs 2015-11-22 19:28:01,181 INFO [Thread-3660] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context static 2015-11-22 19:28:01,181 INFO [Thread-3660] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context applicationhistory 2015-11-22 19:28:01,181 INFO [Thread-3660] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2015-11-22 19:28:01,181 INFO [Thread-3660] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2015-11-22 19:28:01,181 INFO [Thread-3660] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'Timeline Authentication Filter' (class=org.apache.hadoop.yarn.server.timeline.security.TimelineAuthenticationFilter) 2015-11-22 19:28:01,182 INFO [Thread-3660] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /applicationhistory/* 2015-11-22 19:28:01,182 INFO [Thread-3660] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /ws/* Nov 22, 2015 7:28:01 PM com.google.inject.servlet.GuiceFilter setPipeline WARNING: Multiple Servlet injectors detected. This is a warning indicating that you have more than one GuiceFilter running in your web application. If this is deliberate, you may safely ignore this message. If this is NOT deliberate however, your application may not work as expected. 2015-11-22 19:28:01,185 INFO [Thread-3660] webapp.WebApps (WebApps.java:build(288)) - Registered webapp guice modules 2015-11-22 19:28:01,186 INFO [Thread-3660] http.HttpServer2 (HttpServer2.java:openListeners(917)) - Jetty bound to port 41381 2015-11-22 19:28:01,186 INFO [Thread-3660] mortbay.log (Slf4jLog.java:info(67)) - jetty-6.1.26 2015-11-22 19:28:01,196 INFO [Thread-3660] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:28:01,196 INFO [Thread[Thread-3680,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2015-11-22 19:28:01,196 INFO [Thread[Thread-3680,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:28:01,197 INFO [Thread-3660] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:28:01,197 INFO [Thread[Thread-3682,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) Nov 22, 2015 7:28:01 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider as a provider class Nov 22, 2015 7:28:01 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.AHSWebServices as a root resource class Nov 22, 2015 7:28:01 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.timeline.webapp.TimelineWebServices as a root resource class Nov 22, 2015 7:28:01 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a provider class Nov 22, 2015 7:28:01 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate INFO: Initiating Jersey application, version 'Jersey: 1.9 09/02/2011 11:17 AM' Nov 22, 2015 7:28:01 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.webapp.GenericExceptionHandler to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:28:01 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:28:01 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.AHSWebServices to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:28:01 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.timeline.webapp.TimelineWebServices to GuiceManagedComponentProvider with the scope "Singleton" 2015-11-22 19:28:01,282 INFO [Thread-3660] mortbay.log (Slf4jLog.java:info(67)) - Started HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:41381 2015-11-22 19:28:01,283 INFO [Thread-3660] applicationhistoryservice.ApplicationHistoryServer (ApplicationHistoryServer.java:startWebApp(342)) - Instantiating AHSWebApp at 41381 2015-11-22 19:28:02,672 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(739)) - MiniYARN ApplicationHistoryServer address: ip-172-31-20-42.ap-northeast-1.compute.internal:32936 2015-11-22 19:28:02,673 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(741)) - MiniYARN ApplicationHistoryServer web address: ip-172-31-20-42.ap-northeast-1.compute.internal:41381 2015-11-22 19:28:04,696 INFO [Thread-3684] distributedshell.TestDistributedShell (TestDistributedShell.java:testDSShellWithCustomLogPropertyFile(496)) - Initializing DS Client 2015-11-22 19:28:04,739 INFO [Thread-3684] impl.TimelineClientImpl (TimelineClientImpl.java:serviceInit(296)) - Timeline service address: http://ip-172-31-20-42.ap-northeast-1.compute.internal:41381/ws/v1/timeline/ 2015-11-22 19:28:04,743 INFO [Thread-3684] distributedshell.TestDistributedShell (TestDistributedShell.java:testDSShellWithCustomLogPropertyFile(501)) - Running DS Client 2015-11-22 19:28:04,743 INFO [Thread-3684] distributedshell.Client (Client.java:run(443)) - Running Client 2015-11-22 19:28:04,755 INFO [Thread-3684] client.RMProxy (RMProxy.java:createRMProxy(98)) - Connecting to ResourceManager at ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 2015-11-22 19:28:04,755 DEBUG [Thread-3684] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.yarn.client.RMProxy.getProxy(RMProxy.java:136) 2015-11-22 19:28:04,755 DEBUG [Thread-3684] ipc.YarnRPC (YarnRPC.java:create(59)) - Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC 2015-11-22 19:28:04,755 DEBUG [Thread-3684] ipc.HadoopYarnProtoRPC (HadoopYarnProtoRPC.java:getProxy(47)) - Creating a HadoopYarnProtoRpc proxy for protocol interface org.apache.hadoop.yarn.api.ApplicationClientProtocol 2015-11-22 19:28:04,756 DEBUG [Thread-3684] ipc.Client (ClientCache.java:getClient(63)) - getting client out of cache: org.apache.hadoop.ipc.Client@67cf996b 2015-11-22 19:28:04,756 DEBUG [Thread-3684] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl is started 2015-11-22 19:28:04,756 DEBUG [Thread-3684] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.client.api.impl.YarnClientImpl is started 2015-11-22 19:28:04,756 DEBUG [Thread-3684] ipc.Client (Client.java:(439)) - The ping interval is 60000 ms. 2015-11-22 19:28:04,756 DEBUG [Thread-3684] ipc.Client (Client.java:setupIOstreams(723)) - Connecting to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 2015-11-22 19:28:04,757 DEBUG [IPC Server listener on 52028] ipc.Server (Server.java:register(2883)) - Server connection from 172.31.20.42:53762; # active connections: 1; # queued calls: 0 2015-11-22 19:28:04,758 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:run(1000)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu: starting, having connections 1 2015-11-22 19:28:04,759 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #811 2015-11-22 19:28:04,760 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #-3 2015-11-22 19:28:04,762 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:authorizeConnection(2097)) - Successfully authorized userInfo { effectiveUser: "ubuntu" } protocol: "org.apache.hadoop.yarn.api.ApplicationClientProtocolPB" 2015-11-22 19:28:04,763 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #811 2015-11-22 19:28:04,763 DEBUG [IPC Server handler 0 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 0 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getClusterMetrics from 172.31.20.42:53762 Call#811 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:04,763 DEBUG [IPC Server handler 0 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:04,763 DEBUG [IPC Server handler 0 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getClusterMetrics queueTime= 0 procesingTime= 0 2015-11-22 19:28:04,764 DEBUG [IPC Server handler 0 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 0 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getClusterMetrics from 172.31.20.42:53762 Call#811 Retry#0 2015-11-22 19:28:04,765 DEBUG [IPC Server handler 0 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 0 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getClusterMetrics from 172.31.20.42:53762 Call#811 Retry#0 Wrote 47 bytes. 2015-11-22 19:28:04,765 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #811 2015-11-22 19:28:04,765 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getClusterMetrics took 9ms 2015-11-22 19:28:04,765 INFO [Thread-3684] distributedshell.Client (Client.java:run(447)) - Got Cluster metric info from ASM, numNodeManagers=1 2015-11-22 19:28:04,765 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #812 2015-11-22 19:28:04,765 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #812 2015-11-22 19:28:04,765 DEBUG [IPC Server handler 1 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 1 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getClusterNodes from 172.31.20.42:53762 Call#812 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:04,766 DEBUG [IPC Server handler 1 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:04,766 DEBUG [IPC Server handler 1 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getClusterNodes queueTime= 1 procesingTime= 0 2015-11-22 19:28:04,767 DEBUG [IPC Server handler 1 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 1 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getClusterNodes from 172.31.20.42:53762 Call#812 Retry#0 2015-11-22 19:28:04,767 DEBUG [IPC Server handler 1 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 1 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getClusterNodes from 172.31.20.42:53762 Call#812 Retry#0 Wrote 186 bytes. 2015-11-22 19:28:04,767 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #812 2015-11-22 19:28:04,767 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getClusterNodes took 2ms 2015-11-22 19:28:04,767 INFO [Thread-3684] distributedshell.Client (Client.java:run(452)) - Got Cluster node info from ASM 2015-11-22 19:28:04,767 INFO [Thread-3684] distributedshell.Client (Client.java:run(454)) - Got node report from ASM for, nodeId=ip-172-31-20-42.ap-northeast-1.compute.internal:50663, nodeAddress=ip-172-31-20-42.ap-northeast-1.compute.internal:49669, nodeRackName=/default-rack, nodeNumContainers=0 2015-11-22 19:28:04,768 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #813 2015-11-22 19:28:04,768 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #813 2015-11-22 19:28:04,768 DEBUG [IPC Server handler 2 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 2 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getQueueInfo from 172.31.20.42:53762 Call#813 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:04,768 DEBUG [IPC Server handler 2 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:04,768 DEBUG [IPC Server handler 2 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getQueueInfo queueTime= 0 procesingTime= 0 2015-11-22 19:28:04,768 DEBUG [IPC Server handler 2 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 2 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getQueueInfo from 172.31.20.42:53762 Call#813 Retry#0 2015-11-22 19:28:04,768 DEBUG [IPC Server handler 2 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 2 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getQueueInfo from 172.31.20.42:53762 Call#813 Retry#0 Wrote 106 bytes. 2015-11-22 19:28:04,768 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #813 2015-11-22 19:28:04,768 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getQueueInfo took 1ms 2015-11-22 19:28:04,769 INFO [Thread-3684] distributedshell.Client (Client.java:run(462)) - Queue info, queueName=default, queueCurrentCapacity=0.0, queueMaxCapacity=1.0, queueApplicationCount=0, queueChildQueueCount=0 2015-11-22 19:28:04,769 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #814 2015-11-22 19:28:04,769 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #814 2015-11-22 19:28:04,769 DEBUG [IPC Server handler 4 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 4 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getQueueUserAcls from 172.31.20.42:53762 Call#814 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:04,769 DEBUG [IPC Server handler 4 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:04,769 DEBUG [IPC Server handler 4 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getQueueUserAcls queueTime= 0 procesingTime= 0 2015-11-22 19:28:04,769 DEBUG [IPC Server handler 4 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 4 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getQueueUserAcls from 172.31.20.42:53762 Call#814 Retry#0 2015-11-22 19:28:04,770 DEBUG [IPC Server handler 4 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 4 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getQueueUserAcls from 172.31.20.42:53762 Call#814 Retry#0 Wrote 60 bytes. 2015-11-22 19:28:04,770 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #814 2015-11-22 19:28:04,770 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getQueueUserAcls took 1ms 2015-11-22 19:28:04,770 INFO [Thread-3684] distributedshell.Client (Client.java:run(472)) - User ACL Info for Queue, queueName=root, userAcl=SUBMIT_APPLICATIONS 2015-11-22 19:28:04,770 INFO [Thread-3684] distributedshell.Client (Client.java:run(472)) - User ACL Info for Queue, queueName=root, userAcl=ADMINISTER_QUEUE 2015-11-22 19:28:04,770 INFO [Thread-3684] distributedshell.Client (Client.java:run(472)) - User ACL Info for Queue, queueName=default, userAcl=SUBMIT_APPLICATIONS 2015-11-22 19:28:04,770 INFO [Thread-3684] distributedshell.Client (Client.java:run(472)) - User ACL Info for Queue, queueName=default, userAcl=ADMINISTER_QUEUE 2015-11-22 19:28:04,770 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #815 2015-11-22 19:28:04,770 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #815 2015-11-22 19:28:04,771 DEBUG [IPC Server handler 3 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 3 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getNewApplication from 172.31.20.42:53762 Call#815 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:04,771 DEBUG [IPC Server handler 3 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:04,771 INFO [IPC Server handler 3 on 52028] resourcemanager.ClientRMService (ClientRMService.java:getNewApplicationId(305)) - Allocated new applicationId: 1 2015-11-22 19:28:04,771 DEBUG [IPC Server handler 3 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getNewApplication queueTime= 0 procesingTime= 0 2015-11-22 19:28:04,771 DEBUG [IPC Server handler 3 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 3 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getNewApplication from 172.31.20.42:53762 Call#815 Retry#0 2015-11-22 19:28:04,771 DEBUG [IPC Server handler 3 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 3 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getNewApplication from 172.31.20.42:53762 Call#815 Retry#0 Wrote 51 bytes. 2015-11-22 19:28:04,771 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #815 2015-11-22 19:28:04,771 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getNewApplication took 1ms 2015-11-22 19:28:04,771 INFO [Thread-3684] distributedshell.Client (Client.java:run(491)) - Max mem capability of resources in this cluster 4096 2015-11-22 19:28:04,772 INFO [Thread-3684] distributedshell.Client (Client.java:run(502)) - Max virtual cores capability of resources in this cluster 4 2015-11-22 19:28:04,772 INFO [Thread-3684] distributedshell.Client (Client.java:run(528)) - Copy App Master jar from local filesystem and add to local environment 2015-11-22 19:28:04,784 INFO [Thread-3684] distributedshell.Client (Client.java:run(577)) - Set the environment for the application master 2015-11-22 19:28:04,784 INFO [Thread-3684] distributedshell.Client (Client.java:run(619)) - Setting up app master command 2015-11-22 19:28:04,784 INFO [Thread-3684] distributedshell.Client (Client.java:run(650)) - Completed setting up app master command {{JAVA_HOME}}/bin/java -Xmx512m org.apache.hadoop.yarn.applications.distributedshell.ApplicationMaster --container_memory 128 --container_vcores 1 --num_containers 3 --priority 0 1>/AppMaster.stdout 2>/AppMaster.stderr 2015-11-22 19:28:04,784 INFO [Thread-3684] distributedshell.Client (Client.java:run(706)) - Submitting application to ASM 2015-11-22 19:28:04,785 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #816 2015-11-22 19:28:04,785 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #816 2015-11-22 19:28:04,785 DEBUG [IPC Server handler 5 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 5 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.submitApplication from 172.31.20.42:53762 Call#816 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:04,785 DEBUG [IPC Server handler 5 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:04,785 INFO [IPC Server handler 5 on 52028] capacity.CapacityScheduler (CapacityScheduler.java:checkAndGetApplicationPriority(1941)) - Priority '0' is acceptable in queue : default for application: application_1448220478671_0001 for the user: ubuntu 2015-11-22 19:28:04,786 WARN [IPC Server handler 5 on 52028] rmapp.RMAppImpl (RMAppImpl.java:(420)) - The specific max attempts: 0 for application: 1 is invalid, because it is out of the range [1, 2]. Use the global max attempts instead. 2015-11-22 19:28:04,786 INFO [IPC Server handler 5 on 52028] resourcemanager.ClientRMService (ClientRMService.java:submitApplication(602)) - Application with id 1 submitted by user ubuntu 2015-11-22 19:28:04,786 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEvent.EventType: START 2015-11-22 19:28:04,786 INFO [IPC Server handler 5 on 52028] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(154)) - USER=ubuntu IP=172.31.20.42 OPERATION=Submit Application Request TARGET=ClientRMService RESULT=SUCCESS APPID=application_1448220478671_0001 2015-11-22 19:28:04,786 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(777)) - Processing event for application_1448220478671_0001 of type START 2015-11-22 19:28:04,786 DEBUG [IPC Server handler 5 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: submitApplication queueTime= 0 procesingTime= 1 2015-11-22 19:28:04,786 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:transition(1079)) - Storing application with id application_1448220478671_0001 2015-11-22 19:28:04,786 DEBUG [IPC Server handler 5 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 5 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.submitApplication from 172.31.20.42:53762 Call#816 Retry#0 2015-11-22 19:28:04,786 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220478671_0001 State change from NEW to NEW_SAVING 2015-11-22 19:28:04,786 DEBUG [IPC Server handler 5 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 5 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.submitApplication from 172.31.20.42:53762 Call#816 Retry#0 Wrote 33 bytes. 2015-11-22 19:28:04,786 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStoreAppEvent.EventType: STORE_APP 2015-11-22 19:28:04,787 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #816 2015-11-22 19:28:04,787 DEBUG [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:handleStoreEvent(1016)) - Processing event of type STORE_APP 2015-11-22 19:28:04,787 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: submitApplication took 2ms 2015-11-22 19:28:04,787 INFO [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:transition(208)) - Storing info for app: application_1448220478671_0001 2015-11-22 19:28:04,787 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEvent.EventType: APP_NEW_SAVED 2015-11-22 19:28:04,787 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(777)) - Processing event for application_1448220478671_0001 of type APP_NEW_SAVED 2015-11-22 19:28:04,787 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #817 2015-11-22 19:28:04,787 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220478671_0001 State change from NEW_SAVING to SUBMITTED 2015-11-22 19:28:04,787 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.AppAddedSchedulerEvent.EventType: APP_ADDED 2015-11-22 19:28:04,788 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #817 2015-11-22 19:28:04,788 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:addApplication(349)) - Application added - appId: application_1448220478671_0001 user: ubuntu leaf-queue of parent: root #applications: 1 2015-11-22 19:28:04,788 INFO [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:addApplication(765)) - Accepted application application_1448220478671_0001 from user: ubuntu, in queue: default 2015-11-22 19:28:04,788 DEBUG [IPC Server handler 6 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 6 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#817 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:04,788 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEvent.EventType: APP_ACCEPTED 2015-11-22 19:28:04,788 DEBUG [IPC Server handler 6 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:04,788 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(777)) - Processing event for application_1448220478671_0001 of type APP_ACCEPTED 2015-11-22 19:28:04,788 DEBUG [IPC Server handler 6 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:28:04,788 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220478671_0001 State change from SUBMITTED to ACCEPTED 2015-11-22 19:28:04,789 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppStartAttemptEvent.EventType: START 2015-11-22 19:28:04,789 DEBUG [IPC Server handler 6 on 52028] scheduler.AbstractYarnScheduler (AbstractYarnScheduler.java:getAppResourceUsageReport(322)) - Request for appInfo of unknown attempt appattempt_1448220478671_0001_000001 2015-11-22 19:28:04,789 DEBUG [IPC Server handler 6 on 52028] scheduler.AbstractYarnScheduler (AbstractYarnScheduler.java:getAppResourceUsageReport(322)) - Request for appInfo of unknown attempt appattempt_1448220478671_0001_000001 2015-11-22 19:28:04,789 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type START 2015-11-22 19:28:04,789 INFO [AsyncDispatcher event handler] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:registerAppAttempt(661)) - Registering app attempt : appattempt_1448220478671_0001_000001 2015-11-22 19:28:04,789 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220478671_0001_000001 State change from NEW to SUBMITTED 2015-11-22 19:28:04,789 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.AppAttemptAddedSchedulerEvent.EventType: APP_ATTEMPT_ADDED 2015-11-22 19:28:04,789 DEBUG [IPC Server handler 6 on 52028] scheduler.AbstractYarnScheduler (AbstractYarnScheduler.java:getAppResourceUsageReport(322)) - Request for appInfo of unknown attempt appattempt_1448220478671_0001_000001 2015-11-22 19:28:04,790 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:activateApplications(638)) - application AMResource maxAMResourcePerQueuePercent 0.1 amLimit lastClusterResource amIfStarted AM node-partition name 2015-11-22 19:28:04,790 INFO [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:activateApplications(705)) - Application application_1448220478671_0001 from user: ubuntu activated in queue: default 2015-11-22 19:28:04,790 INFO [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:addApplicationAttempt(720)) - Application added - appId: application_1448220478671_0001 user: ubuntu, leaf-queue: default #user-pending-applications: 0 #user-active-applications: 1 #queue-pending-applications: 0 #queue-active-applications: 1 2015-11-22 19:28:04,790 DEBUG [IPC Server handler 6 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 2 2015-11-22 19:28:04,790 INFO [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:addApplicationAttempt(805)) - Added Application Attempt appattempt_1448220478671_0001_000001 to scheduler from user ubuntu in queue default 2015-11-22 19:28:04,790 DEBUG [IPC Server handler 6 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 6 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#817 Retry#0 2015-11-22 19:28:04,790 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: ATTEMPT_ADDED 2015-11-22 19:28:04,790 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #817 2015-11-22 19:28:04,790 DEBUG [IPC Server handler 6 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 6 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#817 Retry#0 Wrote 393 bytes. 2015-11-22 19:28:04,790 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 3ms 2015-11-22 19:28:04,790 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type ATTEMPT_ADDED 2015-11-22 19:28:04,792 DEBUG [AsyncDispatcher event handler] blacklist.SimpleBlacklistManager (SimpleBlacklistManager.java:getBlacklistUpdates(69)) - blacklist size 0 is less than failure threshold ratio 0.800000011920929 out of total usable nodes 1 2015-11-22 19:28:04,792 INFO [Thread-3684] impl.YarnClientImpl (YarnClientImpl.java:submitApplication(279)) - Submitted application application_1448220478671_0001 2015-11-22 19:28:04,793 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:transition(1003)) - Using blacklist for AM: additions([]) and removals([]) 2015-11-22 19:28:04,793 DEBUG [AsyncDispatcher event handler] capacity.CapacityScheduler (CapacityScheduler.java:allocate(941)) - allocate: pre-update appattempt_1448220478671_0001_000001 ask size =1 2015-11-22 19:28:04,793 DEBUG [AsyncDispatcher event handler] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:updateResourceRequests(320)) - update: application=application_1448220478671_0001 request={Priority: 0, Capability: , # Containers: 1, Location: *, Relax Locality: true, Node Label Expression: } 2015-11-22 19:28:04,794 DEBUG [AsyncDispatcher event handler] scheduler.ActiveUsersManager (ActiveUsersManager.java:activateApplication(68)) - User ubuntu added to activeUsers, currently: 1 2015-11-22 19:28:04,795 DEBUG [AsyncDispatcher event handler] capacity.CapacityScheduler (CapacityScheduler.java:allocate(952)) - allocate: post-update 2015-11-22 19:28:04,795 DEBUG [AsyncDispatcher event handler] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(470)) - showRequests: application=application_1448220478671_0001 headRoom= currentConsumption=0 2015-11-22 19:28:04,795 DEBUG [AsyncDispatcher event handler] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(474)) - showRequests: application=application_1448220478671_0001 request={Priority: 0, Capability: , # Containers: 1, Location: *, Relax Locality: true, Node Label Expression: } 2015-11-22 19:28:04,795 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220478671_0001_000001 State change from SUBMITTED to SCHEDULED 2015-11-22 19:28:05,301 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:28:05,301 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 0 container statuses: [] 2015-11-22 19:28:05,301 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:05,301 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:28:05,301 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:28:05,301 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:28:05,301 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:28:05,301 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:28:05,302 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(408)) - Trying to assign containers to child-queue of root 2015-11-22 19:28:05,302 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:printChildQueues(617)) - printChildQueues - queue: root child-queues: root.defaultusedCapacity=(0.0), label=(*) 2015-11-22 19:28:05,302 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(564)) - Trying to assign to queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=1, numContainers=0 2015-11-22 19:28:05,302 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:assignContainers(817)) - assignContainers: node=ip-172-31-20-42.ap-northeast-1.compute.internal #applications=1 2015-11-22 19:28:05,302 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2015-11-22 19:28:05,303 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimitAndSetHeadroom(984)) - Headroom calculation for user ubuntu: userLimit= queueMaxAvailRes= consumed= headroom= 2015-11-22 19:28:05,303 DEBUG [ResourceManager Event Processor] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:assignContainers(477)) - pre-assignContainers for application application_1448220478671_0001 2015-11-22 19:28:05,303 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2015-11-22 19:28:05,303 DEBUG [ResourceManager Event Processor] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(470)) - showRequests: application=application_1448220478671_0001 headRoom= currentConsumption=0 2015-11-22 19:28:05,304 DEBUG [ResourceManager Event Processor] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(474)) - showRequests: application=application_1448220478671_0001 request={Priority: 0, Capability: , # Containers: 1, Location: *, Relax Locality: true, Node Label Expression: } 2015-11-22 19:28:05,304 DEBUG [ResourceManager Event Processor] allocator.IncreaseContainerAllocator (IncreaseContainerAllocator.java:assignContainers(184)) - Skip allocating increase request since we don't have any increase request on this node=ip-172-31-20-42.ap-northeast-1.compute.internal:50663 2015-11-22 19:28:05,305 DEBUG [ResourceManager Event Processor] allocator.RegularContainerAllocator (RegularContainerAllocator.java:assignContainer(407)) - assignContainers: node=ip-172-31-20-42.ap-northeast-1.compute.internal application=application_1448220478671_0001 priority=0 request={Priority: 0, Capability: , # Containers: 1, Location: *, Relax Locality: true, Node Label Expression: } type=OFF_SWITCH 2015-11-22 19:28:05,305 DEBUG [ResourceManager Event Processor] scheduler.ActiveUsersManager (ActiveUsersManager.java:deactivateApplication(94)) - User ubuntu removed from activeUsers, currently: 0 2015-11-22 19:28:05,306 DEBUG [ResourceManager Event Processor] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:allocate(594)) - allocate: applicationId=application_1448220478671_0001 container=container_1448220478671_0001_01_000001 host=ip-172-31-20-42.ap-northeast-1.compute.internal:50663 user=ubuntu resource= type=OFF_SWITCH 2015-11-22 19:28:05,306 DEBUG [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(424)) - Processing container_1448220478671_0001_01_000001 of type START 2015-11-22 19:28:05,306 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220478671_0001_01_000001 Container Transitioned from NEW to ALLOCATED 2015-11-22 19:28:05,306 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: CONTAINER_ALLOCATED 2015-11-22 19:28:05,306 DEBUG [ResourceManager Event Processor] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:allocate(208)) - allocate: applicationAttemptId=appattempt_1448220478671_0001_000001 container=container_1448220478671_0001_01_000001 host=ip-172-31-20-42.ap-northeast-1.compute.internal type=OFF_SWITCH 2015-11-22 19:28:05,306 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type CONTAINER_ALLOCATED 2015-11-22 19:28:05,306 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220478671_0001 CONTAINERID=container_1448220478671_0001_01_000001 2015-11-22 19:28:05,307 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(154)) - Assigned container container_1448220478671_0001_01_000001 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:50663, which has 1 containers, used and available after allocation 2015-11-22 19:28:05,307 INFO [ResourceManager Event Processor] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(89)) - assignedContainer application attempt=appattempt_1448220478671_0001_000001 container=container_1448220478671_0001_01_000001 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@33e07bad clusterResource= type=OFF_SWITCH 2015-11-22 19:28:05,307 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:assignContainers(881)) - post-assignContainers for application application_1448220478671_0001 2015-11-22 19:28:05,307 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:05,307 DEBUG [ResourceManager Event Processor] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(470)) - showRequests: application=application_1448220478671_0001 headRoom= currentConsumption=512 2015-11-22 19:28:05,307 DEBUG [ResourceManager Event Processor] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(474)) - showRequests: application=application_1448220478671_0001 request={Priority: 0, Capability: , # Containers: 0, Location: *, Relax Locality: true, Node Label Expression: } 2015-11-22 19:28:05,307 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:05,307 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:05,308 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:05,309 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:allocateResource(1277)) - default user=ubuntu used= numContainers=1 headroom = user-resources= 2015-11-22 19:28:05,310 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(575)) - Assigned to queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.125, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 --> , OFF_SWITCH 2015-11-22 19:28:05,310 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(589)) - Re-sorting assigned queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.125, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 2015-11-22 19:28:05,310 DEBUG [AsyncDispatcher event handler] security.BaseContainerTokenSecretManager (BaseContainerTokenSecretManager.java:createPassword(90)) - Creating password for container_1448220478671_0001_01_000001 for user container_1448220478671_0001_01_000001 (auth:SIMPLE) to be run on NM ip-172-31-20-42.ap-northeast-1.compute.internal:50663 2015-11-22 19:28:05,310 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:printChildQueues(617)) - printChildQueues - queue: root child-queues: root.defaultusedCapacity=(0.125), label=(*) 2015-11-22 19:28:05,310 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(461)) - assignedContainer queue=root usedCapacity=0.125 absoluteUsedCapacity=0.125 used= cluster= 2015-11-22 19:28:05,310 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(473)) - ParentQ=root assignedSoFarInThisIteration= usedCapacity=0.125 absoluteUsedCapacity=0.125 2015-11-22 19:28:05,310 DEBUG [AsyncDispatcher event handler] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(197)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 1 cluster_timestamp: 1448220478671 } attemptId: 1 } id: 1 } nmHostAddr: "ip-172-31-20-42.ap-northeast-1.compute.internal:50663" appSubmitter: "ubuntu" resource { memory: 512 virtual_cores: 1 } expiryTimeStamp: 1448221085309 masterKeyId: -550760804 rmIdentifier: 1448220478671 priority { priority: 0 } creationTime: 1448220485305 nodeLabelExpression: "" containerType: APPLICATION_MASTER 2015-11-22 19:28:05,311 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(484)) - Not assigning more than one off-switch container, assignments so far: resource:; type:OFF_SWITCH; excessReservation:null; applicationid:null; skipped:false; fulfilled reservation:false; allocations(count/resource):1/; reservations(count/resource):0/ 2015-11-22 19:28:05,311 DEBUG [AsyncDispatcher event handler] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(197)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 1 cluster_timestamp: 1448220478671 } attemptId: 1 } id: 1 } nmHostAddr: "ip-172-31-20-42.ap-northeast-1.compute.internal:50663" appSubmitter: "ubuntu" resource { memory: 512 virtual_cores: 1 } expiryTimeStamp: 1448221085309 masterKeyId: -550760804 rmIdentifier: 1448220478671 priority { priority: 0 } creationTime: 1448220485305 nodeLabelExpression: "" containerType: APPLICATION_MASTER 2015-11-22 19:28:05,311 INFO [AsyncDispatcher event handler] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:createAndGetNMToken(200)) - Sending NMToken for nodeId : ip-172-31-20-42.ap-northeast-1.compute.internal:50663 for container : container_1448220478671_0001_01_000001 2015-11-22 19:28:05,312 DEBUG [AsyncDispatcher event handler] security.BaseNMTokenSecretManager (BaseNMTokenSecretManager.java:createPassword(75)) - creating password for appattempt_1448220478671_0001_000001 for user ubuntu to run on NM ip-172-31-20-42.ap-northeast-1.compute.internal:50663 2015-11-22 19:28:05,312 DEBUG [AsyncDispatcher event handler] security.NMTokenIdentifier (NMTokenIdentifier.java:write(96)) - Writing NMTokenIdentifier to RPC layer: appAttemptId { application_id { id: 1 cluster_timestamp: 1448220478671 } attemptId: 1 } nodeId { host: "ip-172-31-20-42.ap-northeast-1.compute.internal" port: 50663 } appSubmitter: "ubuntu" keyId: 744666633 2015-11-22 19:28:05,313 DEBUG [AsyncDispatcher event handler] security.NMTokenIdentifier (NMTokenIdentifier.java:write(96)) - Writing NMTokenIdentifier to RPC layer: appAttemptId { application_id { id: 1 cluster_timestamp: 1448220478671 } attemptId: 1 } nodeId { host: "ip-172-31-20-42.ap-northeast-1.compute.internal" port: 50663 } appSubmitter: "ubuntu" keyId: 744666633 2015-11-22 19:28:05,313 DEBUG [AsyncDispatcher event handler] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(424)) - Processing container_1448220478671_0001_01_000001 of type ACQUIRED 2015-11-22 19:28:05,313 INFO [AsyncDispatcher event handler] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220478671_0001_01_000001 Container Transitioned from ALLOCATED to ACQUIRED 2015-11-22 19:28:05,313 DEBUG [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:05,313 INFO [AsyncDispatcher event handler] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:clearNodeSetForAttempt(146)) - Clear node set for appattempt_1448220478671_0001_000001 2015-11-22 19:28:05,313 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:storeAttempt(2026)) - Storing attempt: AppId: application_1448220478671_0001 AttemptId: appattempt_1448220478671_0001_000001 MasterContainer: Container: [ContainerId: container_1448220478671_0001_01_000001, NodeId: ip-172-31-20-42.ap-northeast-1.compute.internal:50663, NodeHttpAddress: ip-172-31-20-42.ap-northeast-1.compute.internal:49669, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.31.20.42:50663 }, ] 2015-11-22 19:28:05,314 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220478671_0001_000001 State change from SCHEDULED to ALLOCATED_SAVING 2015-11-22 19:28:05,314 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStoreAppAttemptEvent.EventType: STORE_APP_ATTEMPT 2015-11-22 19:28:05,314 DEBUG [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:handleStoreEvent(1016)) - Processing event of type STORE_APP_ATTEMPT 2015-11-22 19:28:05,314 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppRunningOnNodeEvent.EventType: APP_RUNNING_ON_NODE 2015-11-22 19:28:05,314 DEBUG [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:transition(295)) - Storing info for attempt: appattempt_1448220478671_0001_000001 2015-11-22 19:28:05,314 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(777)) - Processing event for application_1448220478671_0001 of type APP_RUNNING_ON_NODE 2015-11-22 19:28:05,315 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: ATTEMPT_NEW_SAVED 2015-11-22 19:28:05,315 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type ATTEMPT_NEW_SAVED 2015-11-22 19:28:05,315 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220478671_0001_000001 State change from ALLOCATED_SAVING to ALLOCATED 2015-11-22 19:28:05,315 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncherEvent.EventType: LAUNCH 2015-11-22 19:28:05,315 INFO [ApplicationMasterLauncher #0] amlauncher.AMLauncher (AMLauncher.java:run(254)) - Launching masterappattempt_1448220478671_0001_000001 2015-11-22 19:28:05,315 DEBUG [ApplicationMasterLauncher #0] ipc.YarnRPC (YarnRPC.java:create(59)) - Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC 2015-11-22 19:28:05,315 DEBUG [ApplicationMasterLauncher #0] security.BaseNMTokenSecretManager (BaseNMTokenSecretManager.java:createPassword(75)) - creating password for appattempt_1448220478671_0001_000001 for user ubuntu to run on NM ip-172-31-20-42.ap-northeast-1.compute.internal:50663 2015-11-22 19:28:05,316 DEBUG [ApplicationMasterLauncher #0] security.NMTokenIdentifier (NMTokenIdentifier.java:write(96)) - Writing NMTokenIdentifier to RPC layer: appAttemptId { application_id { id: 1 cluster_timestamp: 1448220478671 } attemptId: 1 } nodeId { host: "ip-172-31-20-42.ap-northeast-1.compute.internal" port: 50663 } appSubmitter: "ubuntu" keyId: 744666633 2015-11-22 19:28:05,316 DEBUG [ApplicationMasterLauncher #0] security.NMTokenIdentifier (NMTokenIdentifier.java:write(96)) - Writing NMTokenIdentifier to RPC layer: appAttemptId { application_id { id: 1 cluster_timestamp: 1448220478671 } attemptId: 1 } nodeId { host: "ip-172-31-20-42.ap-northeast-1.compute.internal" port: 50663 } appSubmitter: "ubuntu" keyId: 744666633 2015-11-22 19:28:05,316 DEBUG [ApplicationMasterLauncher #0] security.SecurityUtil (SecurityUtil.java:setTokenService(389)) - Acquired token Kind: NMToken, Service: 172.31.20.42:50663, Ident: (appAttemptId { application_id { id: 1 cluster_timestamp: 1448220478671 } attemptId: 1 } nodeId { host: "ip-172-31-20-42.ap-northeast-1.compute.internal" port: 50663 } appSubmitter: "ubuntu" keyId: 744666633) 2015-11-22 19:28:05,316 DEBUG [ApplicationMasterLauncher #0] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:SIMPLE) from:org.apache.hadoop.yarn.client.ServerProxy.createRetriableProxy(ServerProxy.java:93) 2015-11-22 19:28:05,316 DEBUG [ApplicationMasterLauncher #0] ipc.HadoopYarnProtoRPC (HadoopYarnProtoRPC.java:getProxy(47)) - Creating a HadoopYarnProtoRpc proxy for protocol interface org.apache.hadoop.yarn.api.ContainerManagementProtocol 2015-11-22 19:28:05,317 DEBUG [ApplicationMasterLauncher #0] ipc.Client (ClientCache.java:getClient(63)) - getting client out of cache: org.apache.hadoop.ipc.Client@67cf996b 2015-11-22 19:28:05,317 INFO [ApplicationMasterLauncher #0] amlauncher.AMLauncher (AMLauncher.java:launch(106)) - Setting up container Container: [ContainerId: container_1448220478671_0001_01_000001, NodeId: ip-172-31-20-42.ap-northeast-1.compute.internal:50663, NodeHttpAddress: ip-172-31-20-42.ap-northeast-1.compute.internal:49669, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.31.20.42:50663 }, ] for AM appattempt_1448220478671_0001_000001 2015-11-22 19:28:05,317 INFO [ApplicationMasterLauncher #0] amlauncher.AMLauncher (AMLauncher.java:createAMContainerLaunchContext(188)) - Command to launch container container_1448220478671_0001_01_000001 : {{JAVA_HOME}}/bin/java -Xmx512m org.apache.hadoop.yarn.applications.distributedshell.ApplicationMaster --container_memory 128 --container_vcores 1 --num_containers 3 --priority 0 1>/AppMaster.stdout 2>/AppMaster.stderr 2015-11-22 19:28:05,317 INFO [ApplicationMasterLauncher #0] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:createAndGetAMRMToken(195)) - Create AMRMToken for ApplicationAttempt: appattempt_1448220478671_0001_000001 2015-11-22 19:28:05,318 INFO [ApplicationMasterLauncher #0] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:createPassword(307)) - Creating password for appattempt_1448220478671_0001_000001 2015-11-22 19:28:05,318 DEBUG [ApplicationMasterLauncher #0] ipc.Client (Client.java:(439)) - The ping interval is 60000 ms. 2015-11-22 19:28:05,318 DEBUG [ApplicationMasterLauncher #0] ipc.Client (Client.java:setupIOstreams(723)) - Connecting to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:50663 2015-11-22 19:28:05,319 DEBUG [ApplicationMasterLauncher #0] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:SIMPLE) from:org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:745) 2015-11-22 19:28:05,319 DEBUG [IPC Server listener on 50663] ipc.Server (Server.java:register(2883)) - Server connection from 172.31.20.42:35345; # active connections: 1; # queued calls: 0 2015-11-22 19:28:05,324 DEBUG [ApplicationMasterLauncher #0] security.SaslRpcClient (SaslRpcClient.java:sendSaslMessage(454)) - Sending sasl message state: NEGOTIATE 2015-11-22 19:28:05,325 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processOneRpc(1897)) - got #-33 2015-11-22 19:28:05,325 DEBUG [Socket Reader #1 for port 50663] security.SaslRpcServer (SaslRpcServer.java:create(175)) - Created SASL server with mechanism = DIGEST-MD5 2015-11-22 19:28:05,325 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processResponse(1099)) - Socket Reader #1 for port 50663: responding to null from 172.31.20.42:35345 Call#-33 Retry#-1 2015-11-22 19:28:05,325 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processResponse(1118)) - Socket Reader #1 for port 50663: responding to null from 172.31.20.42:35345 Call#-33 Retry#-1 Wrote 166 bytes. 2015-11-22 19:28:05,325 DEBUG [ApplicationMasterLauncher #0] security.SaslRpcClient (SaslRpcClient.java:getServerToken(264)) - Get token info proto:interface org.apache.hadoop.yarn.api.ContainerManagementProtocolPB info:org.apache.hadoop.yarn.security.ContainerManagerSecurityInfo$1@6d5cc728 2015-11-22 19:28:05,326 DEBUG [ApplicationMasterLauncher #0] security.NMTokenSelector (NMTokenSelector.java:selectToken(45)) - Looking for service: 172.31.20.42:50663. Current token is Kind: NMToken, Service: 172.31.20.42:50663, Ident: (appAttemptId { application_id { id: 1 cluster_timestamp: 1448220478671 } attemptId: 1 } nodeId { host: "ip-172-31-20-42.ap-northeast-1.compute.internal" port: 50663 } appSubmitter: "ubuntu" keyId: 744666633) 2015-11-22 19:28:05,326 DEBUG [ApplicationMasterLauncher #0] security.SaslRpcClient (SaslRpcClient.java:createSaslClient(247)) - Creating SASL DIGEST-MD5(TOKEN) client to authenticate to service at default 2015-11-22 19:28:05,326 DEBUG [ApplicationMasterLauncher #0] security.SaslRpcClient (SaslRpcClient.java:selectSaslClient(176)) - Use TOKEN authentication for protocol ContainerManagementProtocolPB 2015-11-22 19:28:05,327 DEBUG [ApplicationMasterLauncher #0] security.SaslRpcClient (SaslRpcClient.java:handle(680)) - SASL client callback: setting username: Cg0KCQgBEM+Zq4WTKhABEjUKL2lwLTE3Mi0zMS0yMC00Mi5hcC1ub3J0aGVhc3QtMS5jb21wdXRlLmludGVybmFsEOeLAxoGdWJ1bnR1IInsiuMC 2015-11-22 19:28:05,327 DEBUG [ApplicationMasterLauncher #0] security.SaslRpcClient (SaslRpcClient.java:handle(685)) - SASL client callback: setting userPassword 2015-11-22 19:28:05,327 DEBUG [ApplicationMasterLauncher #0] security.SaslRpcClient (SaslRpcClient.java:handle(690)) - SASL client callback: setting realm: default 2015-11-22 19:28:05,328 DEBUG [ApplicationMasterLauncher #0] security.SaslRpcClient (SaslRpcClient.java:sendSaslMessage(454)) - Sending sasl message state: INITIATE token: "charset=utf-8,username=\"Cg0KCQgBEM+Zq4WTKhABEjUKL2lwLTE3Mi0zMS0yMC00Mi5hcC1ub3J0aGVhc3QtMS5jb21wdXRlLmludGVybmFsEOeLAxoGdWJ1bnR1IInsiuMC\",realm=\"default\",nonce=\"x07Wd2joSHAHGTDJXlVkPnNSipeXLGSmZowgxqLr\",nc=00000001,cnonce=\"x8LH2aBhQQSC8AEyLzCP+oQHJBGLSZ2CGR97jz6w\",digest-uri=\"/default\",maxbuf=65536,response=f4c33e6fa550ac497f8d7e6751ebb394,qop=auth" auths { method: "TOKEN" mechanism: "DIGEST-MD5" protocol: "" serverId: "default" } 2015-11-22 19:28:05,328 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processOneRpc(1897)) - got #-33 2015-11-22 19:28:05,328 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processSaslToken(1529)) - Have read input token of size 350 for processing by saslServer.evaluateResponse() 2015-11-22 19:28:05,329 DEBUG [Socket Reader #1 for port 50663] security.BaseNMTokenSecretManager (BaseNMTokenSecretManager.java:retrivePasswordInternal(103)) - creating password for appattempt_1448220478671_0001_000001 for user ubuntu to run on NM ip-172-31-20-42.ap-northeast-1.compute.internal:50663 2015-11-22 19:28:05,329 DEBUG [Socket Reader #1 for port 50663] security.NMTokenIdentifier (NMTokenIdentifier.java:write(96)) - Writing NMTokenIdentifier to RPC layer: appAttemptId { application_id { id: 1 cluster_timestamp: 1448220478671 } attemptId: 1 } nodeId { host: "ip-172-31-20-42.ap-northeast-1.compute.internal" port: 50663 } appSubmitter: "ubuntu" keyId: 744666633 2015-11-22 19:28:05,329 DEBUG [Socket Reader #1 for port 50663] security.NMTokenSecretManagerInNM (NMTokenSecretManagerInNM.java:retrievePassword(180)) - NMToken password retrieved successfully!! 2015-11-22 19:28:05,329 DEBUG [Socket Reader #1 for port 50663] security.SaslRpcServer (SaslRpcServer.java:handle(310)) - SASL server DIGEST-MD5 callback: setting password for client: appattempt_1448220478671_0001_000001 (auth:SIMPLE) 2015-11-22 19:28:05,329 DEBUG [Socket Reader #1 for port 50663] security.SaslRpcServer (SaslRpcServer.java:handle(328)) - SASL server DIGEST-MD5 callback: setting canonicalized client ID: appattempt_1448220478671_0001_000001 2015-11-22 19:28:05,329 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:buildSaslResponse(1546)) - Will send SUCCESS token of size 40 from saslServer. 2015-11-22 19:28:05,329 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:saslProcess(1434)) - SASL server context established. Negotiated QoP is auth 2015-11-22 19:28:05,330 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:saslProcess(1439)) - SASL server successfully authenticated client: appattempt_1448220478671_0001_000001 (auth:SIMPLE) 2015-11-22 19:28:05,330 INFO [Socket Reader #1 for port 50663] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220478671_0001_000001 (auth:SIMPLE) 2015-11-22 19:28:05,330 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processResponse(1099)) - Socket Reader #1 for port 50663: responding to null from 172.31.20.42:35345 Call#-33 Retry#-1 2015-11-22 19:28:05,330 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processResponse(1118)) - Socket Reader #1 for port 50663: responding to null from 172.31.20.42:35345 Call#-33 Retry#-1 Wrote 64 bytes. 2015-11-22 19:28:05,330 DEBUG [ApplicationMasterLauncher #0] ipc.Client (Client.java:setupIOstreams(773)) - Negotiated QOP is :auth 2015-11-22 19:28:05,331 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:50663 from appattempt_1448220478671_0001_000001] ipc.Client (Client.java:run(1000)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:50663 from appattempt_1448220478671_0001_000001: starting, having connections 2 2015-11-22 19:28:05,331 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:50663 from appattempt_1448220478671_0001_000001 sending #818 2015-11-22 19:28:05,332 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processOneRpc(1897)) - got #-3 2015-11-22 19:28:05,332 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:authorizeConnection(2097)) - Successfully authorized userInfo { } protocol: "org.apache.hadoop.yarn.api.ContainerManagementProtocolPB" 2015-11-22 19:28:05,332 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processOneRpc(1897)) - got #818 2015-11-22 19:28:05,333 DEBUG [IPC Server handler 2 on 50663] ipc.Server (Server.java:run(2185)) - IPC Server handler 2 on 50663: org.apache.hadoop.yarn.api.ContainerManagementProtocolPB.startContainers from 172.31.20.42:35345 Call#818 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:05,333 DEBUG [IPC Server handler 2 on 50663] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:05,334 DEBUG [IPC Server handler 2 on 50663] security.BaseContainerTokenSecretManager (BaseContainerTokenSecretManager.java:retrievePasswordInternal(118)) - Retrieving password for container_1448220478671_0001_01_000001 for user container_1448220478671_0001_01_000001 (auth:SIMPLE) to be run on NM ip-172-31-20-42.ap-northeast-1.compute.internal:50663 2015-11-22 19:28:05,335 DEBUG [IPC Server handler 2 on 50663] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(197)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 1 cluster_timestamp: 1448220478671 } attemptId: 1 } id: 1 } nmHostAddr: "ip-172-31-20-42.ap-northeast-1.compute.internal:50663" appSubmitter: "ubuntu" resource { memory: 512 virtual_cores: 1 } expiryTimeStamp: 1448221085309 masterKeyId: -550760804 rmIdentifier: 1448220478671 priority { priority: 0 } creationTime: 1448220485305 nodeLabelExpression: "" containerType: APPLICATION_MASTER 2015-11-22 19:28:05,335 DEBUG [IPC Server handler 2 on 50663] security.NMTokenSecretManagerInNM (NMTokenSecretManagerInNM.java:appAttemptStartContainer(226)) - NMToken key updated for application attempt : appattempt_1448220478671_0001_000001 2015-11-22 19:28:05,335 INFO [IPC Server handler 2 on 50663] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:startContainerInternal(889)) - Start request for container_1448220478671_0001_01_000001 by user ubuntu 2015-11-22 19:28:05,335 DEBUG [IPC Server handler 2 on 50663] utils.YarnServerSecurityUtils (YarnServerSecurityUtils.java:parseCredentials(135)) - = Kind: YARN_AM_RM_TOKEN, Service: , Ident: (appAttemptId { application_id { id: 1 cluster_timestamp: 1448220478671 } attemptId: 1 } keyId: -1778540031) 2015-11-22 19:28:05,336 INFO [IPC Server handler 2 on 50663] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:startContainerInternal(930)) - Creating a new application reference for app application_1448220478671_0001 2015-11-22 19:28:05,336 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationInitEvent.EventType: INIT_APPLICATION 2015-11-22 19:28:05,336 INFO [IPC Server handler 2 on 50663] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu IP=172.31.20.42 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1448220478671_0001 CONTAINERID=container_1448220478671_0001_01_000001 2015-11-22 19:28:05,337 DEBUG [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(452)) - Processing application_1448220478671_0001 of type INIT_APPLICATION 2015-11-22 19:28:05,337 DEBUG [IPC Server handler 2 on 50663] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: startContainers queueTime= 0 procesingTime= 4 2015-11-22 19:28:05,337 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(463)) - Application application_1448220478671_0001 transitioned from NEW to INITING 2015-11-22 19:28:05,338 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationContainerInitEvent.EventType: INIT_CONTAINER 2015-11-22 19:28:05,338 DEBUG [IPC Server handler 2 on 50663] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 2 on 50663: responding to org.apache.hadoop.yarn.api.ContainerManagementProtocolPB.startContainers from 172.31.20.42:35345 Call#818 Retry#0 2015-11-22 19:28:05,339 DEBUG [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(452)) - Processing application_1448220478671_0001 of type INIT_CONTAINER 2015-11-22 19:28:05,339 DEBUG [IPC Server handler 2 on 50663] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 2 on 50663: responding to org.apache.hadoop.yarn.api.ContainerManagementProtocolPB.startContainers from 172.31.20.42:35345 Call#818 Retry#0 Wrote 52 bytes. 2015-11-22 19:28:05,339 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(303)) - Adding container_1448220478671_0001_01_000001 to application application_1448220478671_0001 2015-11-22 19:28:05,339 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:50663 from appattempt_1448220478671_0001_000001] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:50663 from appattempt_1448220478671_0001_000001 got value #818 2015-11-22 19:28:05,339 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.event.LogHandlerAppStartedEvent.EventType: APPLICATION_STARTED 2015-11-22 19:28:05,339 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationEvent.EventType: APPLICATION_LOG_HANDLING_INITED 2015-11-22 19:28:05,339 DEBUG [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(452)) - Processing application_1448220478671_0001 of type APPLICATION_LOG_HANDLING_INITED 2015-11-22 19:28:05,339 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.ApplicationLocalizationEvent.EventType: INIT_APPLICATION_RESOURCES 2015-11-22 19:28:05,340 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationInitedEvent.EventType: APPLICATION_INITED 2015-11-22 19:28:05,340 DEBUG [ApplicationMasterLauncher #0] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: startContainers took 22ms 2015-11-22 19:28:05,340 DEBUG [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(452)) - Processing application_1448220478671_0001 of type APPLICATION_INITED 2015-11-22 19:28:05,340 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(463)) - Application application_1448220478671_0001 transitioned from INITING to RUNNING 2015-11-22 19:28:05,340 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerInitEvent.EventType: INIT_CONTAINER 2015-11-22 19:28:05,340 DEBUG [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1131)) - Processing container_1448220478671_0001_01_000001 of type INIT_CONTAINER 2015-11-22 19:28:05,340 INFO [ApplicationMasterLauncher #0] amlauncher.AMLauncher (AMLauncher.java:launch(127)) - Done launching container Container: [ContainerId: container_1448220478671_0001_01_000001, NodeId: ip-172-31-20-42.ap-northeast-1.compute.internal:50663, NodeHttpAddress: ip-172-31-20-42.ap-northeast-1.compute.internal:49669, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.31.20.42:50663 }, ] for AM appattempt_1448220478671_0001_000001 2015-11-22 19:28:05,341 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: LAUNCHED 2015-11-22 19:28:05,341 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220478671_0001_01_000001 transitioned from NEW to LOCALIZING 2015-11-22 19:28:05,341 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type LAUNCHED 2015-11-22 19:28:05,341 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServicesEvent.EventType: CONTAINER_INIT 2015-11-22 19:28:05,341 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_INIT for appId application_1448220478671_0001 2015-11-22 19:28:05,341 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220478671_0001_000001 State change from ALLOCATED to LAUNCHED 2015-11-22 19:28:05,341 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.ContainerLocalizationRequestEvent.EventType: INIT_CONTAINER_RESOURCES 2015-11-22 19:28:05,342 DEBUG [AsyncDispatcher event handler] localizer.LocalizedResource (LocalizedResource.java:handle(193)) - Processing file:/home/ubuntu/DistributedShell/application_1448220478671_0001/AppMaster.jar of type REQUEST 2015-11-22 19:28:05,342 INFO [AsyncDispatcher event handler] localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource file:/home/ubuntu/DistributedShell/application_1448220478671_0001/AppMaster.jar transitioned from INIT to DOWNLOADING 2015-11-22 19:28:05,342 DEBUG [AsyncDispatcher event handler] localizer.ResourceLocalizationService (ResourceLocalizationService.java:handleInitContainerResources(480)) - Localizing file:/home/ubuntu/DistributedShell/application_1448220478671_0001/AppMaster.jar for container container_1448220478671_0001_01_000001 2015-11-22 19:28:05,342 DEBUG [AsyncDispatcher event handler] localizer.LocalizedResource (LocalizedResource.java:handle(193)) - Processing file:/home/ubuntu/DistributedShell/application_1448220478671_0001/shellCommands of type REQUEST 2015-11-22 19:28:05,342 INFO [AsyncDispatcher event handler] localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource file:/home/ubuntu/DistributedShell/application_1448220478671_0001/shellCommands transitioned from INIT to DOWNLOADING 2015-11-22 19:28:05,342 DEBUG [AsyncDispatcher event handler] localizer.ResourceLocalizationService (ResourceLocalizationService.java:handleInitContainerResources(480)) - Localizing file:/home/ubuntu/DistributedShell/application_1448220478671_0001/shellCommands for container container_1448220478671_0001_01_000001 2015-11-22 19:28:05,342 DEBUG [AsyncDispatcher event handler] localizer.LocalizedResource (LocalizedResource.java:handle(193)) - Processing file:/home/ubuntu/DistributedShell/application_1448220478671_0001/shellArgs of type REQUEST 2015-11-22 19:28:05,342 INFO [AsyncDispatcher event handler] localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource file:/home/ubuntu/DistributedShell/application_1448220478671_0001/shellArgs transitioned from INIT to DOWNLOADING 2015-11-22 19:28:05,342 DEBUG [AsyncDispatcher event handler] localizer.ResourceLocalizationService (ResourceLocalizationService.java:handleInitContainerResources(480)) - Localizing file:/home/ubuntu/DistributedShell/application_1448220478671_0001/shellArgs for container container_1448220478671_0001_01_000001 2015-11-22 19:28:05,343 DEBUG [AsyncDispatcher event handler] localizer.LocalizedResource (LocalizedResource.java:handle(193)) - Processing file:/home/ubuntu/DistributedShell/application_1448220478671_0001/log4j.properties of type REQUEST 2015-11-22 19:28:05,343 INFO [AsyncDispatcher event handler] localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource file:/home/ubuntu/DistributedShell/application_1448220478671_0001/log4j.properties transitioned from INIT to DOWNLOADING 2015-11-22 19:28:05,343 DEBUG [AsyncDispatcher event handler] localizer.ResourceLocalizationService (ResourceLocalizationService.java:handleInitContainerResources(480)) - Localizing file:/home/ubuntu/DistributedShell/application_1448220478671_0001/log4j.properties for container container_1448220478671_0001_01_000001 2015-11-22 19:28:05,343 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizerResourceRequestEvent.EventType: REQUEST_RESOURCE_LOCALIZATION 2015-11-22 19:28:05,343 INFO [AsyncDispatcher event handler] localizer.ResourceLocalizationService (ResourceLocalizationService.java:handle(742)) - Created localizer for container_1448220478671_0001_01_000001 2015-11-22 19:28:05,343 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizerResourceRequestEvent.EventType: REQUEST_RESOURCE_LOCALIZATION 2015-11-22 19:28:05,343 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizerResourceRequestEvent.EventType: REQUEST_RESOURCE_LOCALIZATION 2015-11-22 19:28:05,344 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizerResourceRequestEvent.EventType: REQUEST_RESOURCE_LOCALIZATION 2015-11-22 19:28:05,344 DEBUG [LocalizerRunner for container_1448220478671_0001_01_000001] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.fs.FileContext.getAbstractFileSystem(FileContext.java:339) 2015-11-22 19:28:05,348 INFO [LocalizerRunner for container_1448220478671_0001_01_000001] localizer.ResourceLocalizationService (ResourceLocalizationService.java:writeCredentials(1221)) - Writing credentials to the nmPrivate file /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/container_1448220478671_0001_01_000001.tokens. Credentials list: 2015-11-22 19:28:05,350 DEBUG [LocalizerRunner for container_1448220478671_0001_01_000001] localizer.ResourceLocalizationService (ResourceLocalizationService.java:writeCredentials(1226)) - Kind: YARN_AM_RM_TOKEN, Service: , Ident: (appAttemptId { application_id { id: 1 cluster_timestamp: 1448220478671 } attemptId: 1 } keyId: -1778540031) : be 76 de f5 19 ed 21 c6 95 f1 2015-11-22 19:28:05,351 INFO [LocalizerRunner for container_1448220478671_0001_01_000001] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:createUserCacheDirs(632)) - Initializing user ubuntu 2015-11-22 19:28:05,358 INFO [LocalizerRunner for container_1448220478671_0001_01_000001] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:startLocalizer(126)) - Copying from /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/container_1448220478671_0001_01_000001.tokens to /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220478671_0001/container_1448220478671_0001_01_000001.tokens 2015-11-22 19:28:05,358 INFO [LocalizerRunner for container_1448220478671_0001_01_000001] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:startLocalizer(133)) - Localizer CWD set to /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220478671_0001 = file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220478671_0001 2015-11-22 19:28:05,370 DEBUG [LocalizerRunner for container_1448220478671_0001_01_000001] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.runLocalization(ContainerLocalizer.java:152) 2015-11-22 19:28:05,370 DEBUG [LocalizerRunner for container_1448220478671_0001_01_000001] ipc.YarnRPC (YarnRPC.java:create(59)) - Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC 2015-11-22 19:28:05,370 DEBUG [LocalizerRunner for container_1448220478671_0001_01_000001] ipc.HadoopYarnProtoRPC (HadoopYarnProtoRPC.java:getProxy(47)) - Creating a HadoopYarnProtoRpc proxy for protocol interface org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocol 2015-11-22 19:28:05,370 DEBUG [LocalizerRunner for container_1448220478671_0001_01_000001] ipc.Client (ClientCache.java:getClient(63)) - getting client out of cache: org.apache.hadoop.ipc.Client@67cf996b 2015-11-22 19:28:05,371 DEBUG [LocalizerRunner for container_1448220478671_0001_01_000001] ipc.Client (Client.java:(439)) - The ping interval is 60000 ms. 2015-11-22 19:28:05,371 DEBUG [LocalizerRunner for container_1448220478671_0001_01_000001] ipc.Client (Client.java:setupIOstreams(723)) - Connecting to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:47651 2015-11-22 19:28:05,371 DEBUG [IPC Server listener on 47651] ipc.Server (Server.java:register(2883)) - Server connection from 172.31.20.42:40571; # active connections: 1; # queued calls: 0 2015-11-22 19:28:05,372 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:47651 from ubuntu sending #819 2015-11-22 19:28:05,372 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:47651 from ubuntu] ipc.Client (Client.java:run(1000)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:47651 from ubuntu: starting, having connections 3 2015-11-22 19:28:05,373 DEBUG [Socket Reader #1 for port 47651] ipc.Server (Server.java:processOneRpc(1897)) - got #-3 2015-11-22 19:28:05,373 DEBUG [Socket Reader #1 for port 47651] ipc.Server (Server.java:authorizeConnection(2097)) - Successfully authorized userInfo { effectiveUser: "ubuntu" } protocol: "org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB" 2015-11-22 19:28:05,373 DEBUG [Socket Reader #1 for port 47651] ipc.Server (Server.java:processOneRpc(1897)) - got #819 2015-11-22 19:28:05,374 DEBUG [IPC Server handler 0 on 47651] ipc.Server (Server.java:run(2185)) - IPC Server handler 0 on 47651: org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 172.31.20.42:40571 Call#819 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:05,374 DEBUG [IPC Server handler 0 on 47651] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:05,375 DEBUG [IPC Server handler 0 on 47651] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: heartbeat queueTime= 0 procesingTime= 1 2015-11-22 19:28:05,376 DEBUG [IPC Server handler 0 on 47651] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 0 on 47651: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 172.31.20.42:40571 Call#819 Retry#0 2015-11-22 19:28:05,377 DEBUG [IPC Server handler 0 on 47651] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 0 on 47651: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 172.31.20.42:40571 Call#819 Retry#0 Wrote 421 bytes. 2015-11-22 19:28:05,377 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:47651 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:47651 from ubuntu got value #819 2015-11-22 19:28:05,377 DEBUG [LocalizerRunner for container_1448220478671_0001_01_000001] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: heartbeat took 6ms 2015-11-22 19:28:05,381 DEBUG [ContainerLocalizer Downloader] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:359) 2015-11-22 19:28:05,385 DEBUG [ContainerLocalizer Downloader] util.FSDownload (FSDownload.java:changePermissions(412)) - Changing permissions for path file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220478671_0001/filecache/10_tmp/AppMaster.jar to perm r-x------ 2015-11-22 19:28:05,386 DEBUG [ContainerLocalizer Downloader] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.yarn.util.FSDownload.changePermissions(FSDownload.java:419) 2015-11-22 19:28:05,391 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:47651 from ubuntu sending #820 2015-11-22 19:28:05,392 DEBUG [Socket Reader #1 for port 47651] ipc.Server (Server.java:processOneRpc(1897)) - got #820 2015-11-22 19:28:05,392 DEBUG [IPC Server handler 2 on 47651] ipc.Server (Server.java:run(2185)) - IPC Server handler 2 on 47651: org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 172.31.20.42:40571 Call#820 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:05,396 DEBUG [IPC Server handler 2 on 47651] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:05,397 DEBUG [IPC Server handler 2 on 47651] localizer.LocalizedResource (LocalizedResource.java:handle(193)) - Processing file:/home/ubuntu/DistributedShell/application_1448220478671_0001/AppMaster.jar of type LOCALIZED 2015-11-22 19:28:05,397 INFO [IPC Server handler 2 on 47651] localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource file:/home/ubuntu/DistributedShell/application_1448220478671_0001/AppMaster.jar(->/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220478671_0001/filecache/10/AppMaster.jar) transitioned from DOWNLOADING to LOCALIZED 2015-11-22 19:28:05,397 DEBUG [IPC Server handler 2 on 47651] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: heartbeat queueTime= 4 procesingTime= 1 2015-11-22 19:28:05,398 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerResourceLocalizedEvent.EventType: RESOURCE_LOCALIZED 2015-11-22 19:28:05,398 DEBUG [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1131)) - Processing container_1448220478671_0001_01_000001 of type RESOURCE_LOCALIZED 2015-11-22 19:28:05,398 DEBUG [IPC Server handler 2 on 47651] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 2 on 47651: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 172.31.20.42:40571 Call#820 Retry#0 2015-11-22 19:28:05,399 DEBUG [IPC Server handler 2 on 47651] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 2 on 47651: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 172.31.20.42:40571 Call#820 Retry#0 Wrote 421 bytes. 2015-11-22 19:28:05,399 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:47651 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:47651 from ubuntu got value #820 2015-11-22 19:28:05,399 DEBUG [LocalizerRunner for container_1448220478671_0001_01_000001] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: heartbeat took 8ms 2015-11-22 19:28:05,405 DEBUG [ContainerLocalizer Downloader] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:359) 2015-11-22 19:28:05,408 DEBUG [ContainerLocalizer Downloader] util.FSDownload (FSDownload.java:changePermissions(412)) - Changing permissions for path file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220478671_0001/filecache/11_tmp/shellCommands to perm r-x------ 2015-11-22 19:28:05,408 DEBUG [ContainerLocalizer Downloader] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.yarn.util.FSDownload.changePermissions(FSDownload.java:419) 2015-11-22 19:28:05,414 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:47651 from ubuntu sending #821 2015-11-22 19:28:05,414 DEBUG [Socket Reader #1 for port 47651] ipc.Server (Server.java:processOneRpc(1897)) - got #821 2015-11-22 19:28:05,414 DEBUG [IPC Server handler 3 on 47651] ipc.Server (Server.java:run(2185)) - IPC Server handler 3 on 47651: org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 172.31.20.42:40571 Call#821 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:05,414 DEBUG [IPC Server handler 3 on 47651] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:05,414 DEBUG [IPC Server handler 3 on 47651] localizer.LocalizedResource (LocalizedResource.java:handle(193)) - Processing file:/home/ubuntu/DistributedShell/application_1448220478671_0001/shellCommands of type LOCALIZED 2015-11-22 19:28:05,414 INFO [IPC Server handler 3 on 47651] localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource file:/home/ubuntu/DistributedShell/application_1448220478671_0001/shellCommands(->/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220478671_0001/filecache/11/shellCommands) transitioned from DOWNLOADING to LOCALIZED 2015-11-22 19:28:05,414 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerResourceLocalizedEvent.EventType: RESOURCE_LOCALIZED 2015-11-22 19:28:05,415 DEBUG [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1131)) - Processing container_1448220478671_0001_01_000001 of type RESOURCE_LOCALIZED 2015-11-22 19:28:05,415 DEBUG [IPC Server handler 3 on 47651] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: heartbeat queueTime= 0 procesingTime= 1 2015-11-22 19:28:05,415 DEBUG [IPC Server handler 3 on 47651] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 3 on 47651: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 172.31.20.42:40571 Call#821 Retry#0 2015-11-22 19:28:05,416 DEBUG [IPC Server handler 3 on 47651] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 3 on 47651: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 172.31.20.42:40571 Call#821 Retry#0 Wrote 417 bytes. 2015-11-22 19:28:05,416 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:47651 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:47651 from ubuntu got value #821 2015-11-22 19:28:05,416 DEBUG [LocalizerRunner for container_1448220478671_0001_01_000001] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: heartbeat took 2ms 2015-11-22 19:28:05,421 DEBUG [ContainerLocalizer Downloader] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:359) 2015-11-22 19:28:05,424 DEBUG [ContainerLocalizer Downloader] util.FSDownload (FSDownload.java:changePermissions(412)) - Changing permissions for path file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220478671_0001/filecache/12_tmp/shellArgs to perm r-x------ 2015-11-22 19:28:05,424 DEBUG [ContainerLocalizer Downloader] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.yarn.util.FSDownload.changePermissions(FSDownload.java:419) 2015-11-22 19:28:05,430 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:47651 from ubuntu sending #822 2015-11-22 19:28:05,430 DEBUG [Socket Reader #1 for port 47651] ipc.Server (Server.java:processOneRpc(1897)) - got #822 2015-11-22 19:28:05,432 DEBUG [IPC Server handler 4 on 47651] ipc.Server (Server.java:run(2185)) - IPC Server handler 4 on 47651: org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 172.31.20.42:40571 Call#822 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:05,432 DEBUG [IPC Server handler 4 on 47651] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:05,432 DEBUG [IPC Server handler 4 on 47651] localizer.LocalizedResource (LocalizedResource.java:handle(193)) - Processing file:/home/ubuntu/DistributedShell/application_1448220478671_0001/shellArgs of type LOCALIZED 2015-11-22 19:28:05,432 INFO [IPC Server handler 4 on 47651] localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource file:/home/ubuntu/DistributedShell/application_1448220478671_0001/shellArgs(->/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220478671_0001/filecache/12/shellArgs) transitioned from DOWNLOADING to LOCALIZED 2015-11-22 19:28:05,432 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerResourceLocalizedEvent.EventType: RESOURCE_LOCALIZED 2015-11-22 19:28:05,432 DEBUG [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1131)) - Processing container_1448220478671_0001_01_000001 of type RESOURCE_LOCALIZED 2015-11-22 19:28:05,433 DEBUG [IPC Server handler 4 on 47651] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: heartbeat queueTime= 2 procesingTime= 1 2015-11-22 19:28:05,433 DEBUG [IPC Server handler 4 on 47651] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 4 on 47651: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 172.31.20.42:40571 Call#822 Retry#0 2015-11-22 19:28:05,433 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:47651 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:47651 from ubuntu got value #822 2015-11-22 19:28:05,433 DEBUG [IPC Server handler 4 on 47651] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 4 on 47651: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 172.31.20.42:40571 Call#822 Retry#0 Wrote 424 bytes. 2015-11-22 19:28:05,433 DEBUG [LocalizerRunner for container_1448220478671_0001_01_000001] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: heartbeat took 3ms 2015-11-22 19:28:05,437 DEBUG [ContainerLocalizer Downloader] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:359) 2015-11-22 19:28:05,440 DEBUG [ContainerLocalizer Downloader] util.FSDownload (FSDownload.java:changePermissions(412)) - Changing permissions for path file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220478671_0001/filecache/13_tmp/log4j.properties to perm r-x------ 2015-11-22 19:28:05,440 DEBUG [ContainerLocalizer Downloader] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.yarn.util.FSDownload.changePermissions(FSDownload.java:419) 2015-11-22 19:28:05,446 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:47651 from ubuntu sending #823 2015-11-22 19:28:05,448 DEBUG [Socket Reader #1 for port 47651] ipc.Server (Server.java:processOneRpc(1897)) - got #823 2015-11-22 19:28:05,448 DEBUG [IPC Server handler 1 on 47651] ipc.Server (Server.java:run(2185)) - IPC Server handler 1 on 47651: org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 172.31.20.42:40571 Call#823 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:05,448 DEBUG [IPC Server handler 1 on 47651] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:05,448 DEBUG [IPC Server handler 1 on 47651] localizer.LocalizedResource (LocalizedResource.java:handle(193)) - Processing file:/home/ubuntu/DistributedShell/application_1448220478671_0001/log4j.properties of type LOCALIZED 2015-11-22 19:28:05,448 INFO [IPC Server handler 1 on 47651] localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource file:/home/ubuntu/DistributedShell/application_1448220478671_0001/log4j.properties(->/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220478671_0001/filecache/13/log4j.properties) transitioned from DOWNLOADING to LOCALIZED 2015-11-22 19:28:05,448 DEBUG [IPC Server handler 1 on 47651] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: heartbeat queueTime= 0 procesingTime= 0 2015-11-22 19:28:05,449 DEBUG [IPC Server handler 1 on 47651] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 1 on 47651: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 172.31.20.42:40571 Call#823 Retry#0 2015-11-22 19:28:05,449 DEBUG [IPC Server handler 1 on 47651] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 1 on 47651: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 172.31.20.42:40571 Call#823 Retry#0 Wrote 35 bytes. 2015-11-22 19:28:05,449 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerResourceLocalizedEvent.EventType: RESOURCE_LOCALIZED 2015-11-22 19:28:05,449 DEBUG [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1131)) - Processing container_1448220478671_0001_01_000001 of type RESOURCE_LOCALIZED 2015-11-22 19:28:05,449 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220478671_0001_01_000001 transitioned from LOCALIZING to LOCALIZED 2015-11-22 19:28:05,449 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.ContainerLocalizationEvent.EventType: CONTAINER_RESOURCES_LOCALIZED 2015-11-22 19:28:05,450 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncherEvent.EventType: LAUNCH_CONTAINER 2015-11-22 19:28:05,450 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.sharedcache.SharedCacheUploadEvent.EventType: UPLOAD 2015-11-22 19:28:05,450 DEBUG [ContainersLauncher #0] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.fs.FileContext.getAbstractFileSystem(FileContext.java:339) 2015-11-22 19:28:05,456 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:47651 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:47651 from ubuntu got value #823 2015-11-22 19:28:05,456 DEBUG [LocalizerRunner for container_1448220478671_0001_01_000001] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: heartbeat took 10ms 2015-11-22 19:28:05,473 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerEvent.EventType: CONTAINER_LAUNCHED 2015-11-22 19:28:05,474 DEBUG [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1131)) - Processing container_1448220478671_0001_01_000001 of type CONTAINER_LAUNCHED 2015-11-22 19:28:05,474 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220478671_0001_01_000001 transitioned from LOCALIZED to RUNNING 2015-11-22 19:28:05,474 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerStartMonitoringEvent.EventType: START_MONITORING_CONTAINER 2015-11-22 19:28:05,491 INFO [ContainersLauncher #0] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:buildCommandExecutor(281)) - launchContainer: [nice, -n, 0, bash, /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220478671_0001/container_1448220478671_0001_01_000001/default_container_executor.sh] 2015-11-22 19:28:05,793 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #824 2015-11-22 19:28:05,793 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #824 2015-11-22 19:28:05,793 DEBUG [IPC Server handler 7 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 7 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#824 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:05,794 DEBUG [IPC Server handler 7 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:05,794 DEBUG [IPC Server handler 7 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:28:05,794 DEBUG [IPC Server handler 7 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 1 procesingTime= 0 2015-11-22 19:28:05,796 DEBUG [IPC Server handler 7 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 7 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#824 Retry#0 2015-11-22 19:28:05,796 DEBUG [IPC Server handler 7 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 7 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#824 Retry#0 Wrote 333 bytes. 2015-11-22 19:28:05,796 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #824 2015-11-22 19:28:05,796 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 3ms 2015-11-22 19:28:05,796 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=N/A, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=ACCEPTED, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:28:06,301 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:28:06,302 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:28:06,302 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:06,302 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:28:06,302 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handleContainerStatus(1208)) - Container container_1448220478671_0001_01_000001 is the first container get launched for application application_1448220478671_0001 2015-11-22 19:28:06,302 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:28:06,302 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:28:06,303 DEBUG [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(424)) - Processing container_1448220478671_0001_01_000001 of type LAUNCHED 2015-11-22 19:28:06,303 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220478671_0001_01_000001 Container Transitioned from ACQUIRED to RUNNING 2015-11-22 19:28:06,303 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:28:06,303 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:28:06,303 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:28:06,456 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:47651 from ubuntu sending #825 2015-11-22 19:28:06,457 DEBUG [Socket Reader #1 for port 47651] ipc.Server (Server.java:processOneRpc(1897)) - got #825 2015-11-22 19:28:06,457 DEBUG [IPC Server handler 0 on 47651] ipc.Server (Server.java:run(2185)) - IPC Server handler 0 on 47651: org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 172.31.20.42:40571 Call#825 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:06,457 DEBUG [IPC Server handler 0 on 47651] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:06,457 DEBUG [IPC Server handler 0 on 47651] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: heartbeat queueTime= 0 procesingTime= 0 2015-11-22 19:28:06,457 DEBUG [IPC Server handler 0 on 47651] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 0 on 47651: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 172.31.20.42:40571 Call#825 Retry#0 2015-11-22 19:28:06,457 DEBUG [IPC Server handler 0 on 47651] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 0 on 47651: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 172.31.20.42:40571 Call#825 Retry#0 Wrote 35 bytes. 2015-11-22 19:28:06,458 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:47651 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:47651 from ubuntu got value #825 2015-11-22 19:28:06,458 DEBUG [LocalizerRunner for container_1448220478671_0001_01_000001] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: heartbeat took 2ms 2015-11-22 19:28:06,458 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:47651 from ubuntu sending #826 2015-11-22 19:28:06,458 DEBUG [Socket Reader #1 for port 47651] ipc.Server (Server.java:processOneRpc(1897)) - got #826 2015-11-22 19:28:06,458 DEBUG [IPC Server handler 2 on 47651] ipc.Server (Server.java:run(2185)) - IPC Server handler 2 on 47651: org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 172.31.20.42:40571 Call#826 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:06,458 DEBUG [IPC Server handler 2 on 47651] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:06,458 DEBUG [IPC Server handler 2 on 47651] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: heartbeat queueTime= 0 procesingTime= 0 2015-11-22 19:28:06,459 DEBUG [IPC Server handler 2 on 47651] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 2 on 47651: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 172.31.20.42:40571 Call#826 Retry#0 2015-11-22 19:28:06,459 DEBUG [IPC Server handler 2 on 47651] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 2 on 47651: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 172.31.20.42:40571 Call#826 Retry#0 Wrote 35 bytes. 2015-11-22 19:28:06,459 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:47651 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:47651 from ubuntu got value #826 2015-11-22 19:28:06,459 DEBUG [LocalizerRunner for container_1448220478671_0001_01_000001] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: heartbeat took 1ms 2015-11-22 19:28:06,480 DEBUG [DeletionService #0] nodemanager.DeletionService (DeletionService.java:run(266)) - FileDeletionTask : user : null subDir : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/container_1448220478671_0001_01_000001.tokens baseDir : null 2015-11-22 19:28:06,481 DEBUG [DeletionService #0] nodemanager.DeletionService (DeletionService.java:run(271)) - NM deleting absolute path : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/container_1448220478671_0001_01_000001.tokens 2015-11-22 19:28:06,797 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #827 2015-11-22 19:28:06,797 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #827 2015-11-22 19:28:06,797 DEBUG [IPC Server handler 8 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 8 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#827 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:06,797 DEBUG [IPC Server handler 8 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:06,798 DEBUG [IPC Server handler 8 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:28:06,798 DEBUG [IPC Server handler 8 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 1 procesingTime= 0 2015-11-22 19:28:06,798 DEBUG [IPC Server handler 8 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 8 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#827 Retry#0 2015-11-22 19:28:06,798 DEBUG [IPC Server handler 8 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 8 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#827 Retry#0 Wrote 333 bytes. 2015-11-22 19:28:06,799 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #827 2015-11-22 19:28:06,799 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:28:06,799 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=N/A, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=ACCEPTED, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:28:07,302 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:28:07,302 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:28:07,303 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:07,303 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:28:07,303 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:28:07,303 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:28:07,303 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:28:07,304 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:28:07,304 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:28:07,469 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - REQUEST /ws/v1/timeline/ on org.mortbay.jetty.HttpConnection@37a181f9 2015-11-22 19:28:07,470 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - sessionManager=org.mortbay.jetty.servlet.HashSessionManager@3966697d 2015-11-22 19:28:07,470 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - session=null 2015-11-22 19:28:07,470 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - servlet=org.mortbay.jetty.servlet.DefaultServlet-55176785 2015-11-22 19:28:07,471 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - chain=NoCacheFilter->NoCacheFilter->safety->Timeline Authentication Filter->RMAuthenticationFilter->static_user_filter->guice->org.mortbay.jetty.servlet.DefaultServlet-55176785 2015-11-22 19:28:07,471 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - servlet holder=org.mortbay.jetty.servlet.DefaultServlet-55176785 2015-11-22 19:28:07,471 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter NoCacheFilter 2015-11-22 19:28:07,472 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter NoCacheFilter 2015-11-22 19:28:07,472 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter safety 2015-11-22 19:28:07,472 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter Timeline Authentication Filter 2015-11-22 19:28:07,472 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(557)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:41381/ws/v1/timeline/?user.name=ubuntu] triggering authentication 2015-11-22 19:28:07,472 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(576)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:41381/ws/v1/timeline/?user.name=ubuntu] user [ubuntu] authenticated 2015-11-22 19:28:07,473 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter RMAuthenticationFilter 2015-11-22 19:28:07,473 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(557)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:41381/ws/v1/timeline/?user.name=ubuntu] triggering authentication 2015-11-22 19:28:07,473 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(576)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:41381/ws/v1/timeline/?user.name=ubuntu] user [ubuntu] authenticated 2015-11-22 19:28:07,473 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter static_user_filter 2015-11-22 19:28:07,473 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter guice 2015-11-22 19:28:07,478 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - getResource(META-INF/services/javax.xml.bind.JAXBContext)=jar:file:/home/ubuntu/.m2/repository/com/sun/xml/bind/jaxb-impl/2.2.3-1/jaxb-impl-2.2.3-1.jar!/META-INF/services/javax.xml.bind.JAXBContext 2015-11-22 19:28:07,479 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.xml.bind.v2.ContextFactory from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:28:07,519 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - RESPONSE /ws/v1/timeline/ 200 2015-11-22 19:28:07,788 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - REQUEST /ws/v1/timeline/ on org.mortbay.jetty.HttpConnection@6aba846 2015-11-22 19:28:07,788 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - sessionManager=org.mortbay.jetty.servlet.HashSessionManager@3966697d 2015-11-22 19:28:07,788 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - session=null 2015-11-22 19:28:07,788 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - servlet=org.mortbay.jetty.servlet.DefaultServlet-55176785 2015-11-22 19:28:07,789 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - chain=NoCacheFilter->NoCacheFilter->safety->Timeline Authentication Filter->RMAuthenticationFilter->static_user_filter->guice->org.mortbay.jetty.servlet.DefaultServlet-55176785 2015-11-22 19:28:07,789 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - servlet holder=org.mortbay.jetty.servlet.DefaultServlet-55176785 2015-11-22 19:28:07,789 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter NoCacheFilter 2015-11-22 19:28:07,789 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter NoCacheFilter 2015-11-22 19:28:07,789 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter safety 2015-11-22 19:28:07,790 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter Timeline Authentication Filter 2015-11-22 19:28:07,790 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(576)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:41381/ws/v1/timeline/] user [ubuntu] authenticated 2015-11-22 19:28:07,790 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter RMAuthenticationFilter 2015-11-22 19:28:07,791 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(576)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:41381/ws/v1/timeline/] user [ubuntu] authenticated 2015-11-22 19:28:07,791 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter static_user_filter 2015-11-22 19:28:07,791 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter guice 2015-11-22 19:28:07,799 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #828 2015-11-22 19:28:07,800 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #828 2015-11-22 19:28:07,800 DEBUG [IPC Server handler 10 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 10 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#828 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:07,800 DEBUG [IPC Server handler 10 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:07,800 DEBUG [IPC Server handler 10 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:28:07,801 DEBUG [IPC Server handler 10 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 1 2015-11-22 19:28:07,801 DEBUG [IPC Server handler 10 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 10 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#828 Retry#0 2015-11-22 19:28:07,801 DEBUG [IPC Server handler 10 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 10 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#828 Retry#0 Wrote 333 bytes. 2015-11-22 19:28:07,801 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #828 2015-11-22 19:28:07,801 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:28:07,802 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=N/A, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=ACCEPTED, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:28:07,811 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] security.TimelineACLsManager (TimelineACLsManager.java:checkAccess(106)) - Verifying the access of ubuntu on the timeline entity { id: appattempt_1448220478671_0001_000001, type: DS_APP_ATTEMPT } 2015-11-22 19:28:07,823 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - RESPONSE /ws/v1/timeline/ 200 2015-11-22 19:28:07,965 DEBUG [2048592079@qtp-847268847-0] mortbay.log (Slf4jLog.java:debug(40)) - EOF 2015-11-22 19:28:07,968 DEBUG [IPC Server listener on 46284] ipc.Server (Server.java:register(2883)) - Server connection from 172.31.20.42:53059; # active connections: 1; # queued calls: 0 2015-11-22 19:28:07,993 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #-33 2015-11-22 19:28:07,993 DEBUG [Socket Reader #1 for port 46284] security.SaslRpcServer (SaslRpcServer.java:create(175)) - Created SASL server with mechanism = DIGEST-MD5 2015-11-22 19:28:07,994 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processResponse(1099)) - Socket Reader #1 for port 46284: responding to null from 172.31.20.42:53059 Call#-33 Retry#-1 2015-11-22 19:28:07,994 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processResponse(1118)) - Socket Reader #1 for port 46284: responding to null from 172.31.20.42:53059 Call#-33 Retry#-1 Wrote 166 bytes. 2015-11-22 19:28:08,017 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #-33 2015-11-22 19:28:08,017 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processSaslToken(1529)) - Have read input token of size 274 for processing by saslServer.evaluateResponse() 2015-11-22 19:28:08,017 DEBUG [Socket Reader #1 for port 46284] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:retrievePassword(247)) - Trying to retrieve password for appattempt_1448220478671_0001_000001 2015-11-22 19:28:08,018 DEBUG [Socket Reader #1 for port 46284] security.SaslRpcServer (SaslRpcServer.java:handle(310)) - SASL server DIGEST-MD5 callback: setting password for client: appattempt_1448220478671_0001_000001 (auth:SIMPLE) 2015-11-22 19:28:08,018 DEBUG [Socket Reader #1 for port 46284] security.SaslRpcServer (SaslRpcServer.java:handle(328)) - SASL server DIGEST-MD5 callback: setting canonicalized client ID: appattempt_1448220478671_0001_000001 2015-11-22 19:28:08,018 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:buildSaslResponse(1546)) - Will send SUCCESS token of size 40 from saslServer. 2015-11-22 19:28:08,018 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:saslProcess(1434)) - SASL server context established. Negotiated QoP is auth 2015-11-22 19:28:08,018 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:saslProcess(1439)) - SASL server successfully authenticated client: appattempt_1448220478671_0001_000001 (auth:SIMPLE) 2015-11-22 19:28:08,019 INFO [Socket Reader #1 for port 46284] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220478671_0001_000001 (auth:SIMPLE) 2015-11-22 19:28:08,019 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processResponse(1099)) - Socket Reader #1 for port 46284: responding to null from 172.31.20.42:53059 Call#-33 Retry#-1 2015-11-22 19:28:08,019 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processResponse(1118)) - Socket Reader #1 for port 46284: responding to null from 172.31.20.42:53059 Call#-33 Retry#-1 Wrote 64 bytes. 2015-11-22 19:28:08,026 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #-3 2015-11-22 19:28:08,026 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:authorizeConnection(2097)) - Successfully authorized userInfo { } protocol: "org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB" 2015-11-22 19:28:08,026 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #0 2015-11-22 19:28:08,027 DEBUG [IPC Server handler 0 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 0 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.registerApplicationMaster from 172.31.20.42:53059 Call#0 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:08,028 DEBUG [IPC Server handler 0 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:08,029 INFO [IPC Server handler 0 on 46284] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:registerApplicationMaster(220)) - AM registration appattempt_1448220478671_0001_000001 2015-11-22 19:28:08,029 INFO [IPC Server handler 0 on 46284] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(133)) - USER=ubuntu IP=172.31.20.42 OPERATION=Register App Master TARGET=ApplicationMasterService RESULT=SUCCESS APPID=application_1448220478671_0001 APPATTEMPTID=appattempt_1448220478671_0001_000001 2015-11-22 19:28:08,029 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptRegistrationEvent.EventType: REGISTERED 2015-11-22 19:28:08,029 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type REGISTERED 2015-11-22 19:28:08,029 DEBUG [IPC Server handler 0 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: registerApplicationMaster queueTime= 1 procesingTime= 1 2015-11-22 19:28:08,038 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220478671_0001_000001 State change from LAUNCHED to RUNNING 2015-11-22 19:28:08,040 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEvent.EventType: ATTEMPT_REGISTERED 2015-11-22 19:28:08,040 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(777)) - Processing event for application_1448220478671_0001 of type ATTEMPT_REGISTERED 2015-11-22 19:28:08,040 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220478671_0001 State change from ACCEPTED to RUNNING 2015-11-22 19:28:08,047 DEBUG [IPC Server handler 0 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 0 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.registerApplicationMaster from 172.31.20.42:53059 Call#0 Retry#0 2015-11-22 19:28:08,048 DEBUG [IPC Server handler 0 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 0 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.registerApplicationMaster from 172.31.20.42:53059 Call#0 Retry#0 Wrote 50 bytes. 2015-11-22 19:28:08,092 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #1 2015-11-22 19:28:08,093 DEBUG [IPC Server handler 2 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 2 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#1 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:08,093 DEBUG [IPC Server handler 2 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:08,094 DEBUG [IPC Server handler 2 on 46284] capacity.CapacityScheduler (CapacityScheduler.java:allocate(941)) - allocate: pre-update appattempt_1448220478671_0001_000001 ask size =1 2015-11-22 19:28:08,094 DEBUG [IPC Server handler 2 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:08,094 DEBUG [IPC Server handler 2 on 46284] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(470)) - showRequests: application=application_1448220478671_0001 headRoom= currentConsumption=512 2015-11-22 19:28:08,095 DEBUG [IPC Server handler 2 on 46284] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(474)) - showRequests: application=application_1448220478671_0001 request={Priority: 0, Capability: , # Containers: 0, Location: *, Relax Locality: true, Node Label Expression: } 2015-11-22 19:28:08,095 DEBUG [IPC Server handler 2 on 46284] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:updateResourceRequests(320)) - update: application=application_1448220478671_0001 request={Priority: 0, Capability: , # Containers: 1, Location: *, Relax Locality: true, Node Label Expression: } 2015-11-22 19:28:08,095 DEBUG [IPC Server handler 2 on 46284] scheduler.ActiveUsersManager (ActiveUsersManager.java:activateApplication(68)) - User ubuntu added to activeUsers, currently: 1 2015-11-22 19:28:08,095 DEBUG [IPC Server handler 2 on 46284] capacity.CapacityScheduler (CapacityScheduler.java:allocate(952)) - allocate: post-update 2015-11-22 19:28:08,095 DEBUG [IPC Server handler 2 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2015-11-22 19:28:08,095 DEBUG [IPC Server handler 2 on 46284] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(470)) - showRequests: application=application_1448220478671_0001 headRoom= currentConsumption=512 2015-11-22 19:28:08,095 DEBUG [IPC Server handler 2 on 46284] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(474)) - showRequests: application=application_1448220478671_0001 request={Priority: 0, Capability: , # Containers: 1, Location: *, Relax Locality: true, Node Label Expression: } 2015-11-22 19:28:08,095 DEBUG [IPC Server handler 2 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2015-11-22 19:28:08,096 DEBUG [IPC Server handler 2 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 1 procesingTime= 3 2015-11-22 19:28:08,096 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:08,096 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:28:08,097 DEBUG [IPC Server handler 2 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 2 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#1 Retry#0 2015-11-22 19:28:08,097 DEBUG [IPC Server handler 2 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 2 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#1 Retry#0 Wrote 47 bytes. 2015-11-22 19:28:08,303 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:28:08,303 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:28:08,303 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:08,304 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:28:08,304 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:28:08,304 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:28:08,304 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:28:08,304 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:28:08,304 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(408)) - Trying to assign containers to child-queue of root 2015-11-22 19:28:08,304 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:printChildQueues(617)) - printChildQueues - queue: root child-queues: root.defaultusedCapacity=(0.125), label=(*) 2015-11-22 19:28:08,304 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(564)) - Trying to assign to queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.125, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 2015-11-22 19:28:08,304 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:assignContainers(817)) - assignContainers: node=ip-172-31-20-42.ap-northeast-1.compute.internal #applications=1 2015-11-22 19:28:08,305 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2015-11-22 19:28:08,305 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimitAndSetHeadroom(984)) - Headroom calculation for user ubuntu: userLimit= queueMaxAvailRes= consumed= headroom= 2015-11-22 19:28:08,305 DEBUG [ResourceManager Event Processor] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:assignContainers(477)) - pre-assignContainers for application application_1448220478671_0001 2015-11-22 19:28:08,305 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2015-11-22 19:28:08,305 DEBUG [ResourceManager Event Processor] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(470)) - showRequests: application=application_1448220478671_0001 headRoom= currentConsumption=512 2015-11-22 19:28:08,306 DEBUG [ResourceManager Event Processor] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(474)) - showRequests: application=application_1448220478671_0001 request={Priority: 0, Capability: , # Containers: 1, Location: *, Relax Locality: true, Node Label Expression: } 2015-11-22 19:28:08,306 DEBUG [ResourceManager Event Processor] allocator.IncreaseContainerAllocator (IncreaseContainerAllocator.java:assignContainers(184)) - Skip allocating increase request since we don't have any increase request on this node=ip-172-31-20-42.ap-northeast-1.compute.internal:50663 2015-11-22 19:28:08,306 DEBUG [ResourceManager Event Processor] allocator.RegularContainerAllocator (RegularContainerAllocator.java:assignContainer(407)) - assignContainers: node=ip-172-31-20-42.ap-northeast-1.compute.internal application=application_1448220478671_0001 priority=0 request={Priority: 0, Capability: , # Containers: 1, Location: *, Relax Locality: true, Node Label Expression: } type=OFF_SWITCH 2015-11-22 19:28:08,307 DEBUG [ResourceManager Event Processor] scheduler.ActiveUsersManager (ActiveUsersManager.java:deactivateApplication(94)) - User ubuntu removed from activeUsers, currently: 0 2015-11-22 19:28:08,307 DEBUG [ResourceManager Event Processor] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:allocate(594)) - allocate: applicationId=application_1448220478671_0001 container=container_1448220478671_0001_01_000002 host=ip-172-31-20-42.ap-northeast-1.compute.internal:50663 user=ubuntu resource= type=OFF_SWITCH 2015-11-22 19:28:08,307 DEBUG [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(424)) - Processing container_1448220478671_0001_01_000002 of type START 2015-11-22 19:28:08,307 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220478671_0001_01_000002 Container Transitioned from NEW to ALLOCATED 2015-11-22 19:28:08,307 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: CONTAINER_ALLOCATED 2015-11-22 19:28:08,307 DEBUG [ResourceManager Event Processor] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:allocate(208)) - allocate: applicationAttemptId=appattempt_1448220478671_0001_000001 container=container_1448220478671_0001_01_000002 host=ip-172-31-20-42.ap-northeast-1.compute.internal type=OFF_SWITCH 2015-11-22 19:28:08,307 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type CONTAINER_ALLOCATED 2015-11-22 19:28:08,307 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220478671_0001 CONTAINERID=container_1448220478671_0001_01_000002 2015-11-22 19:28:08,308 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(154)) - Assigned container container_1448220478671_0001_01_000002 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:50663, which has 2 containers, used and available after allocation 2015-11-22 19:28:08,308 INFO [ResourceManager Event Processor] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(89)) - assignedContainer application attempt=appattempt_1448220478671_0001_000001 container=container_1448220478671_0001_01_000002 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@33e07bad clusterResource= type=OFF_SWITCH 2015-11-22 19:28:08,308 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:assignContainers(881)) - post-assignContainers for application application_1448220478671_0001 2015-11-22 19:28:08,308 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:08,308 DEBUG [ResourceManager Event Processor] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(470)) - showRequests: application=application_1448220478671_0001 headRoom= currentConsumption=640 2015-11-22 19:28:08,308 DEBUG [ResourceManager Event Processor] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(474)) - showRequests: application=application_1448220478671_0001 request={Priority: 0, Capability: , # Containers: 0, Location: *, Relax Locality: true, Node Label Expression: } 2015-11-22 19:28:08,309 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:08,309 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:08,309 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:08,309 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:allocateResource(1277)) - default user=ubuntu used= numContainers=2 headroom = user-resources= 2015-11-22 19:28:08,309 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(575)) - Assigned to queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.15625, absoluteUsedCapacity=0.15625, numApps=1, numContainers=2 --> , OFF_SWITCH 2015-11-22 19:28:08,309 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(589)) - Re-sorting assigned queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.15625, absoluteUsedCapacity=0.15625, numApps=1, numContainers=2 2015-11-22 19:28:08,309 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:printChildQueues(617)) - printChildQueues - queue: root child-queues: root.defaultusedCapacity=(0.15625), label=(*) 2015-11-22 19:28:08,309 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(461)) - assignedContainer queue=root usedCapacity=0.15625 absoluteUsedCapacity=0.15625 used= cluster= 2015-11-22 19:28:08,310 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(473)) - ParentQ=root assignedSoFarInThisIteration= usedCapacity=0.15625 absoluteUsedCapacity=0.15625 2015-11-22 19:28:08,310 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(484)) - Not assigning more than one off-switch container, assignments so far: resource:; type:OFF_SWITCH; excessReservation:null; applicationid:null; skipped:false; fulfilled reservation:false; allocations(count/resource):1/; reservations(count/resource):0/ 2015-11-22 19:28:08,678 DEBUG [IPC Server idle connection scanner for port 48629] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 48629: task running 2015-11-22 19:28:08,685 DEBUG [IPC Server idle connection scanner for port 46284] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 46284: task running 2015-11-22 19:28:08,692 DEBUG [IPC Server idle connection scanner for port 52028] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 52028: task running 2015-11-22 19:28:08,802 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #829 2015-11-22 19:28:08,802 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #829 2015-11-22 19:28:08,802 DEBUG [IPC Server handler 9 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 9 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#829 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:08,803 DEBUG [IPC Server handler 9 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:08,803 DEBUG [IPC Server handler 9 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:28:08,803 DEBUG [IPC Server handler 9 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 1 procesingTime= 0 2015-11-22 19:28:08,803 DEBUG [IPC Server handler 9 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 9 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#829 Retry#0 2015-11-22 19:28:08,803 DEBUG [IPC Server handler 9 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 9 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#829 Retry#0 Wrote 359 bytes. 2015-11-22 19:28:08,804 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #829 2015-11-22 19:28:08,804 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:28:08,804 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:28:08,948 DEBUG [IPC Server idle connection scanner for port 53669] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 53669: task running 2015-11-22 19:28:09,118 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #2 2015-11-22 19:28:09,118 DEBUG [IPC Server handler 5 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 5 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#2 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:09,118 DEBUG [IPC Server handler 5 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:09,119 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:09,119 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:28:09,119 DEBUG [IPC Server handler 5 on 46284] capacity.CapacityScheduler (CapacityScheduler.java:allocate(941)) - allocate: pre-update appattempt_1448220478671_0001_000001 ask size =1 2015-11-22 19:28:09,119 DEBUG [IPC Server handler 5 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:09,119 DEBUG [IPC Server handler 5 on 46284] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(470)) - showRequests: application=application_1448220478671_0001 headRoom= currentConsumption=640 2015-11-22 19:28:09,119 DEBUG [IPC Server handler 5 on 46284] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(474)) - showRequests: application=application_1448220478671_0001 request={Priority: 0, Capability: , # Containers: 0, Location: *, Relax Locality: true, Node Label Expression: } 2015-11-22 19:28:09,120 DEBUG [IPC Server handler 5 on 46284] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:updateResourceRequests(320)) - update: application=application_1448220478671_0001 request={Priority: 0, Capability: , # Containers: 3, Location: *, Relax Locality: true, Node Label Expression: } 2015-11-22 19:28:09,120 DEBUG [IPC Server handler 5 on 46284] scheduler.ActiveUsersManager (ActiveUsersManager.java:activateApplication(68)) - User ubuntu added to activeUsers, currently: 1 2015-11-22 19:28:09,120 DEBUG [IPC Server handler 5 on 46284] capacity.CapacityScheduler (CapacityScheduler.java:allocate(952)) - allocate: post-update 2015-11-22 19:28:09,120 DEBUG [IPC Server handler 5 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2015-11-22 19:28:09,120 DEBUG [IPC Server handler 5 on 46284] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(470)) - showRequests: application=application_1448220478671_0001 headRoom= currentConsumption=640 2015-11-22 19:28:09,120 DEBUG [IPC Server handler 5 on 46284] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(474)) - showRequests: application=application_1448220478671_0001 request={Priority: 0, Capability: , # Containers: 3, Location: *, Relax Locality: true, Node Label Expression: } 2015-11-22 19:28:09,120 DEBUG [IPC Server handler 5 on 46284] security.BaseContainerTokenSecretManager (BaseContainerTokenSecretManager.java:createPassword(90)) - Creating password for container_1448220478671_0001_01_000002 for user container_1448220478671_0001_01_000002 (auth:SIMPLE) to be run on NM ip-172-31-20-42.ap-northeast-1.compute.internal:50663 2015-11-22 19:28:09,121 DEBUG [IPC Server handler 5 on 46284] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(197)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 1 cluster_timestamp: 1448220478671 } attemptId: 1 } id: 2 } nmHostAddr: "ip-172-31-20-42.ap-northeast-1.compute.internal:50663" appSubmitter: "ubuntu" resource { memory: 128 virtual_cores: 1 } expiryTimeStamp: 1448221089120 masterKeyId: -550760804 rmIdentifier: 1448220478671 priority { priority: 0 } creationTime: 1448220488307 nodeLabelExpression: "" containerType: TASK 2015-11-22 19:28:09,121 DEBUG [IPC Server handler 5 on 46284] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(197)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 1 cluster_timestamp: 1448220478671 } attemptId: 1 } id: 2 } nmHostAddr: "ip-172-31-20-42.ap-northeast-1.compute.internal:50663" appSubmitter: "ubuntu" resource { memory: 128 virtual_cores: 1 } expiryTimeStamp: 1448221089120 masterKeyId: -550760804 rmIdentifier: 1448220478671 priority { priority: 0 } creationTime: 1448220488307 nodeLabelExpression: "" containerType: TASK 2015-11-22 19:28:09,121 INFO [IPC Server handler 5 on 46284] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:createAndGetNMToken(200)) - Sending NMToken for nodeId : ip-172-31-20-42.ap-northeast-1.compute.internal:50663 for container : container_1448220478671_0001_01_000002 2015-11-22 19:28:09,121 DEBUG [IPC Server handler 5 on 46284] security.BaseNMTokenSecretManager (BaseNMTokenSecretManager.java:createPassword(75)) - creating password for appattempt_1448220478671_0001_000001 for user ubuntu to run on NM ip-172-31-20-42.ap-northeast-1.compute.internal:50663 2015-11-22 19:28:09,121 DEBUG [IPC Server handler 5 on 46284] security.NMTokenIdentifier (NMTokenIdentifier.java:write(96)) - Writing NMTokenIdentifier to RPC layer: appAttemptId { application_id { id: 1 cluster_timestamp: 1448220478671 } attemptId: 1 } nodeId { host: "ip-172-31-20-42.ap-northeast-1.compute.internal" port: 50663 } appSubmitter: "ubuntu" keyId: 744666633 2015-11-22 19:28:09,122 DEBUG [IPC Server handler 5 on 46284] security.NMTokenIdentifier (NMTokenIdentifier.java:write(96)) - Writing NMTokenIdentifier to RPC layer: appAttemptId { application_id { id: 1 cluster_timestamp: 1448220478671 } attemptId: 1 } nodeId { host: "ip-172-31-20-42.ap-northeast-1.compute.internal" port: 50663 } appSubmitter: "ubuntu" keyId: 744666633 2015-11-22 19:28:09,122 DEBUG [IPC Server handler 5 on 46284] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(424)) - Processing container_1448220478671_0001_01_000002 of type ACQUIRED 2015-11-22 19:28:09,122 INFO [IPC Server handler 5 on 46284] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220478671_0001_01_000002 Container Transitioned from ALLOCATED to ACQUIRED 2015-11-22 19:28:09,122 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppRunningOnNodeEvent.EventType: APP_RUNNING_ON_NODE 2015-11-22 19:28:09,122 DEBUG [IPC Server handler 5 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2015-11-22 19:28:09,123 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(777)) - Processing event for application_1448220478671_0001 of type APP_RUNNING_ON_NODE 2015-11-22 19:28:09,123 DEBUG [IPC Server handler 5 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 5 2015-11-22 19:28:09,123 DEBUG [IPC Server handler 5 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 5 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#2 Retry#0 2015-11-22 19:28:09,124 DEBUG [IPC Server handler 5 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 5 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#2 Retry#0 Wrote 582 bytes. 2015-11-22 19:28:09,188 DEBUG [IPC Server listener on 50663] ipc.Server (Server.java:register(2883)) - Server connection from 172.31.20.42:35350; # active connections: 2; # queued calls: 0 2015-11-22 19:28:09,189 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processOneRpc(1897)) - got #-33 2015-11-22 19:28:09,189 DEBUG [Socket Reader #1 for port 50663] security.SaslRpcServer (SaslRpcServer.java:create(175)) - Created SASL server with mechanism = DIGEST-MD5 2015-11-22 19:28:09,189 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processResponse(1099)) - Socket Reader #1 for port 50663: responding to null from 172.31.20.42:35350 Call#-33 Retry#-1 2015-11-22 19:28:09,189 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processResponse(1118)) - Socket Reader #1 for port 50663: responding to null from 172.31.20.42:35350 Call#-33 Retry#-1 Wrote 166 bytes. 2015-11-22 19:28:09,195 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processOneRpc(1897)) - got #-33 2015-11-22 19:28:09,195 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processSaslToken(1529)) - Have read input token of size 350 for processing by saslServer.evaluateResponse() 2015-11-22 19:28:09,195 DEBUG [Socket Reader #1 for port 50663] security.BaseNMTokenSecretManager (BaseNMTokenSecretManager.java:retrivePasswordInternal(103)) - creating password for appattempt_1448220478671_0001_000001 for user ubuntu to run on NM ip-172-31-20-42.ap-northeast-1.compute.internal:50663 2015-11-22 19:28:09,195 DEBUG [Socket Reader #1 for port 50663] security.NMTokenIdentifier (NMTokenIdentifier.java:write(96)) - Writing NMTokenIdentifier to RPC layer: appAttemptId { application_id { id: 1 cluster_timestamp: 1448220478671 } attemptId: 1 } nodeId { host: "ip-172-31-20-42.ap-northeast-1.compute.internal" port: 50663 } appSubmitter: "ubuntu" keyId: 744666633 2015-11-22 19:28:09,196 DEBUG [Socket Reader #1 for port 50663] security.NMTokenSecretManagerInNM (NMTokenSecretManagerInNM.java:retrievePassword(180)) - NMToken password retrieved successfully!! 2015-11-22 19:28:09,196 DEBUG [Socket Reader #1 for port 50663] security.SaslRpcServer (SaslRpcServer.java:handle(310)) - SASL server DIGEST-MD5 callback: setting password for client: appattempt_1448220478671_0001_000001 (auth:SIMPLE) 2015-11-22 19:28:09,196 DEBUG [Socket Reader #1 for port 50663] security.SaslRpcServer (SaslRpcServer.java:handle(328)) - SASL server DIGEST-MD5 callback: setting canonicalized client ID: appattempt_1448220478671_0001_000001 2015-11-22 19:28:09,196 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:buildSaslResponse(1546)) - Will send SUCCESS token of size 40 from saslServer. 2015-11-22 19:28:09,196 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:saslProcess(1434)) - SASL server context established. Negotiated QoP is auth 2015-11-22 19:28:09,196 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:saslProcess(1439)) - SASL server successfully authenticated client: appattempt_1448220478671_0001_000001 (auth:SIMPLE) 2015-11-22 19:28:09,197 INFO [Socket Reader #1 for port 50663] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220478671_0001_000001 (auth:SIMPLE) 2015-11-22 19:28:09,197 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processResponse(1099)) - Socket Reader #1 for port 50663: responding to null from 172.31.20.42:35350 Call#-33 Retry#-1 2015-11-22 19:28:09,197 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processResponse(1118)) - Socket Reader #1 for port 50663: responding to null from 172.31.20.42:35350 Call#-33 Retry#-1 Wrote 64 bytes. 2015-11-22 19:28:09,203 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processOneRpc(1897)) - got #-3 2015-11-22 19:28:09,203 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:authorizeConnection(2097)) - Successfully authorized userInfo { } protocol: "org.apache.hadoop.yarn.api.ContainerManagementProtocolPB" 2015-11-22 19:28:09,204 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processOneRpc(1897)) - got #3 2015-11-22 19:28:09,204 DEBUG [IPC Server handler 2 on 50663] ipc.Server (Server.java:run(2185)) - IPC Server handler 2 on 50663: org.apache.hadoop.yarn.api.ContainerManagementProtocolPB.startContainers from 172.31.20.42:35350 Call#3 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:09,205 DEBUG [IPC Server handler 2 on 50663] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:09,205 DEBUG [IPC Server handler 2 on 50663] security.BaseContainerTokenSecretManager (BaseContainerTokenSecretManager.java:retrievePasswordInternal(118)) - Retrieving password for container_1448220478671_0001_01_000002 for user container_1448220478671_0001_01_000002 (auth:SIMPLE) to be run on NM ip-172-31-20-42.ap-northeast-1.compute.internal:50663 2015-11-22 19:28:09,205 DEBUG [IPC Server handler 2 on 50663] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(197)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 1 cluster_timestamp: 1448220478671 } attemptId: 1 } id: 2 } nmHostAddr: "ip-172-31-20-42.ap-northeast-1.compute.internal:50663" appSubmitter: "ubuntu" resource { memory: 128 virtual_cores: 1 } expiryTimeStamp: 1448221089120 masterKeyId: -550760804 rmIdentifier: 1448220478671 priority { priority: 0 } creationTime: 1448220488307 nodeLabelExpression: "" containerType: TASK 2015-11-22 19:28:09,206 INFO [IPC Server handler 2 on 50663] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:startContainerInternal(889)) - Start request for container_1448220478671_0001_01_000002 by user ubuntu 2015-11-22 19:28:09,206 DEBUG [IPC Server handler 2 on 50663] utils.YarnServerSecurityUtils (YarnServerSecurityUtils.java:parseCredentials(135)) - = Kind: YARN_AM_RM_TOKEN, Service: , Ident: (appAttemptId { application_id { id: 1 cluster_timestamp: 1448220478671 } attemptId: 1 } keyId: -1778540031) 2015-11-22 19:28:09,206 INFO [IPC Server handler 2 on 50663] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu IP=172.31.20.42 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1448220478671_0001 CONTAINERID=container_1448220478671_0001_01_000002 2015-11-22 19:28:09,206 DEBUG [IPC Server handler 2 on 50663] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: startContainers queueTime= 1 procesingTime= 1 2015-11-22 19:28:09,206 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationContainerInitEvent.EventType: INIT_CONTAINER 2015-11-22 19:28:09,207 DEBUG [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(452)) - Processing application_1448220478671_0001 of type INIT_CONTAINER 2015-11-22 19:28:09,207 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(303)) - Adding container_1448220478671_0001_01_000002 to application application_1448220478671_0001 2015-11-22 19:28:09,207 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerInitEvent.EventType: INIT_CONTAINER 2015-11-22 19:28:09,208 DEBUG [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1131)) - Processing container_1448220478671_0001_01_000002 of type INIT_CONTAINER 2015-11-22 19:28:09,208 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220478671_0001_01_000002 transitioned from NEW to LOCALIZED 2015-11-22 19:28:09,208 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServicesEvent.EventType: CONTAINER_INIT 2015-11-22 19:28:09,208 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_INIT for appId application_1448220478671_0001 2015-11-22 19:28:09,208 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncherEvent.EventType: LAUNCH_CONTAINER 2015-11-22 19:28:09,207 DEBUG [IPC Server handler 2 on 50663] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 2 on 50663: responding to org.apache.hadoop.yarn.api.ContainerManagementProtocolPB.startContainers from 172.31.20.42:35350 Call#3 Retry#0 2015-11-22 19:28:09,208 DEBUG [IPC Server handler 2 on 50663] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 2 on 50663: responding to org.apache.hadoop.yarn.api.ContainerManagementProtocolPB.startContainers from 172.31.20.42:35350 Call#3 Retry#0 Wrote 51 bytes. 2015-11-22 19:28:09,215 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:close(2894)) - Socket Reader #1 for port 50663: disconnecting client 172.31.20.42:35350. Number of active connections: 1 2015-11-22 19:28:09,217 DEBUG [ContainersLauncher #1] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.fs.FileContext.getAbstractFileSystem(FileContext.java:339) 2015-11-22 19:28:09,218 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - REQUEST /ws/v1/timeline/ on org.mortbay.jetty.HttpConnection@6aba846 2015-11-22 19:28:09,218 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - sessionManager=org.mortbay.jetty.servlet.HashSessionManager@3966697d 2015-11-22 19:28:09,219 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - session=null 2015-11-22 19:28:09,219 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - servlet=org.mortbay.jetty.servlet.DefaultServlet-55176785 2015-11-22 19:28:09,219 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - chain=NoCacheFilter->NoCacheFilter->safety->Timeline Authentication Filter->RMAuthenticationFilter->static_user_filter->guice->org.mortbay.jetty.servlet.DefaultServlet-55176785 2015-11-22 19:28:09,219 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - servlet holder=org.mortbay.jetty.servlet.DefaultServlet-55176785 2015-11-22 19:28:09,219 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter NoCacheFilter 2015-11-22 19:28:09,219 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter NoCacheFilter 2015-11-22 19:28:09,219 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter safety 2015-11-22 19:28:09,219 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter Timeline Authentication Filter 2015-11-22 19:28:09,220 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(557)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:41381/ws/v1/timeline/?user.name=ubuntu] triggering authentication 2015-11-22 19:28:09,220 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(576)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:41381/ws/v1/timeline/?user.name=ubuntu] user [ubuntu] authenticated 2015-11-22 19:28:09,220 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter RMAuthenticationFilter 2015-11-22 19:28:09,220 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(557)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:41381/ws/v1/timeline/?user.name=ubuntu] triggering authentication 2015-11-22 19:28:09,220 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(576)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:41381/ws/v1/timeline/?user.name=ubuntu] user [ubuntu] authenticated 2015-11-22 19:28:09,221 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter static_user_filter 2015-11-22 19:28:09,221 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter guice 2015-11-22 19:28:09,225 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - getResource(META-INF/services/javax.xml.bind.JAXBContext)=jar:file:/home/ubuntu/.m2/repository/com/sun/xml/bind/jaxb-impl/2.2.3-1/jaxb-impl-2.2.3-1.jar!/META-INF/services/javax.xml.bind.JAXBContext 2015-11-22 19:28:09,225 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.xml.bind.v2.ContextFactory from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:28:09,229 DEBUG [IPC Server listener on 50663] ipc.Server (Server.java:register(2883)) - Server connection from 172.31.20.42:35351; # active connections: 2; # queued calls: 0 2015-11-22 19:28:09,230 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processOneRpc(1897)) - got #-33 2015-11-22 19:28:09,230 DEBUG [Socket Reader #1 for port 50663] security.SaslRpcServer (SaslRpcServer.java:create(175)) - Created SASL server with mechanism = DIGEST-MD5 2015-11-22 19:28:09,230 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processResponse(1099)) - Socket Reader #1 for port 50663: responding to null from 172.31.20.42:35351 Call#-33 Retry#-1 2015-11-22 19:28:09,230 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processResponse(1118)) - Socket Reader #1 for port 50663: responding to null from 172.31.20.42:35351 Call#-33 Retry#-1 Wrote 166 bytes. 2015-11-22 19:28:09,234 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processOneRpc(1897)) - got #-33 2015-11-22 19:28:09,234 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processSaslToken(1529)) - Have read input token of size 350 for processing by saslServer.evaluateResponse() 2015-11-22 19:28:09,235 DEBUG [Socket Reader #1 for port 50663] security.BaseNMTokenSecretManager (BaseNMTokenSecretManager.java:retrivePasswordInternal(103)) - creating password for appattempt_1448220478671_0001_000001 for user ubuntu to run on NM ip-172-31-20-42.ap-northeast-1.compute.internal:50663 2015-11-22 19:28:09,235 DEBUG [Socket Reader #1 for port 50663] security.NMTokenIdentifier (NMTokenIdentifier.java:write(96)) - Writing NMTokenIdentifier to RPC layer: appAttemptId { application_id { id: 1 cluster_timestamp: 1448220478671 } attemptId: 1 } nodeId { host: "ip-172-31-20-42.ap-northeast-1.compute.internal" port: 50663 } appSubmitter: "ubuntu" keyId: 744666633 2015-11-22 19:28:09,235 DEBUG [Socket Reader #1 for port 50663] security.NMTokenSecretManagerInNM (NMTokenSecretManagerInNM.java:retrievePassword(180)) - NMToken password retrieved successfully!! 2015-11-22 19:28:09,235 DEBUG [Socket Reader #1 for port 50663] security.SaslRpcServer (SaslRpcServer.java:handle(310)) - SASL server DIGEST-MD5 callback: setting password for client: appattempt_1448220478671_0001_000001 (auth:SIMPLE) 2015-11-22 19:28:09,235 DEBUG [Socket Reader #1 for port 50663] security.SaslRpcServer (SaslRpcServer.java:handle(328)) - SASL server DIGEST-MD5 callback: setting canonicalized client ID: appattempt_1448220478671_0001_000001 2015-11-22 19:28:09,236 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:buildSaslResponse(1546)) - Will send SUCCESS token of size 40 from saslServer. 2015-11-22 19:28:09,236 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:saslProcess(1434)) - SASL server context established. Negotiated QoP is auth 2015-11-22 19:28:09,236 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:saslProcess(1439)) - SASL server successfully authenticated client: appattempt_1448220478671_0001_000001 (auth:SIMPLE) 2015-11-22 19:28:09,236 INFO [Socket Reader #1 for port 50663] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220478671_0001_000001 (auth:SIMPLE) 2015-11-22 19:28:09,236 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processResponse(1099)) - Socket Reader #1 for port 50663: responding to null from 172.31.20.42:35351 Call#-33 Retry#-1 2015-11-22 19:28:09,236 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processResponse(1118)) - Socket Reader #1 for port 50663: responding to null from 172.31.20.42:35351 Call#-33 Retry#-1 Wrote 64 bytes. 2015-11-22 19:28:09,237 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processOneRpc(1897)) - got #-3 2015-11-22 19:28:09,238 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:authorizeConnection(2097)) - Successfully authorized userInfo { } protocol: "org.apache.hadoop.yarn.api.ContainerManagementProtocolPB" 2015-11-22 19:28:09,238 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processOneRpc(1897)) - got #4 2015-11-22 19:28:09,238 DEBUG [IPC Server handler 3 on 50663] ipc.Server (Server.java:run(2185)) - IPC Server handler 3 on 50663: org.apache.hadoop.yarn.api.ContainerManagementProtocolPB.getContainerStatuses from 172.31.20.42:35351 Call#4 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:09,239 DEBUG [IPC Server handler 3 on 50663] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:09,239 INFO [IPC Server handler 3 on 50663] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:getContainerStatusInternal(1221)) - Getting container-status for container_1448220478671_0001_01_000002 2015-11-22 19:28:09,239 INFO [IPC Server handler 3 on 50663] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:getContainerStatusInternal(1235)) - Returning ContainerStatus: [ContainerId: container_1448220478671_0001_01_000002, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ] 2015-11-22 19:28:09,239 DEBUG [IPC Server handler 3 on 50663] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getContainerStatuses queueTime= 1 procesingTime= 0 2015-11-22 19:28:09,240 DEBUG [IPC Server handler 3 on 50663] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 3 on 50663: responding to org.apache.hadoop.yarn.api.ContainerManagementProtocolPB.getContainerStatuses from 172.31.20.42:35351 Call#4 Retry#0 2015-11-22 19:28:09,240 DEBUG [IPC Server handler 3 on 50663] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 3 on 50663: responding to org.apache.hadoop.yarn.api.ContainerManagementProtocolPB.getContainerStatuses from 172.31.20.42:35351 Call#4 Retry#0 Wrote 75 bytes. 2015-11-22 19:28:09,242 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:close(2894)) - Socket Reader #1 for port 50663: disconnecting client 172.31.20.42:35351. Number of active connections: 1 2015-11-22 19:28:09,265 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - RESPONSE /ws/v1/timeline/ 200 2015-11-22 19:28:09,268 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - REQUEST /ws/v1/timeline/ on org.mortbay.jetty.HttpConnection@1211b454 2015-11-22 19:28:09,269 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - sessionManager=org.mortbay.jetty.servlet.HashSessionManager@3966697d 2015-11-22 19:28:09,269 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - session=null 2015-11-22 19:28:09,269 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerEvent.EventType: CONTAINER_LAUNCHED 2015-11-22 19:28:09,269 DEBUG [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1131)) - Processing container_1448220478671_0001_01_000002 of type CONTAINER_LAUNCHED 2015-11-22 19:28:09,269 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220478671_0001_01_000002 transitioned from LOCALIZED to RUNNING 2015-11-22 19:28:09,270 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerStartMonitoringEvent.EventType: START_MONITORING_CONTAINER 2015-11-22 19:28:09,270 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - servlet=org.mortbay.jetty.servlet.DefaultServlet-55176785 2015-11-22 19:28:09,270 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - chain=NoCacheFilter->NoCacheFilter->safety->Timeline Authentication Filter->RMAuthenticationFilter->static_user_filter->guice->org.mortbay.jetty.servlet.DefaultServlet-55176785 2015-11-22 19:28:09,270 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - servlet holder=org.mortbay.jetty.servlet.DefaultServlet-55176785 2015-11-22 19:28:09,270 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter NoCacheFilter 2015-11-22 19:28:09,270 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter NoCacheFilter 2015-11-22 19:28:09,270 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter safety 2015-11-22 19:28:09,271 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter Timeline Authentication Filter 2015-11-22 19:28:09,271 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(576)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:41381/ws/v1/timeline/] user [ubuntu] authenticated 2015-11-22 19:28:09,271 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter RMAuthenticationFilter 2015-11-22 19:28:09,271 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(576)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:41381/ws/v1/timeline/] user [ubuntu] authenticated 2015-11-22 19:28:09,271 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter static_user_filter 2015-11-22 19:28:09,272 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter guice 2015-11-22 19:28:09,272 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] security.TimelineACLsManager (TimelineACLsManager.java:checkAccess(106)) - Verifying the access of ubuntu on the timeline entity { id: container_1448220478671_0001_01_000002, type: DS_CONTAINER } 2015-11-22 19:28:09,279 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - RESPONSE /ws/v1/timeline/ 200 2015-11-22 19:28:09,279 DEBUG [2048592079@qtp-847268847-0] mortbay.log (Slf4jLog.java:debug(40)) - EOF 2015-11-22 19:28:09,303 INFO [ContainersLauncher #1] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:buildCommandExecutor(281)) - launchContainer: [nice, -n, 0, bash, /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220478671_0001/container_1448220478671_0001_01_000002/default_container_executor.sh] 2015-11-22 19:28:09,304 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:28:09,304 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 2 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ], ContainerStatus: [ContainerId: container_1448220478671_0001_01_000002, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:28:09,304 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:09,304 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:28:09,304 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:28:09,305 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:28:09,305 DEBUG [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(424)) - Processing container_1448220478671_0001_01_000002 of type LAUNCHED 2015-11-22 19:28:09,305 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220478671_0001_01_000002 Container Transitioned from ACQUIRED to RUNNING 2015-11-22 19:28:09,305 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:28:09,305 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:28:09,305 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(408)) - Trying to assign containers to child-queue of root 2015-11-22 19:28:09,305 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:printChildQueues(617)) - printChildQueues - queue: root child-queues: root.defaultusedCapacity=(0.15625), label=(*) 2015-11-22 19:28:09,305 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(564)) - Trying to assign to queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.15625, absoluteUsedCapacity=0.15625, numApps=1, numContainers=2 2015-11-22 19:28:09,305 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:assignContainers(817)) - assignContainers: node=ip-172-31-20-42.ap-northeast-1.compute.internal #applications=1 2015-11-22 19:28:09,306 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2015-11-22 19:28:09,306 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimitAndSetHeadroom(984)) - Headroom calculation for user ubuntu: userLimit= queueMaxAvailRes= consumed= headroom= 2015-11-22 19:28:09,306 DEBUG [ResourceManager Event Processor] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:assignContainers(477)) - pre-assignContainers for application application_1448220478671_0001 2015-11-22 19:28:09,306 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2015-11-22 19:28:09,306 DEBUG [ResourceManager Event Processor] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(470)) - showRequests: application=application_1448220478671_0001 headRoom= currentConsumption=640 2015-11-22 19:28:09,307 DEBUG [ResourceManager Event Processor] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(474)) - showRequests: application=application_1448220478671_0001 request={Priority: 0, Capability: , # Containers: 3, Location: *, Relax Locality: true, Node Label Expression: } 2015-11-22 19:28:09,307 DEBUG [ResourceManager Event Processor] allocator.IncreaseContainerAllocator (IncreaseContainerAllocator.java:assignContainers(184)) - Skip allocating increase request since we don't have any increase request on this node=ip-172-31-20-42.ap-northeast-1.compute.internal:50663 2015-11-22 19:28:09,307 DEBUG [ResourceManager Event Processor] allocator.RegularContainerAllocator (RegularContainerAllocator.java:assignContainer(407)) - assignContainers: node=ip-172-31-20-42.ap-northeast-1.compute.internal application=application_1448220478671_0001 priority=0 request={Priority: 0, Capability: , # Containers: 3, Location: *, Relax Locality: true, Node Label Expression: } type=OFF_SWITCH 2015-11-22 19:28:09,307 DEBUG [ResourceManager Event Processor] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:allocate(594)) - allocate: applicationId=application_1448220478671_0001 container=container_1448220478671_0001_01_000003 host=ip-172-31-20-42.ap-northeast-1.compute.internal:50663 user=ubuntu resource= type=OFF_SWITCH 2015-11-22 19:28:09,307 DEBUG [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(424)) - Processing container_1448220478671_0001_01_000003 of type START 2015-11-22 19:28:09,307 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220478671_0001_01_000003 Container Transitioned from NEW to ALLOCATED 2015-11-22 19:28:09,307 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: CONTAINER_ALLOCATED 2015-11-22 19:28:09,308 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type CONTAINER_ALLOCATED 2015-11-22 19:28:09,308 DEBUG [ResourceManager Event Processor] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:allocate(208)) - allocate: applicationAttemptId=appattempt_1448220478671_0001_000001 container=container_1448220478671_0001_01_000003 host=ip-172-31-20-42.ap-northeast-1.compute.internal type=OFF_SWITCH 2015-11-22 19:28:09,308 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220478671_0001 CONTAINERID=container_1448220478671_0001_01_000003 2015-11-22 19:28:09,308 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(154)) - Assigned container container_1448220478671_0001_01_000003 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:50663, which has 3 containers, used and available after allocation 2015-11-22 19:28:09,308 INFO [ResourceManager Event Processor] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(89)) - assignedContainer application attempt=appattempt_1448220478671_0001_000001 container=container_1448220478671_0001_01_000003 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@33e07bad clusterResource= type=OFF_SWITCH 2015-11-22 19:28:09,308 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:assignContainers(881)) - post-assignContainers for application application_1448220478671_0001 2015-11-22 19:28:09,308 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2015-11-22 19:28:09,308 DEBUG [ResourceManager Event Processor] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(470)) - showRequests: application=application_1448220478671_0001 headRoom= currentConsumption=768 2015-11-22 19:28:09,309 DEBUG [ResourceManager Event Processor] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(474)) - showRequests: application=application_1448220478671_0001 request={Priority: 0, Capability: , # Containers: 2, Location: *, Relax Locality: true, Node Label Expression: } 2015-11-22 19:28:09,309 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2015-11-22 19:28:09,309 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2015-11-22 19:28:09,309 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2015-11-22 19:28:09,309 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:allocateResource(1277)) - default user=ubuntu used= numContainers=3 headroom = user-resources= 2015-11-22 19:28:09,311 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(575)) - Assigned to queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.1875, absoluteUsedCapacity=0.1875, numApps=1, numContainers=3 --> , OFF_SWITCH 2015-11-22 19:28:09,311 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(589)) - Re-sorting assigned queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.1875, absoluteUsedCapacity=0.1875, numApps=1, numContainers=3 2015-11-22 19:28:09,311 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:printChildQueues(617)) - printChildQueues - queue: root child-queues: root.defaultusedCapacity=(0.1875), label=(*) 2015-11-22 19:28:09,311 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(461)) - assignedContainer queue=root usedCapacity=0.1875 absoluteUsedCapacity=0.1875 used= cluster= 2015-11-22 19:28:09,311 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(473)) - ParentQ=root assignedSoFarInThisIteration= usedCapacity=0.1875 absoluteUsedCapacity=0.1875 2015-11-22 19:28:09,311 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(484)) - Not assigning more than one off-switch container, assignments so far: resource:; type:OFF_SWITCH; excessReservation:null; applicationid:null; skipped:false; fulfilled reservation:false; allocations(count/resource):1/; reservations(count/resource):0/ 2015-11-22 19:28:09,317 DEBUG [ContainersLauncher #1] launcher.ContainerLaunch (ContainerLaunch.java:call(333)) - Container container_1448220478671_0001_01_000002 completed with exit code 0 2015-11-22 19:28:09,317 INFO [ContainersLauncher #1] launcher.ContainerLaunch (ContainerLaunch.java:call(356)) - Container container_1448220478671_0001_01_000002 succeeded 2015-11-22 19:28:09,317 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerEvent.EventType: CONTAINER_EXITED_WITH_SUCCESS 2015-11-22 19:28:09,317 DEBUG [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1131)) - Processing container_1448220478671_0001_01_000002 of type CONTAINER_EXITED_WITH_SUCCESS 2015-11-22 19:28:09,317 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220478671_0001_01_000002 transitioned from RUNNING to EXITED_WITH_SUCCESS 2015-11-22 19:28:09,317 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncherEvent.EventType: CLEANUP_CONTAINER 2015-11-22 19:28:09,318 INFO [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(380)) - Cleaning up container container_1448220478671_0001_01_000002 2015-11-22 19:28:09,318 DEBUG [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(397)) - Marking container container_1448220478671_0001_01_000002 as inactive 2015-11-22 19:28:09,318 DEBUG [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(403)) - Getting pid for container container_1448220478671_0001_01_000002 to kill from pid file /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/application_1448220478671_0001/container_1448220478671_0001_01_000002/container_1448220478671_0001_01_000002.pid 2015-11-22 19:28:09,318 DEBUG [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:getContainerPid(569)) - Accessing pid for container container_1448220478671_0001_01_000002 from pid file /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/application_1448220478671_0001/container_1448220478671_0001_01_000002/container_1448220478671_0001_01_000002.pid 2015-11-22 19:28:09,318 DEBUG [AsyncDispatcher event handler] util.ProcessIdFileReader (ProcessIdFileReader.java:getProcessId(51)) - Accessing pid from pid file /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/application_1448220478671_0001/container_1448220478671_0001_01_000002/container_1448220478671_0001_01_000002.pid 2015-11-22 19:28:09,318 DEBUG [AsyncDispatcher event handler] util.ProcessIdFileReader (ProcessIdFileReader.java:getProcessId(99)) - Got pid 30075 from path /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/application_1448220478671_0001/container_1448220478671_0001_01_000002/container_1448220478671_0001_01_000002.pid 2015-11-22 19:28:09,319 DEBUG [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:getContainerPid(579)) - Got pid 30075 for container container_1448220478671_0001_01_000002 2015-11-22 19:28:09,319 DEBUG [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(421)) - Sending signal to pid 30075 as user ubuntu for container container_1448220478671_0001_01_000002 2015-11-22 19:28:09,319 DEBUG [AsyncDispatcher event handler] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:signalContainer(417)) - Sending signal 15 to pid 30075 as user ubuntu 2015-11-22 19:28:09,321 DEBUG [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(437)) - Sent signal SIGTERM to pid 30075 as user ubuntu for container container_1448220478671_0001_01_000002, result=failed 2015-11-22 19:28:09,321 DEBUG [AsyncDispatcher event handler] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.fs.FileContext.getAbstractFileSystem(FileContext.java:339) 2015-11-22 19:28:09,332 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.ContainerLocalizationCleanupEvent.EventType: CLEANUP_CONTAINER_RESOURCES 2015-11-22 19:28:09,332 DEBUG [DeletionService #0] nodemanager.DeletionService (DeletionService.java:run(266)) - FileDeletionTask : user : ubuntu subDir : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220478671_0001/container_1448220478671_0001_01_000002 baseDir : null 2015-11-22 19:28:09,333 DEBUG [DeletionService #0] nodemanager.DeletionService (DeletionService.java:run(292)) - Deleting path: [/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220478671_0001/container_1448220478671_0001_01_000002] as user: [ubuntu] 2015-11-22 19:28:09,333 INFO [DeletionService #0] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(480)) - Deleting absolute path : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220478671_0001/container_1448220478671_0001_01_000002 2015-11-22 19:28:09,333 DEBUG [DeletionService #1] nodemanager.DeletionService (DeletionService.java:run(266)) - FileDeletionTask : user : null subDir : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/application_1448220478671_0001/container_1448220478671_0001_01_000002 baseDir : null 2015-11-22 19:28:09,333 DEBUG [DeletionService #1] nodemanager.DeletionService (DeletionService.java:run(271)) - NM deleting absolute path : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/application_1448220478671_0001/container_1448220478671_0001_01_000002 2015-11-22 19:28:09,333 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerEvent.EventType: CONTAINER_RESOURCES_CLEANEDUP 2015-11-22 19:28:09,333 DEBUG [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1131)) - Processing container_1448220478671_0001_01_000002 of type CONTAINER_RESOURCES_CLEANEDUP 2015-11-22 19:28:09,333 INFO [AsyncDispatcher event handler] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu OPERATION=Container Finished - Succeeded TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1448220478671_0001 CONTAINERID=container_1448220478671_0001_01_000002 2015-11-22 19:28:09,334 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220478671_0001_01_000002 transitioned from EXITED_WITH_SUCCESS to DONE 2015-11-22 19:28:09,334 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationContainerFinishedEvent.EventType: APPLICATION_CONTAINER_FINISHED 2015-11-22 19:28:09,334 DEBUG [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(452)) - Processing application_1448220478671_0001 of type APPLICATION_CONTAINER_FINISHED 2015-11-22 19:28:09,334 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(346)) - Removing container_1448220478671_0001_01_000002 from application application_1448220478671_0001 2015-11-22 19:28:09,334 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerStopMonitoringEvent.EventType: STOP_MONITORING_CONTAINER 2015-11-22 19:28:09,334 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.event.LogHandlerContainerFinishedEvent.EventType: CONTAINER_FINISHED 2015-11-22 19:28:09,334 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServicesEvent.EventType: CONTAINER_STOP 2015-11-22 19:28:09,334 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_STOP for appId application_1448220478671_0001 2015-11-22 19:28:09,571 DEBUG [Task killer for 30075] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:signalContainer(417)) - Sending signal 9 to pid 30075 as user ubuntu 2015-11-22 19:28:09,804 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #830 2015-11-22 19:28:09,805 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #830 2015-11-22 19:28:09,805 DEBUG [IPC Server handler 12 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 12 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#830 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:09,805 DEBUG [IPC Server handler 12 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:09,805 DEBUG [IPC Server handler 12 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:28:09,805 DEBUG [IPC Server handler 12 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 0 2015-11-22 19:28:09,806 DEBUG [IPC Server handler 12 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 12 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#830 Retry#0 2015-11-22 19:28:09,806 DEBUG [IPC Server handler 12 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 12 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#830 Retry#0 Wrote 359 bytes. 2015-11-22 19:28:09,806 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #830 2015-11-22 19:28:09,806 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:28:09,806 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:28:10,129 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #5 2015-11-22 19:28:10,130 DEBUG [IPC Server handler 6 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 6 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#5 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:10,130 DEBUG [IPC Server handler 6 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:10,130 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:10,130 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:28:10,131 DEBUG [IPC Server handler 6 on 46284] security.BaseContainerTokenSecretManager (BaseContainerTokenSecretManager.java:createPassword(90)) - Creating password for container_1448220478671_0001_01_000003 for user container_1448220478671_0001_01_000003 (auth:SIMPLE) to be run on NM ip-172-31-20-42.ap-northeast-1.compute.internal:50663 2015-11-22 19:28:10,131 DEBUG [IPC Server handler 6 on 46284] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(197)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 1 cluster_timestamp: 1448220478671 } attemptId: 1 } id: 3 } nmHostAddr: "ip-172-31-20-42.ap-northeast-1.compute.internal:50663" appSubmitter: "ubuntu" resource { memory: 128 virtual_cores: 1 } expiryTimeStamp: 1448221090130 masterKeyId: -550760804 rmIdentifier: 1448220478671 priority { priority: 0 } creationTime: 1448220489307 nodeLabelExpression: "" containerType: TASK 2015-11-22 19:28:10,131 DEBUG [IPC Server handler 6 on 46284] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(197)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 1 cluster_timestamp: 1448220478671 } attemptId: 1 } id: 3 } nmHostAddr: "ip-172-31-20-42.ap-northeast-1.compute.internal:50663" appSubmitter: "ubuntu" resource { memory: 128 virtual_cores: 1 } expiryTimeStamp: 1448221090130 masterKeyId: -550760804 rmIdentifier: 1448220478671 priority { priority: 0 } creationTime: 1448220489307 nodeLabelExpression: "" containerType: TASK 2015-11-22 19:28:10,131 DEBUG [IPC Server handler 6 on 46284] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(424)) - Processing container_1448220478671_0001_01_000003 of type ACQUIRED 2015-11-22 19:28:10,131 INFO [IPC Server handler 6 on 46284] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220478671_0001_01_000003 Container Transitioned from ALLOCATED to ACQUIRED 2015-11-22 19:28:10,131 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppRunningOnNodeEvent.EventType: APP_RUNNING_ON_NODE 2015-11-22 19:28:10,132 DEBUG [IPC Server handler 6 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2015-11-22 19:28:10,132 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(777)) - Processing event for application_1448220478671_0001 of type APP_RUNNING_ON_NODE 2015-11-22 19:28:10,132 DEBUG [IPC Server handler 6 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 1 procesingTime= 2 2015-11-22 19:28:10,132 DEBUG [IPC Server handler 6 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 6 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#5 Retry#0 2015-11-22 19:28:10,133 DEBUG [IPC Server handler 6 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 6 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#5 Retry#0 Wrote 384 bytes. 2015-11-22 19:28:10,139 DEBUG [IPC Server listener on 50663] ipc.Server (Server.java:register(2883)) - Server connection from 172.31.20.42:35353; # active connections: 2; # queued calls: 0 2015-11-22 19:28:10,139 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processOneRpc(1897)) - got #-33 2015-11-22 19:28:10,140 DEBUG [Socket Reader #1 for port 50663] security.SaslRpcServer (SaslRpcServer.java:create(175)) - Created SASL server with mechanism = DIGEST-MD5 2015-11-22 19:28:10,140 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processResponse(1099)) - Socket Reader #1 for port 50663: responding to null from 172.31.20.42:35353 Call#-33 Retry#-1 2015-11-22 19:28:10,140 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processResponse(1118)) - Socket Reader #1 for port 50663: responding to null from 172.31.20.42:35353 Call#-33 Retry#-1 Wrote 166 bytes. 2015-11-22 19:28:10,143 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processOneRpc(1897)) - got #-33 2015-11-22 19:28:10,143 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processSaslToken(1529)) - Have read input token of size 350 for processing by saslServer.evaluateResponse() 2015-11-22 19:28:10,143 DEBUG [Socket Reader #1 for port 50663] security.BaseNMTokenSecretManager (BaseNMTokenSecretManager.java:retrivePasswordInternal(103)) - creating password for appattempt_1448220478671_0001_000001 for user ubuntu to run on NM ip-172-31-20-42.ap-northeast-1.compute.internal:50663 2015-11-22 19:28:10,143 DEBUG [Socket Reader #1 for port 50663] security.NMTokenIdentifier (NMTokenIdentifier.java:write(96)) - Writing NMTokenIdentifier to RPC layer: appAttemptId { application_id { id: 1 cluster_timestamp: 1448220478671 } attemptId: 1 } nodeId { host: "ip-172-31-20-42.ap-northeast-1.compute.internal" port: 50663 } appSubmitter: "ubuntu" keyId: 744666633 2015-11-22 19:28:10,143 DEBUG [Socket Reader #1 for port 50663] security.NMTokenSecretManagerInNM (NMTokenSecretManagerInNM.java:retrievePassword(180)) - NMToken password retrieved successfully!! 2015-11-22 19:28:10,143 DEBUG [Socket Reader #1 for port 50663] security.SaslRpcServer (SaslRpcServer.java:handle(310)) - SASL server DIGEST-MD5 callback: setting password for client: appattempt_1448220478671_0001_000001 (auth:SIMPLE) 2015-11-22 19:28:10,143 DEBUG [Socket Reader #1 for port 50663] security.SaslRpcServer (SaslRpcServer.java:handle(328)) - SASL server DIGEST-MD5 callback: setting canonicalized client ID: appattempt_1448220478671_0001_000001 2015-11-22 19:28:10,144 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:buildSaslResponse(1546)) - Will send SUCCESS token of size 40 from saslServer. 2015-11-22 19:28:10,144 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:saslProcess(1434)) - SASL server context established. Negotiated QoP is auth 2015-11-22 19:28:10,144 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:saslProcess(1439)) - SASL server successfully authenticated client: appattempt_1448220478671_0001_000001 (auth:SIMPLE) 2015-11-22 19:28:10,144 INFO [Socket Reader #1 for port 50663] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220478671_0001_000001 (auth:SIMPLE) 2015-11-22 19:28:10,144 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processResponse(1099)) - Socket Reader #1 for port 50663: responding to null from 172.31.20.42:35353 Call#-33 Retry#-1 2015-11-22 19:28:10,144 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processResponse(1118)) - Socket Reader #1 for port 50663: responding to null from 172.31.20.42:35353 Call#-33 Retry#-1 Wrote 64 bytes. 2015-11-22 19:28:10,146 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processOneRpc(1897)) - got #-3 2015-11-22 19:28:10,146 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:authorizeConnection(2097)) - Successfully authorized userInfo { } protocol: "org.apache.hadoop.yarn.api.ContainerManagementProtocolPB" 2015-11-22 19:28:10,146 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processOneRpc(1897)) - got #6 2015-11-22 19:28:10,146 DEBUG [IPC Server handler 1 on 50663] ipc.Server (Server.java:run(2185)) - IPC Server handler 1 on 50663: org.apache.hadoop.yarn.api.ContainerManagementProtocolPB.startContainers from 172.31.20.42:35353 Call#6 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:10,147 DEBUG [IPC Server handler 1 on 50663] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:10,147 DEBUG [IPC Server handler 1 on 50663] security.BaseContainerTokenSecretManager (BaseContainerTokenSecretManager.java:retrievePasswordInternal(118)) - Retrieving password for container_1448220478671_0001_01_000003 for user container_1448220478671_0001_01_000003 (auth:SIMPLE) to be run on NM ip-172-31-20-42.ap-northeast-1.compute.internal:50663 2015-11-22 19:28:10,147 DEBUG [IPC Server handler 1 on 50663] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(197)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 1 cluster_timestamp: 1448220478671 } attemptId: 1 } id: 3 } nmHostAddr: "ip-172-31-20-42.ap-northeast-1.compute.internal:50663" appSubmitter: "ubuntu" resource { memory: 128 virtual_cores: 1 } expiryTimeStamp: 1448221090130 masterKeyId: -550760804 rmIdentifier: 1448220478671 priority { priority: 0 } creationTime: 1448220489307 nodeLabelExpression: "" containerType: TASK 2015-11-22 19:28:10,148 INFO [IPC Server handler 1 on 50663] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:startContainerInternal(889)) - Start request for container_1448220478671_0001_01_000003 by user ubuntu 2015-11-22 19:28:10,148 DEBUG [IPC Server handler 1 on 50663] utils.YarnServerSecurityUtils (YarnServerSecurityUtils.java:parseCredentials(135)) - = Kind: YARN_AM_RM_TOKEN, Service: , Ident: (appAttemptId { application_id { id: 1 cluster_timestamp: 1448220478671 } attemptId: 1 } keyId: -1778540031) 2015-11-22 19:28:10,148 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationContainerInitEvent.EventType: INIT_CONTAINER 2015-11-22 19:28:10,148 INFO [IPC Server handler 1 on 50663] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu IP=172.31.20.42 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1448220478671_0001 CONTAINERID=container_1448220478671_0001_01_000003 2015-11-22 19:28:10,150 DEBUG [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(452)) - Processing application_1448220478671_0001 of type INIT_CONTAINER 2015-11-22 19:28:10,150 DEBUG [IPC Server handler 1 on 50663] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: startContainers queueTime= 1 procesingTime= 3 2015-11-22 19:28:10,150 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(303)) - Adding container_1448220478671_0001_01_000003 to application application_1448220478671_0001 2015-11-22 19:28:10,152 DEBUG [IPC Server handler 1 on 50663] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 1 on 50663: responding to org.apache.hadoop.yarn.api.ContainerManagementProtocolPB.startContainers from 172.31.20.42:35353 Call#6 Retry#0 2015-11-22 19:28:10,152 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerInitEvent.EventType: INIT_CONTAINER 2015-11-22 19:28:10,152 DEBUG [IPC Server handler 1 on 50663] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 1 on 50663: responding to org.apache.hadoop.yarn.api.ContainerManagementProtocolPB.startContainers from 172.31.20.42:35353 Call#6 Retry#0 Wrote 51 bytes. 2015-11-22 19:28:10,152 DEBUG [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1131)) - Processing container_1448220478671_0001_01_000003 of type INIT_CONTAINER 2015-11-22 19:28:10,152 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220478671_0001_01_000003 transitioned from NEW to LOCALIZED 2015-11-22 19:28:10,153 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:close(2894)) - Socket Reader #1 for port 50663: disconnecting client 172.31.20.42:35353. Number of active connections: 1 2015-11-22 19:28:10,153 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServicesEvent.EventType: CONTAINER_INIT 2015-11-22 19:28:10,153 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_INIT for appId application_1448220478671_0001 2015-11-22 19:28:10,153 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncherEvent.EventType: LAUNCH_CONTAINER 2015-11-22 19:28:10,154 DEBUG [ContainersLauncher #1] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.fs.FileContext.getAbstractFileSystem(FileContext.java:339) 2015-11-22 19:28:10,159 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - REQUEST /ws/v1/timeline/ on org.mortbay.jetty.HttpConnection@7ffee0a9 2015-11-22 19:28:10,159 DEBUG [IPC Server listener on 50663] ipc.Server (Server.java:register(2883)) - Server connection from 172.31.20.42:35355; # active connections: 2; # queued calls: 0 2015-11-22 19:28:10,159 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - sessionManager=org.mortbay.jetty.servlet.HashSessionManager@3966697d 2015-11-22 19:28:10,164 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - session=null 2015-11-22 19:28:10,164 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processOneRpc(1897)) - got #-33 2015-11-22 19:28:10,164 DEBUG [Socket Reader #1 for port 50663] security.SaslRpcServer (SaslRpcServer.java:create(175)) - Created SASL server with mechanism = DIGEST-MD5 2015-11-22 19:28:10,164 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processResponse(1099)) - Socket Reader #1 for port 50663: responding to null from 172.31.20.42:35355 Call#-33 Retry#-1 2015-11-22 19:28:10,164 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processResponse(1118)) - Socket Reader #1 for port 50663: responding to null from 172.31.20.42:35355 Call#-33 Retry#-1 Wrote 166 bytes. 2015-11-22 19:28:10,164 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - servlet=org.mortbay.jetty.servlet.DefaultServlet-55176785 2015-11-22 19:28:10,164 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - chain=NoCacheFilter->NoCacheFilter->safety->Timeline Authentication Filter->RMAuthenticationFilter->static_user_filter->guice->org.mortbay.jetty.servlet.DefaultServlet-55176785 2015-11-22 19:28:10,165 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - servlet holder=org.mortbay.jetty.servlet.DefaultServlet-55176785 2015-11-22 19:28:10,165 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter NoCacheFilter 2015-11-22 19:28:10,165 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter NoCacheFilter 2015-11-22 19:28:10,165 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter safety 2015-11-22 19:28:10,165 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter Timeline Authentication Filter 2015-11-22 19:28:10,165 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(557)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:41381/ws/v1/timeline/?user.name=ubuntu] triggering authentication 2015-11-22 19:28:10,165 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(576)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:41381/ws/v1/timeline/?user.name=ubuntu] user [ubuntu] authenticated 2015-11-22 19:28:10,166 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter RMAuthenticationFilter 2015-11-22 19:28:10,166 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(557)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:41381/ws/v1/timeline/?user.name=ubuntu] triggering authentication 2015-11-22 19:28:10,166 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(576)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:41381/ws/v1/timeline/?user.name=ubuntu] user [ubuntu] authenticated 2015-11-22 19:28:10,166 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter static_user_filter 2015-11-22 19:28:10,166 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter guice 2015-11-22 19:28:10,170 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - getResource(META-INF/services/javax.xml.bind.JAXBContext)=jar:file:/home/ubuntu/.m2/repository/com/sun/xml/bind/jaxb-impl/2.2.3-1/jaxb-impl-2.2.3-1.jar!/META-INF/services/javax.xml.bind.JAXBContext 2015-11-22 19:28:10,171 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.xml.bind.v2.ContextFactory from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:28:10,172 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processOneRpc(1897)) - got #-33 2015-11-22 19:28:10,172 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processSaslToken(1529)) - Have read input token of size 350 for processing by saslServer.evaluateResponse() 2015-11-22 19:28:10,172 DEBUG [Socket Reader #1 for port 50663] security.BaseNMTokenSecretManager (BaseNMTokenSecretManager.java:retrivePasswordInternal(103)) - creating password for appattempt_1448220478671_0001_000001 for user ubuntu to run on NM ip-172-31-20-42.ap-northeast-1.compute.internal:50663 2015-11-22 19:28:10,172 DEBUG [Socket Reader #1 for port 50663] security.NMTokenIdentifier (NMTokenIdentifier.java:write(96)) - Writing NMTokenIdentifier to RPC layer: appAttemptId { application_id { id: 1 cluster_timestamp: 1448220478671 } attemptId: 1 } nodeId { host: "ip-172-31-20-42.ap-northeast-1.compute.internal" port: 50663 } appSubmitter: "ubuntu" keyId: 744666633 2015-11-22 19:28:10,173 DEBUG [Socket Reader #1 for port 50663] security.NMTokenSecretManagerInNM (NMTokenSecretManagerInNM.java:retrievePassword(180)) - NMToken password retrieved successfully!! 2015-11-22 19:28:10,173 DEBUG [Socket Reader #1 for port 50663] security.SaslRpcServer (SaslRpcServer.java:handle(310)) - SASL server DIGEST-MD5 callback: setting password for client: appattempt_1448220478671_0001_000001 (auth:SIMPLE) 2015-11-22 19:28:10,173 DEBUG [Socket Reader #1 for port 50663] security.SaslRpcServer (SaslRpcServer.java:handle(328)) - SASL server DIGEST-MD5 callback: setting canonicalized client ID: appattempt_1448220478671_0001_000001 2015-11-22 19:28:10,173 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:buildSaslResponse(1546)) - Will send SUCCESS token of size 40 from saslServer. 2015-11-22 19:28:10,173 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:saslProcess(1434)) - SASL server context established. Negotiated QoP is auth 2015-11-22 19:28:10,173 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:saslProcess(1439)) - SASL server successfully authenticated client: appattempt_1448220478671_0001_000001 (auth:SIMPLE) 2015-11-22 19:28:10,174 INFO [Socket Reader #1 for port 50663] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220478671_0001_000001 (auth:SIMPLE) 2015-11-22 19:28:10,174 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processResponse(1099)) - Socket Reader #1 for port 50663: responding to null from 172.31.20.42:35355 Call#-33 Retry#-1 2015-11-22 19:28:10,174 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processResponse(1118)) - Socket Reader #1 for port 50663: responding to null from 172.31.20.42:35355 Call#-33 Retry#-1 Wrote 64 bytes. 2015-11-22 19:28:10,176 DEBUG [IPC Server idle connection scanner for port 50663] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 50663: task running 2015-11-22 19:28:10,183 DEBUG [IPC Server idle connection scanner for port 47651] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 47651: task running 2015-11-22 19:28:10,189 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerEvent.EventType: CONTAINER_LAUNCHED 2015-11-22 19:28:10,189 DEBUG [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1131)) - Processing container_1448220478671_0001_01_000003 of type CONTAINER_LAUNCHED 2015-11-22 19:28:10,189 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220478671_0001_01_000003 transitioned from LOCALIZED to RUNNING 2015-11-22 19:28:10,189 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerStartMonitoringEvent.EventType: START_MONITORING_CONTAINER 2015-11-22 19:28:10,196 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processOneRpc(1897)) - got #-3 2015-11-22 19:28:10,196 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:authorizeConnection(2097)) - Successfully authorized userInfo { } protocol: "org.apache.hadoop.yarn.api.ContainerManagementProtocolPB" 2015-11-22 19:28:10,196 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processOneRpc(1897)) - got #7 2015-11-22 19:28:10,196 DEBUG [IPC Server handler 2 on 50663] ipc.Server (Server.java:run(2185)) - IPC Server handler 2 on 50663: org.apache.hadoop.yarn.api.ContainerManagementProtocolPB.getContainerStatuses from 172.31.20.42:35355 Call#7 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:10,197 DEBUG [IPC Server handler 2 on 50663] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:10,197 INFO [IPC Server handler 2 on 50663] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:getContainerStatusInternal(1221)) - Getting container-status for container_1448220478671_0001_01_000003 2015-11-22 19:28:10,198 INFO [IPC Server handler 2 on 50663] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:getContainerStatusInternal(1235)) - Returning ContainerStatus: [ContainerId: container_1448220478671_0001_01_000003, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ] 2015-11-22 19:28:10,198 DEBUG [IPC Server handler 2 on 50663] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getContainerStatuses queueTime= 1 procesingTime= 1 2015-11-22 19:28:10,199 DEBUG [IPC Server handler 2 on 50663] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 2 on 50663: responding to org.apache.hadoop.yarn.api.ContainerManagementProtocolPB.getContainerStatuses from 172.31.20.42:35355 Call#7 Retry#0 2015-11-22 19:28:10,200 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:close(2894)) - Socket Reader #1 for port 50663: disconnecting client 172.31.20.42:35355. Number of active connections: 1 2015-11-22 19:28:10,200 DEBUG [IPC Server handler 2 on 50663] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 2 on 50663: responding to org.apache.hadoop.yarn.api.ContainerManagementProtocolPB.getContainerStatuses from 172.31.20.42:35355 Call#7 Retry#0 Wrote 75 bytes. 2015-11-22 19:28:10,221 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - RESPONSE /ws/v1/timeline/ 200 2015-11-22 19:28:10,224 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - REQUEST /ws/v1/timeline/ on org.mortbay.jetty.HttpConnection@5de02011 2015-11-22 19:28:10,225 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - sessionManager=org.mortbay.jetty.servlet.HashSessionManager@3966697d 2015-11-22 19:28:10,225 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - session=null 2015-11-22 19:28:10,225 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - servlet=org.mortbay.jetty.servlet.DefaultServlet-55176785 2015-11-22 19:28:10,225 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - chain=NoCacheFilter->NoCacheFilter->safety->Timeline Authentication Filter->RMAuthenticationFilter->static_user_filter->guice->org.mortbay.jetty.servlet.DefaultServlet-55176785 2015-11-22 19:28:10,225 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - servlet holder=org.mortbay.jetty.servlet.DefaultServlet-55176785 2015-11-22 19:28:10,225 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter NoCacheFilter 2015-11-22 19:28:10,225 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter NoCacheFilter 2015-11-22 19:28:10,226 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter safety 2015-11-22 19:28:10,226 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter Timeline Authentication Filter 2015-11-22 19:28:10,226 INFO [ContainersLauncher #1] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:buildCommandExecutor(281)) - launchContainer: [nice, -n, 0, bash, /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220478671_0001/container_1448220478671_0001_01_000003/default_container_executor.sh] 2015-11-22 19:28:10,228 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(576)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:41381/ws/v1/timeline/] user [ubuntu] authenticated 2015-11-22 19:28:10,228 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter RMAuthenticationFilter 2015-11-22 19:28:10,228 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(576)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:41381/ws/v1/timeline/] user [ubuntu] authenticated 2015-11-22 19:28:10,228 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter static_user_filter 2015-11-22 19:28:10,229 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter guice 2015-11-22 19:28:10,231 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] security.TimelineACLsManager (TimelineACLsManager.java:checkAccess(106)) - Verifying the access of ubuntu on the timeline entity { id: container_1448220478671_0001_01_000003, type: DS_CONTAINER } 2015-11-22 19:28:10,232 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - RESPONSE /ws/v1/timeline/ 200 2015-11-22 19:28:10,242 DEBUG [ContainersLauncher #1] launcher.ContainerLaunch (ContainerLaunch.java:call(333)) - Container container_1448220478671_0001_01_000003 completed with exit code 0 2015-11-22 19:28:10,242 INFO [ContainersLauncher #1] launcher.ContainerLaunch (ContainerLaunch.java:call(356)) - Container container_1448220478671_0001_01_000003 succeeded 2015-11-22 19:28:10,242 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerEvent.EventType: CONTAINER_EXITED_WITH_SUCCESS 2015-11-22 19:28:10,242 DEBUG [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1131)) - Processing container_1448220478671_0001_01_000003 of type CONTAINER_EXITED_WITH_SUCCESS 2015-11-22 19:28:10,242 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220478671_0001_01_000003 transitioned from RUNNING to EXITED_WITH_SUCCESS 2015-11-22 19:28:10,242 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncherEvent.EventType: CLEANUP_CONTAINER 2015-11-22 19:28:10,243 INFO [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(380)) - Cleaning up container container_1448220478671_0001_01_000003 2015-11-22 19:28:10,243 DEBUG [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(397)) - Marking container container_1448220478671_0001_01_000003 as inactive 2015-11-22 19:28:10,243 DEBUG [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(403)) - Getting pid for container container_1448220478671_0001_01_000003 to kill from pid file /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/application_1448220478671_0001/container_1448220478671_0001_01_000003/container_1448220478671_0001_01_000003.pid 2015-11-22 19:28:10,243 DEBUG [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:getContainerPid(569)) - Accessing pid for container container_1448220478671_0001_01_000003 from pid file /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/application_1448220478671_0001/container_1448220478671_0001_01_000003/container_1448220478671_0001_01_000003.pid 2015-11-22 19:28:10,243 DEBUG [AsyncDispatcher event handler] util.ProcessIdFileReader (ProcessIdFileReader.java:getProcessId(51)) - Accessing pid from pid file /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/application_1448220478671_0001/container_1448220478671_0001_01_000003/container_1448220478671_0001_01_000003.pid 2015-11-22 19:28:10,243 DEBUG [AsyncDispatcher event handler] util.ProcessIdFileReader (ProcessIdFileReader.java:getProcessId(99)) - Got pid 30130 from path /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/application_1448220478671_0001/container_1448220478671_0001_01_000003/container_1448220478671_0001_01_000003.pid 2015-11-22 19:28:10,243 DEBUG [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:getContainerPid(579)) - Got pid 30130 for container container_1448220478671_0001_01_000003 2015-11-22 19:28:10,243 DEBUG [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(421)) - Sending signal to pid 30130 as user ubuntu for container container_1448220478671_0001_01_000003 2015-11-22 19:28:10,243 DEBUG [AsyncDispatcher event handler] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:signalContainer(417)) - Sending signal 15 to pid 30130 as user ubuntu 2015-11-22 19:28:10,249 DEBUG [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(437)) - Sent signal SIGTERM to pid 30130 as user ubuntu for container container_1448220478671_0001_01_000003, result=failed 2015-11-22 19:28:10,249 DEBUG [AsyncDispatcher event handler] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.fs.FileContext.getAbstractFileSystem(FileContext.java:339) 2015-11-22 19:28:10,260 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.ContainerLocalizationCleanupEvent.EventType: CLEANUP_CONTAINER_RESOURCES 2015-11-22 19:28:10,260 DEBUG [DeletionService #1] nodemanager.DeletionService (DeletionService.java:run(266)) - FileDeletionTask : user : ubuntu subDir : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220478671_0001/container_1448220478671_0001_01_000003 baseDir : null 2015-11-22 19:28:10,262 DEBUG [DeletionService #1] nodemanager.DeletionService (DeletionService.java:run(292)) - Deleting path: [/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220478671_0001/container_1448220478671_0001_01_000003] as user: [ubuntu] 2015-11-22 19:28:10,262 INFO [DeletionService #1] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(480)) - Deleting absolute path : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220478671_0001/container_1448220478671_0001_01_000003 2015-11-22 19:28:10,263 DEBUG [DeletionService #3] nodemanager.DeletionService (DeletionService.java:run(266)) - FileDeletionTask : user : null subDir : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/application_1448220478671_0001/container_1448220478671_0001_01_000003 baseDir : null 2015-11-22 19:28:10,264 DEBUG [DeletionService #3] nodemanager.DeletionService (DeletionService.java:run(271)) - NM deleting absolute path : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/application_1448220478671_0001/container_1448220478671_0001_01_000003 2015-11-22 19:28:10,263 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerEvent.EventType: CONTAINER_RESOURCES_CLEANEDUP 2015-11-22 19:28:10,264 DEBUG [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1131)) - Processing container_1448220478671_0001_01_000003 of type CONTAINER_RESOURCES_CLEANEDUP 2015-11-22 19:28:10,264 INFO [AsyncDispatcher event handler] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu OPERATION=Container Finished - Succeeded TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1448220478671_0001 CONTAINERID=container_1448220478671_0001_01_000003 2015-11-22 19:28:10,264 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220478671_0001_01_000003 transitioned from EXITED_WITH_SUCCESS to DONE 2015-11-22 19:28:10,264 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationContainerFinishedEvent.EventType: APPLICATION_CONTAINER_FINISHED 2015-11-22 19:28:10,264 DEBUG [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(452)) - Processing application_1448220478671_0001 of type APPLICATION_CONTAINER_FINISHED 2015-11-22 19:28:10,265 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(346)) - Removing container_1448220478671_0001_01_000003 from application application_1448220478671_0001 2015-11-22 19:28:10,265 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerStopMonitoringEvent.EventType: STOP_MONITORING_CONTAINER 2015-11-22 19:28:10,265 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.event.LogHandlerContainerFinishedEvent.EventType: CONTAINER_FINISHED 2015-11-22 19:28:10,265 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServicesEvent.EventType: CONTAINER_STOP 2015-11-22 19:28:10,265 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_STOP for appId application_1448220478671_0001 2015-11-22 19:28:10,304 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:28:10,305 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 3 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ], ContainerStatus: [ContainerId: container_1448220478671_0001_01_000003, State: COMPLETE, Capability: , Diagnostics: , ExitStatus: 0, ], ContainerStatus: [ContainerId: container_1448220478671_0001_01_000002, State: COMPLETE, Capability: , Diagnostics: , ExitStatus: 0, ]] 2015-11-22 19:28:10,305 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:10,305 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:28:10,305 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:28:10,305 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:28:10,306 DEBUG [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(424)) - Processing container_1448220478671_0001_01_000003 of type FINISHED 2015-11-22 19:28:10,306 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220478671_0001_01_000003 Container Transitioned from ACQUIRED to COMPLETED 2015-11-22 19:28:10,306 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptContainerFinishedEvent.EventType: CONTAINER_FINISHED 2015-11-22 19:28:10,306 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220478671_0001 CONTAINERID=container_1448220478671_0001_01_000003 2015-11-22 19:28:10,306 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type CONTAINER_FINISHED 2015-11-22 19:28:10,306 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:releaseContainer(248)) - Released container container_1448220478671_0001_01_000003 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:50663, which currently has 2 containers, used and available, release resources=true 2015-11-22 19:28:10,306 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2015-11-22 19:28:10,306 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:releaseResource(1313)) - default used= numContainers=2 user=ubuntu user-resources= 2015-11-22 19:28:10,307 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:internalReleaseResource(630)) - completedContainer root: numChildQueue= 1, capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.15625, numApps=1, numContainers=2, cluster= 2015-11-22 19:28:10,307 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:internalReleaseResource(643)) - Re-sorting completed queue: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.15625, absoluteUsedCapacity=0.15625, numApps=1, numContainers=2 2015-11-22 19:28:10,307 DEBUG [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(424)) - Processing container_1448220478671_0001_01_000002 of type FINISHED 2015-11-22 19:28:10,307 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220478671_0001_01_000002 Container Transitioned from RUNNING to COMPLETED 2015-11-22 19:28:10,307 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220478671_0001 CONTAINERID=container_1448220478671_0001_01_000002 2015-11-22 19:28:10,307 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:releaseContainer(248)) - Released container container_1448220478671_0001_01_000002 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:50663, which currently has 1 containers, used and available, release resources=true 2015-11-22 19:28:10,307 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2015-11-22 19:28:10,307 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:releaseResource(1313)) - default used= numContainers=1 user=ubuntu user-resources= 2015-11-22 19:28:10,307 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:internalReleaseResource(630)) - completedContainer root: numChildQueue= 1, capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.125, numApps=1, numContainers=1, cluster= 2015-11-22 19:28:10,308 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:internalReleaseResource(643)) - Re-sorting completed queue: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.125, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 2015-11-22 19:28:10,308 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:28:10,308 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:28:10,308 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(408)) - Trying to assign containers to child-queue of root 2015-11-22 19:28:10,308 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:printChildQueues(617)) - printChildQueues - queue: root child-queues: root.defaultusedCapacity=(0.125), label=(*) 2015-11-22 19:28:10,308 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(564)) - Trying to assign to queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.125, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 2015-11-22 19:28:10,308 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:assignContainers(817)) - assignContainers: node=ip-172-31-20-42.ap-northeast-1.compute.internal #applications=1 2015-11-22 19:28:10,309 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2015-11-22 19:28:10,309 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimitAndSetHeadroom(984)) - Headroom calculation for user ubuntu: userLimit= queueMaxAvailRes= consumed= headroom= 2015-11-22 19:28:10,309 DEBUG [ResourceManager Event Processor] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:assignContainers(477)) - pre-assignContainers for application application_1448220478671_0001 2015-11-22 19:28:10,309 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2015-11-22 19:28:10,309 DEBUG [ResourceManager Event Processor] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(470)) - showRequests: application=application_1448220478671_0001 headRoom= currentConsumption=512 2015-11-22 19:28:10,309 DEBUG [ResourceManager Event Processor] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(474)) - showRequests: application=application_1448220478671_0001 request={Priority: 0, Capability: , # Containers: 2, Location: *, Relax Locality: true, Node Label Expression: } 2015-11-22 19:28:10,309 DEBUG [ResourceManager Event Processor] allocator.IncreaseContainerAllocator (IncreaseContainerAllocator.java:assignContainers(184)) - Skip allocating increase request since we don't have any increase request on this node=ip-172-31-20-42.ap-northeast-1.compute.internal:50663 2015-11-22 19:28:10,309 DEBUG [ResourceManager Event Processor] allocator.RegularContainerAllocator (RegularContainerAllocator.java:assignContainer(407)) - assignContainers: node=ip-172-31-20-42.ap-northeast-1.compute.internal application=application_1448220478671_0001 priority=0 request={Priority: 0, Capability: , # Containers: 2, Location: *, Relax Locality: true, Node Label Expression: } type=OFF_SWITCH 2015-11-22 19:28:10,310 DEBUG [ResourceManager Event Processor] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:allocate(594)) - allocate: applicationId=application_1448220478671_0001 container=container_1448220478671_0001_01_000004 host=ip-172-31-20-42.ap-northeast-1.compute.internal:50663 user=ubuntu resource= type=OFF_SWITCH 2015-11-22 19:28:10,310 DEBUG [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(424)) - Processing container_1448220478671_0001_01_000004 of type START 2015-11-22 19:28:10,310 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220478671_0001_01_000004 Container Transitioned from NEW to ALLOCATED 2015-11-22 19:28:10,310 DEBUG [ResourceManager Event Processor] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:allocate(208)) - allocate: applicationAttemptId=appattempt_1448220478671_0001_000001 container=container_1448220478671_0001_01_000004 host=ip-172-31-20-42.ap-northeast-1.compute.internal type=OFF_SWITCH 2015-11-22 19:28:10,310 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220478671_0001 CONTAINERID=container_1448220478671_0001_01_000004 2015-11-22 19:28:10,310 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(154)) - Assigned container container_1448220478671_0001_01_000004 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:50663, which has 2 containers, used and available after allocation 2015-11-22 19:28:10,310 INFO [ResourceManager Event Processor] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(89)) - assignedContainer application attempt=appattempt_1448220478671_0001_000001 container=container_1448220478671_0001_01_000004 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@33e07bad clusterResource= type=OFF_SWITCH 2015-11-22 19:28:10,310 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:assignContainers(881)) - post-assignContainers for application application_1448220478671_0001 2015-11-22 19:28:10,310 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2015-11-22 19:28:10,311 DEBUG [ResourceManager Event Processor] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(470)) - showRequests: application=application_1448220478671_0001 headRoom= currentConsumption=640 2015-11-22 19:28:10,312 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptContainerFinishedEvent.EventType: CONTAINER_FINISHED 2015-11-22 19:28:10,312 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type CONTAINER_FINISHED 2015-11-22 19:28:10,312 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: CONTAINER_ALLOCATED 2015-11-22 19:28:10,312 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type CONTAINER_ALLOCATED 2015-11-22 19:28:10,312 DEBUG [ResourceManager Event Processor] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(474)) - showRequests: application=application_1448220478671_0001 request={Priority: 0, Capability: , # Containers: 1, Location: *, Relax Locality: true, Node Label Expression: } 2015-11-22 19:28:10,312 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2015-11-22 19:28:10,313 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2015-11-22 19:28:10,313 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2015-11-22 19:28:10,313 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:allocateResource(1277)) - default user=ubuntu used= numContainers=2 headroom = user-resources= 2015-11-22 19:28:10,313 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(575)) - Assigned to queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.15625, absoluteUsedCapacity=0.15625, numApps=1, numContainers=2 --> , OFF_SWITCH 2015-11-22 19:28:10,313 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(589)) - Re-sorting assigned queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.15625, absoluteUsedCapacity=0.15625, numApps=1, numContainers=2 2015-11-22 19:28:10,314 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:printChildQueues(617)) - printChildQueues - queue: root child-queues: root.defaultusedCapacity=(0.15625), label=(*) 2015-11-22 19:28:10,314 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(461)) - assignedContainer queue=root usedCapacity=0.15625 absoluteUsedCapacity=0.15625 used= cluster= 2015-11-22 19:28:10,314 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(473)) - ParentQ=root assignedSoFarInThisIteration= usedCapacity=0.15625 absoluteUsedCapacity=0.15625 2015-11-22 19:28:10,314 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(484)) - Not assigning more than one off-switch container, assignments so far: resource:; type:OFF_SWITCH; excessReservation:null; applicationid:null; skipped:false; fulfilled reservation:false; allocations(count/resource):1/; reservations(count/resource):0/ 2015-11-22 19:28:10,499 DEBUG [Task killer for 30130] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:signalContainer(417)) - Sending signal 9 to pid 30130 as user ubuntu 2015-11-22 19:28:10,807 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #831 2015-11-22 19:28:10,807 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #831 2015-11-22 19:28:10,807 DEBUG [IPC Server handler 13 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 13 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#831 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:10,807 DEBUG [IPC Server handler 13 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:10,808 DEBUG [IPC Server handler 13 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:28:10,808 DEBUG [IPC Server handler 13 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 1 2015-11-22 19:28:10,808 DEBUG [IPC Server handler 13 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 13 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#831 Retry#0 2015-11-22 19:28:10,808 DEBUG [IPC Server handler 13 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 13 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#831 Retry#0 Wrote 359 bytes. 2015-11-22 19:28:10,808 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #831 2015-11-22 19:28:10,808 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:28:10,809 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:28:11,134 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #8 2015-11-22 19:28:11,134 DEBUG [IPC Server handler 8 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 8 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#8 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:11,135 DEBUG [IPC Server handler 8 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:11,135 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:11,136 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:28:11,136 DEBUG [IPC Server handler 8 on 46284] security.BaseContainerTokenSecretManager (BaseContainerTokenSecretManager.java:createPassword(90)) - Creating password for container_1448220478671_0001_01_000004 for user container_1448220478671_0001_01_000004 (auth:SIMPLE) to be run on NM ip-172-31-20-42.ap-northeast-1.compute.internal:50663 2015-11-22 19:28:11,136 DEBUG [IPC Server handler 8 on 46284] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(197)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 1 cluster_timestamp: 1448220478671 } attemptId: 1 } id: 4 } nmHostAddr: "ip-172-31-20-42.ap-northeast-1.compute.internal:50663" appSubmitter: "ubuntu" resource { memory: 128 virtual_cores: 1 } expiryTimeStamp: 1448221091136 masterKeyId: -550760804 rmIdentifier: 1448220478671 priority { priority: 0 } creationTime: 1448220490310 nodeLabelExpression: "" containerType: TASK 2015-11-22 19:28:11,136 DEBUG [IPC Server handler 8 on 46284] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(197)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 1 cluster_timestamp: 1448220478671 } attemptId: 1 } id: 4 } nmHostAddr: "ip-172-31-20-42.ap-northeast-1.compute.internal:50663" appSubmitter: "ubuntu" resource { memory: 128 virtual_cores: 1 } expiryTimeStamp: 1448221091136 masterKeyId: -550760804 rmIdentifier: 1448220478671 priority { priority: 0 } creationTime: 1448220490310 nodeLabelExpression: "" containerType: TASK 2015-11-22 19:28:11,136 DEBUG [IPC Server handler 8 on 46284] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(424)) - Processing container_1448220478671_0001_01_000004 of type ACQUIRED 2015-11-22 19:28:11,137 INFO [IPC Server handler 8 on 46284] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220478671_0001_01_000004 Container Transitioned from ALLOCATED to ACQUIRED 2015-11-22 19:28:11,137 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppRunningOnNodeEvent.EventType: APP_RUNNING_ON_NODE 2015-11-22 19:28:11,137 DEBUG [IPC Server handler 8 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2015-11-22 19:28:11,137 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(777)) - Processing event for application_1448220478671_0001 of type APP_RUNNING_ON_NODE 2015-11-22 19:28:11,138 DEBUG [IPC Server handler 8 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 1 procesingTime= 3 2015-11-22 19:28:11,138 DEBUG [IPC Server handler 8 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 8 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#8 Retry#0 2015-11-22 19:28:11,138 DEBUG [IPC Server handler 8 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 8 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#8 Retry#0 Wrote 452 bytes. 2015-11-22 19:28:11,141 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - REQUEST /ws/v1/timeline/ on org.mortbay.jetty.HttpConnection@5de02011 2015-11-22 19:28:11,141 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - sessionManager=org.mortbay.jetty.servlet.HashSessionManager@3966697d 2015-11-22 19:28:11,141 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - session=null 2015-11-22 19:28:11,141 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - servlet=org.mortbay.jetty.servlet.DefaultServlet-55176785 2015-11-22 19:28:11,141 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - chain=NoCacheFilter->NoCacheFilter->safety->Timeline Authentication Filter->RMAuthenticationFilter->static_user_filter->guice->org.mortbay.jetty.servlet.DefaultServlet-55176785 2015-11-22 19:28:11,141 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - servlet holder=org.mortbay.jetty.servlet.DefaultServlet-55176785 2015-11-22 19:28:11,142 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter NoCacheFilter 2015-11-22 19:28:11,142 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter NoCacheFilter 2015-11-22 19:28:11,142 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter safety 2015-11-22 19:28:11,142 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter Timeline Authentication Filter 2015-11-22 19:28:11,142 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(557)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:41381/ws/v1/timeline/?user.name=ubuntu] triggering authentication 2015-11-22 19:28:11,142 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(576)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:41381/ws/v1/timeline/?user.name=ubuntu] user [ubuntu] authenticated 2015-11-22 19:28:11,142 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter RMAuthenticationFilter 2015-11-22 19:28:11,142 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(557)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:41381/ws/v1/timeline/?user.name=ubuntu] triggering authentication 2015-11-22 19:28:11,143 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(576)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:41381/ws/v1/timeline/?user.name=ubuntu] user [ubuntu] authenticated 2015-11-22 19:28:11,143 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter static_user_filter 2015-11-22 19:28:11,143 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter guice 2015-11-22 19:28:11,146 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - getResource(META-INF/services/javax.xml.bind.JAXBContext)=jar:file:/home/ubuntu/.m2/repository/com/sun/xml/bind/jaxb-impl/2.2.3-1/jaxb-impl-2.2.3-1.jar!/META-INF/services/javax.xml.bind.JAXBContext 2015-11-22 19:28:11,146 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.xml.bind.v2.ContextFactory from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:28:11,171 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - RESPONSE /ws/v1/timeline/ 200 2015-11-22 19:28:11,174 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - REQUEST /ws/v1/timeline/ on org.mortbay.jetty.HttpConnection@55632f77 2015-11-22 19:28:11,174 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - sessionManager=org.mortbay.jetty.servlet.HashSessionManager@3966697d 2015-11-22 19:28:11,174 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - session=null 2015-11-22 19:28:11,175 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - servlet=org.mortbay.jetty.servlet.DefaultServlet-55176785 2015-11-22 19:28:11,175 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - chain=NoCacheFilter->NoCacheFilter->safety->Timeline Authentication Filter->RMAuthenticationFilter->static_user_filter->guice->org.mortbay.jetty.servlet.DefaultServlet-55176785 2015-11-22 19:28:11,175 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - servlet holder=org.mortbay.jetty.servlet.DefaultServlet-55176785 2015-11-22 19:28:11,175 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter NoCacheFilter 2015-11-22 19:28:11,175 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter NoCacheFilter 2015-11-22 19:28:11,175 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter safety 2015-11-22 19:28:11,175 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter Timeline Authentication Filter 2015-11-22 19:28:11,175 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(576)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:41381/ws/v1/timeline/] user [ubuntu] authenticated 2015-11-22 19:28:11,176 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter RMAuthenticationFilter 2015-11-22 19:28:11,176 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(576)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:41381/ws/v1/timeline/] user [ubuntu] authenticated 2015-11-22 19:28:11,176 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter static_user_filter 2015-11-22 19:28:11,176 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter guice 2015-11-22 19:28:11,177 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] security.TimelineACLsManager (TimelineACLsManager.java:checkAccess(106)) - Verifying the access of ubuntu on the timeline entity { id: container_1448220478671_0001_01_000003, type: DS_CONTAINER } 2015-11-22 19:28:11,177 DEBUG [IPC Server idle connection scanner for port 32936] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 32936: task running 2015-11-22 19:28:11,177 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - RESPONSE /ws/v1/timeline/ 200 2015-11-22 19:28:11,179 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - REQUEST /ws/v1/timeline/ on org.mortbay.jetty.HttpConnection@55632f77 2015-11-22 19:28:11,179 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - sessionManager=org.mortbay.jetty.servlet.HashSessionManager@3966697d 2015-11-22 19:28:11,179 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - session=null 2015-11-22 19:28:11,180 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - servlet=org.mortbay.jetty.servlet.DefaultServlet-55176785 2015-11-22 19:28:11,180 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - chain=NoCacheFilter->NoCacheFilter->safety->Timeline Authentication Filter->RMAuthenticationFilter->static_user_filter->guice->org.mortbay.jetty.servlet.DefaultServlet-55176785 2015-11-22 19:28:11,180 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - servlet holder=org.mortbay.jetty.servlet.DefaultServlet-55176785 2015-11-22 19:28:11,180 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter NoCacheFilter 2015-11-22 19:28:11,180 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter NoCacheFilter 2015-11-22 19:28:11,180 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter safety 2015-11-22 19:28:11,181 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter Timeline Authentication Filter 2015-11-22 19:28:11,181 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(557)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:41381/ws/v1/timeline/?user.name=ubuntu] triggering authentication 2015-11-22 19:28:11,181 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(576)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:41381/ws/v1/timeline/?user.name=ubuntu] user [ubuntu] authenticated 2015-11-22 19:28:11,181 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter RMAuthenticationFilter 2015-11-22 19:28:11,181 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(557)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:41381/ws/v1/timeline/?user.name=ubuntu] triggering authentication 2015-11-22 19:28:11,181 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(576)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:41381/ws/v1/timeline/?user.name=ubuntu] user [ubuntu] authenticated 2015-11-22 19:28:11,182 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter static_user_filter 2015-11-22 19:28:11,182 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter guice 2015-11-22 19:28:11,184 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - getResource(META-INF/services/javax.xml.bind.JAXBContext)=jar:file:/home/ubuntu/.m2/repository/com/sun/xml/bind/jaxb-impl/2.2.3-1/jaxb-impl-2.2.3-1.jar!/META-INF/services/javax.xml.bind.JAXBContext 2015-11-22 19:28:11,184 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.xml.bind.v2.ContextFactory from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:28:11,208 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - RESPONSE /ws/v1/timeline/ 200 2015-11-22 19:28:11,210 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - REQUEST /ws/v1/timeline/ on org.mortbay.jetty.HttpConnection@7248fad3 2015-11-22 19:28:11,210 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - sessionManager=org.mortbay.jetty.servlet.HashSessionManager@3966697d 2015-11-22 19:28:11,210 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - session=null 2015-11-22 19:28:11,210 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - servlet=org.mortbay.jetty.servlet.DefaultServlet-55176785 2015-11-22 19:28:11,210 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - chain=NoCacheFilter->NoCacheFilter->safety->Timeline Authentication Filter->RMAuthenticationFilter->static_user_filter->guice->org.mortbay.jetty.servlet.DefaultServlet-55176785 2015-11-22 19:28:11,210 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - servlet holder=org.mortbay.jetty.servlet.DefaultServlet-55176785 2015-11-22 19:28:11,210 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter NoCacheFilter 2015-11-22 19:28:11,210 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter NoCacheFilter 2015-11-22 19:28:11,211 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter safety 2015-11-22 19:28:11,211 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter Timeline Authentication Filter 2015-11-22 19:28:11,211 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(576)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:41381/ws/v1/timeline/] user [ubuntu] authenticated 2015-11-22 19:28:11,211 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter RMAuthenticationFilter 2015-11-22 19:28:11,211 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(576)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:41381/ws/v1/timeline/] user [ubuntu] authenticated 2015-11-22 19:28:11,211 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter static_user_filter 2015-11-22 19:28:11,211 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter guice 2015-11-22 19:28:11,212 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] security.TimelineACLsManager (TimelineACLsManager.java:checkAccess(106)) - Verifying the access of ubuntu on the timeline entity { id: container_1448220478671_0001_01_000002, type: DS_CONTAINER } 2015-11-22 19:28:11,213 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - RESPONSE /ws/v1/timeline/ 200 2015-11-22 19:28:11,224 DEBUG [IPC Server listener on 50663] ipc.Server (Server.java:register(2883)) - Server connection from 172.31.20.42:35359; # active connections: 2; # queued calls: 0 2015-11-22 19:28:11,225 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processOneRpc(1897)) - got #-33 2015-11-22 19:28:11,225 DEBUG [Socket Reader #1 for port 50663] security.SaslRpcServer (SaslRpcServer.java:create(175)) - Created SASL server with mechanism = DIGEST-MD5 2015-11-22 19:28:11,225 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processResponse(1099)) - Socket Reader #1 for port 50663: responding to null from 172.31.20.42:35359 Call#-33 Retry#-1 2015-11-22 19:28:11,225 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processResponse(1118)) - Socket Reader #1 for port 50663: responding to null from 172.31.20.42:35359 Call#-33 Retry#-1 Wrote 166 bytes. 2015-11-22 19:28:11,229 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processOneRpc(1897)) - got #-33 2015-11-22 19:28:11,229 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processSaslToken(1529)) - Have read input token of size 350 for processing by saslServer.evaluateResponse() 2015-11-22 19:28:11,229 DEBUG [Socket Reader #1 for port 50663] security.BaseNMTokenSecretManager (BaseNMTokenSecretManager.java:retrivePasswordInternal(103)) - creating password for appattempt_1448220478671_0001_000001 for user ubuntu to run on NM ip-172-31-20-42.ap-northeast-1.compute.internal:50663 2015-11-22 19:28:11,230 DEBUG [Socket Reader #1 for port 50663] security.NMTokenIdentifier (NMTokenIdentifier.java:write(96)) - Writing NMTokenIdentifier to RPC layer: appAttemptId { application_id { id: 1 cluster_timestamp: 1448220478671 } attemptId: 1 } nodeId { host: "ip-172-31-20-42.ap-northeast-1.compute.internal" port: 50663 } appSubmitter: "ubuntu" keyId: 744666633 2015-11-22 19:28:11,230 DEBUG [Socket Reader #1 for port 50663] security.NMTokenSecretManagerInNM (NMTokenSecretManagerInNM.java:retrievePassword(180)) - NMToken password retrieved successfully!! 2015-11-22 19:28:11,230 DEBUG [Socket Reader #1 for port 50663] security.SaslRpcServer (SaslRpcServer.java:handle(310)) - SASL server DIGEST-MD5 callback: setting password for client: appattempt_1448220478671_0001_000001 (auth:SIMPLE) 2015-11-22 19:28:11,230 DEBUG [Socket Reader #1 for port 50663] security.SaslRpcServer (SaslRpcServer.java:handle(328)) - SASL server DIGEST-MD5 callback: setting canonicalized client ID: appattempt_1448220478671_0001_000001 2015-11-22 19:28:11,230 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:buildSaslResponse(1546)) - Will send SUCCESS token of size 40 from saslServer. 2015-11-22 19:28:11,230 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:saslProcess(1434)) - SASL server context established. Negotiated QoP is auth 2015-11-22 19:28:11,230 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:saslProcess(1439)) - SASL server successfully authenticated client: appattempt_1448220478671_0001_000001 (auth:SIMPLE) 2015-11-22 19:28:11,230 INFO [Socket Reader #1 for port 50663] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220478671_0001_000001 (auth:SIMPLE) 2015-11-22 19:28:11,231 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processResponse(1099)) - Socket Reader #1 for port 50663: responding to null from 172.31.20.42:35359 Call#-33 Retry#-1 2015-11-22 19:28:11,231 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processResponse(1118)) - Socket Reader #1 for port 50663: responding to null from 172.31.20.42:35359 Call#-33 Retry#-1 Wrote 64 bytes. 2015-11-22 19:28:11,234 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processOneRpc(1897)) - got #-3 2015-11-22 19:28:11,234 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:authorizeConnection(2097)) - Successfully authorized userInfo { } protocol: "org.apache.hadoop.yarn.api.ContainerManagementProtocolPB" 2015-11-22 19:28:11,234 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processOneRpc(1897)) - got #9 2015-11-22 19:28:11,235 DEBUG [IPC Server handler 3 on 50663] ipc.Server (Server.java:run(2185)) - IPC Server handler 3 on 50663: org.apache.hadoop.yarn.api.ContainerManagementProtocolPB.startContainers from 172.31.20.42:35359 Call#9 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:11,235 DEBUG [IPC Server handler 3 on 50663] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:11,235 DEBUG [IPC Server handler 3 on 50663] security.BaseContainerTokenSecretManager (BaseContainerTokenSecretManager.java:retrievePasswordInternal(118)) - Retrieving password for container_1448220478671_0001_01_000004 for user container_1448220478671_0001_01_000004 (auth:SIMPLE) to be run on NM ip-172-31-20-42.ap-northeast-1.compute.internal:50663 2015-11-22 19:28:11,236 DEBUG [IPC Server handler 3 on 50663] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(197)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 1 cluster_timestamp: 1448220478671 } attemptId: 1 } id: 4 } nmHostAddr: "ip-172-31-20-42.ap-northeast-1.compute.internal:50663" appSubmitter: "ubuntu" resource { memory: 128 virtual_cores: 1 } expiryTimeStamp: 1448221091136 masterKeyId: -550760804 rmIdentifier: 1448220478671 priority { priority: 0 } creationTime: 1448220490310 nodeLabelExpression: "" containerType: TASK 2015-11-22 19:28:11,236 INFO [IPC Server handler 3 on 50663] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:startContainerInternal(889)) - Start request for container_1448220478671_0001_01_000004 by user ubuntu 2015-11-22 19:28:11,236 DEBUG [IPC Server handler 3 on 50663] utils.YarnServerSecurityUtils (YarnServerSecurityUtils.java:parseCredentials(135)) - = Kind: YARN_AM_RM_TOKEN, Service: , Ident: (appAttemptId { application_id { id: 1 cluster_timestamp: 1448220478671 } attemptId: 1 } keyId: -1778540031) 2015-11-22 19:28:11,237 INFO [IPC Server handler 3 on 50663] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu IP=172.31.20.42 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1448220478671_0001 CONTAINERID=container_1448220478671_0001_01_000004 2015-11-22 19:28:11,237 DEBUG [IPC Server handler 3 on 50663] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: startContainers queueTime= 1 procesingTime= 2 2015-11-22 19:28:11,237 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationContainerInitEvent.EventType: INIT_CONTAINER 2015-11-22 19:28:11,238 DEBUG [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(452)) - Processing application_1448220478671_0001 of type INIT_CONTAINER 2015-11-22 19:28:11,238 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(303)) - Adding container_1448220478671_0001_01_000004 to application application_1448220478671_0001 2015-11-22 19:28:11,238 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerInitEvent.EventType: INIT_CONTAINER 2015-11-22 19:28:11,238 DEBUG [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1131)) - Processing container_1448220478671_0001_01_000004 of type INIT_CONTAINER 2015-11-22 19:28:11,238 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220478671_0001_01_000004 transitioned from NEW to LOCALIZED 2015-11-22 19:28:11,238 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServicesEvent.EventType: CONTAINER_INIT 2015-11-22 19:28:11,238 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_INIT for appId application_1448220478671_0001 2015-11-22 19:28:11,238 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncherEvent.EventType: LAUNCH_CONTAINER 2015-11-22 19:28:11,239 DEBUG [ContainersLauncher #1] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.fs.FileContext.getAbstractFileSystem(FileContext.java:339) 2015-11-22 19:28:11,237 DEBUG [IPC Server handler 3 on 50663] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 3 on 50663: responding to org.apache.hadoop.yarn.api.ContainerManagementProtocolPB.startContainers from 172.31.20.42:35359 Call#9 Retry#0 2015-11-22 19:28:11,240 DEBUG [IPC Server handler 3 on 50663] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 3 on 50663: responding to org.apache.hadoop.yarn.api.ContainerManagementProtocolPB.startContainers from 172.31.20.42:35359 Call#9 Retry#0 Wrote 51 bytes. 2015-11-22 19:28:11,241 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:close(2894)) - Socket Reader #1 for port 50663: disconnecting client 172.31.20.42:35359. Number of active connections: 1 2015-11-22 19:28:11,255 DEBUG [IPC Server listener on 50663] ipc.Server (Server.java:register(2883)) - Server connection from 172.31.20.42:35360; # active connections: 2; # queued calls: 0 2015-11-22 19:28:11,256 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processOneRpc(1897)) - got #-33 2015-11-22 19:28:11,256 DEBUG [Socket Reader #1 for port 50663] security.SaslRpcServer (SaslRpcServer.java:create(175)) - Created SASL server with mechanism = DIGEST-MD5 2015-11-22 19:28:11,256 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processResponse(1099)) - Socket Reader #1 for port 50663: responding to null from 172.31.20.42:35360 Call#-33 Retry#-1 2015-11-22 19:28:11,256 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processResponse(1118)) - Socket Reader #1 for port 50663: responding to null from 172.31.20.42:35360 Call#-33 Retry#-1 Wrote 166 bytes. 2015-11-22 19:28:11,259 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processOneRpc(1897)) - got #-33 2015-11-22 19:28:11,259 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processSaslToken(1529)) - Have read input token of size 350 for processing by saslServer.evaluateResponse() 2015-11-22 19:28:11,260 DEBUG [Socket Reader #1 for port 50663] security.BaseNMTokenSecretManager (BaseNMTokenSecretManager.java:retrivePasswordInternal(103)) - creating password for appattempt_1448220478671_0001_000001 for user ubuntu to run on NM ip-172-31-20-42.ap-northeast-1.compute.internal:50663 2015-11-22 19:28:11,260 DEBUG [Socket Reader #1 for port 50663] security.NMTokenIdentifier (NMTokenIdentifier.java:write(96)) - Writing NMTokenIdentifier to RPC layer: appAttemptId { application_id { id: 1 cluster_timestamp: 1448220478671 } attemptId: 1 } nodeId { host: "ip-172-31-20-42.ap-northeast-1.compute.internal" port: 50663 } appSubmitter: "ubuntu" keyId: 744666633 2015-11-22 19:28:11,260 DEBUG [Socket Reader #1 for port 50663] security.NMTokenSecretManagerInNM (NMTokenSecretManagerInNM.java:retrievePassword(180)) - NMToken password retrieved successfully!! 2015-11-22 19:28:11,260 DEBUG [Socket Reader #1 for port 50663] security.SaslRpcServer (SaslRpcServer.java:handle(310)) - SASL server DIGEST-MD5 callback: setting password for client: appattempt_1448220478671_0001_000001 (auth:SIMPLE) 2015-11-22 19:28:11,260 DEBUG [Socket Reader #1 for port 50663] security.SaslRpcServer (SaslRpcServer.java:handle(328)) - SASL server DIGEST-MD5 callback: setting canonicalized client ID: appattempt_1448220478671_0001_000001 2015-11-22 19:28:11,261 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:buildSaslResponse(1546)) - Will send SUCCESS token of size 40 from saslServer. 2015-11-22 19:28:11,261 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:saslProcess(1434)) - SASL server context established. Negotiated QoP is auth 2015-11-22 19:28:11,261 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:saslProcess(1439)) - SASL server successfully authenticated client: appattempt_1448220478671_0001_000001 (auth:SIMPLE) 2015-11-22 19:28:11,261 INFO [Socket Reader #1 for port 50663] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220478671_0001_000001 (auth:SIMPLE) 2015-11-22 19:28:11,261 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processResponse(1099)) - Socket Reader #1 for port 50663: responding to null from 172.31.20.42:35360 Call#-33 Retry#-1 2015-11-22 19:28:11,261 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processResponse(1118)) - Socket Reader #1 for port 50663: responding to null from 172.31.20.42:35360 Call#-33 Retry#-1 Wrote 64 bytes. 2015-11-22 19:28:11,263 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - REQUEST /ws/v1/timeline/ on org.mortbay.jetty.HttpConnection@7248fad3 2015-11-22 19:28:11,263 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - sessionManager=org.mortbay.jetty.servlet.HashSessionManager@3966697d 2015-11-22 19:28:11,263 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - session=null 2015-11-22 19:28:11,263 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - servlet=org.mortbay.jetty.servlet.DefaultServlet-55176785 2015-11-22 19:28:11,263 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - chain=NoCacheFilter->NoCacheFilter->safety->Timeline Authentication Filter->RMAuthenticationFilter->static_user_filter->guice->org.mortbay.jetty.servlet.DefaultServlet-55176785 2015-11-22 19:28:11,263 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - servlet holder=org.mortbay.jetty.servlet.DefaultServlet-55176785 2015-11-22 19:28:11,263 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter NoCacheFilter 2015-11-22 19:28:11,263 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter NoCacheFilter 2015-11-22 19:28:11,263 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter safety 2015-11-22 19:28:11,264 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter Timeline Authentication Filter 2015-11-22 19:28:11,264 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(557)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:41381/ws/v1/timeline/?user.name=ubuntu] triggering authentication 2015-11-22 19:28:11,264 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(576)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:41381/ws/v1/timeline/?user.name=ubuntu] user [ubuntu] authenticated 2015-11-22 19:28:11,264 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter RMAuthenticationFilter 2015-11-22 19:28:11,264 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(557)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:41381/ws/v1/timeline/?user.name=ubuntu] triggering authentication 2015-11-22 19:28:11,264 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(576)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:41381/ws/v1/timeline/?user.name=ubuntu] user [ubuntu] authenticated 2015-11-22 19:28:11,265 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter static_user_filter 2015-11-22 19:28:11,265 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter guice 2015-11-22 19:28:11,265 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerEvent.EventType: CONTAINER_LAUNCHED 2015-11-22 19:28:11,265 DEBUG [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1131)) - Processing container_1448220478671_0001_01_000004 of type CONTAINER_LAUNCHED 2015-11-22 19:28:11,265 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220478671_0001_01_000004 transitioned from LOCALIZED to RUNNING 2015-11-22 19:28:11,265 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerStartMonitoringEvent.EventType: START_MONITORING_CONTAINER 2015-11-22 19:28:11,268 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - getResource(META-INF/services/javax.xml.bind.JAXBContext)=jar:file:/home/ubuntu/.m2/repository/com/sun/xml/bind/jaxb-impl/2.2.3-1/jaxb-impl-2.2.3-1.jar!/META-INF/services/javax.xml.bind.JAXBContext 2015-11-22 19:28:11,269 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.xml.bind.v2.ContextFactory from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:28:11,276 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processOneRpc(1897)) - got #-3 2015-11-22 19:28:11,276 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:authorizeConnection(2097)) - Successfully authorized userInfo { } protocol: "org.apache.hadoop.yarn.api.ContainerManagementProtocolPB" 2015-11-22 19:28:11,276 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processOneRpc(1897)) - got #10 2015-11-22 19:28:11,276 DEBUG [IPC Server handler 1 on 50663] ipc.Server (Server.java:run(2185)) - IPC Server handler 1 on 50663: org.apache.hadoop.yarn.api.ContainerManagementProtocolPB.getContainerStatuses from 172.31.20.42:35360 Call#10 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:11,277 DEBUG [IPC Server handler 1 on 50663] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:11,277 INFO [IPC Server handler 1 on 50663] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:getContainerStatusInternal(1221)) - Getting container-status for container_1448220478671_0001_01_000004 2015-11-22 19:28:11,277 INFO [IPC Server handler 1 on 50663] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:getContainerStatusInternal(1235)) - Returning ContainerStatus: [ContainerId: container_1448220478671_0001_01_000004, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ] 2015-11-22 19:28:11,277 DEBUG [IPC Server handler 1 on 50663] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getContainerStatuses queueTime= 1 procesingTime= 0 2015-11-22 19:28:11,278 DEBUG [IPC Server handler 1 on 50663] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 1 on 50663: responding to org.apache.hadoop.yarn.api.ContainerManagementProtocolPB.getContainerStatuses from 172.31.20.42:35360 Call#10 Retry#0 2015-11-22 19:28:11,278 DEBUG [IPC Server handler 1 on 50663] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 1 on 50663: responding to org.apache.hadoop.yarn.api.ContainerManagementProtocolPB.getContainerStatuses from 172.31.20.42:35360 Call#10 Retry#0 Wrote 75 bytes. 2015-11-22 19:28:11,280 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:close(2894)) - Socket Reader #1 for port 50663: disconnecting client 172.31.20.42:35360. Number of active connections: 1 2015-11-22 19:28:11,293 INFO [ContainersLauncher #1] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:buildCommandExecutor(281)) - launchContainer: [nice, -n, 0, bash, /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220478671_0001/container_1448220478671_0001_01_000004/default_container_executor.sh] 2015-11-22 19:28:11,301 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - RESPONSE /ws/v1/timeline/ 200 2015-11-22 19:28:11,302 DEBUG [ContainersLauncher #1] launcher.ContainerLaunch (ContainerLaunch.java:call(333)) - Container container_1448220478671_0001_01_000004 completed with exit code 0 2015-11-22 19:28:11,302 INFO [ContainersLauncher #1] launcher.ContainerLaunch (ContainerLaunch.java:call(356)) - Container container_1448220478671_0001_01_000004 succeeded 2015-11-22 19:28:11,302 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerEvent.EventType: CONTAINER_EXITED_WITH_SUCCESS 2015-11-22 19:28:11,302 DEBUG [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1131)) - Processing container_1448220478671_0001_01_000004 of type CONTAINER_EXITED_WITH_SUCCESS 2015-11-22 19:28:11,302 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220478671_0001_01_000004 transitioned from RUNNING to EXITED_WITH_SUCCESS 2015-11-22 19:28:11,302 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncherEvent.EventType: CLEANUP_CONTAINER 2015-11-22 19:28:11,302 INFO [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(380)) - Cleaning up container container_1448220478671_0001_01_000004 2015-11-22 19:28:11,302 DEBUG [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(397)) - Marking container container_1448220478671_0001_01_000004 as inactive 2015-11-22 19:28:11,302 DEBUG [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(403)) - Getting pid for container container_1448220478671_0001_01_000004 to kill from pid file /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/application_1448220478671_0001/container_1448220478671_0001_01_000004/container_1448220478671_0001_01_000004.pid 2015-11-22 19:28:11,302 DEBUG [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:getContainerPid(569)) - Accessing pid for container container_1448220478671_0001_01_000004 from pid file /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/application_1448220478671_0001/container_1448220478671_0001_01_000004/container_1448220478671_0001_01_000004.pid 2015-11-22 19:28:11,302 DEBUG [AsyncDispatcher event handler] util.ProcessIdFileReader (ProcessIdFileReader.java:getProcessId(51)) - Accessing pid from pid file /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/application_1448220478671_0001/container_1448220478671_0001_01_000004/container_1448220478671_0001_01_000004.pid 2015-11-22 19:28:11,303 DEBUG [AsyncDispatcher event handler] util.ProcessIdFileReader (ProcessIdFileReader.java:getProcessId(99)) - Got pid 30192 from path /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/application_1448220478671_0001/container_1448220478671_0001_01_000004/container_1448220478671_0001_01_000004.pid 2015-11-22 19:28:11,303 DEBUG [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:getContainerPid(579)) - Got pid 30192 for container container_1448220478671_0001_01_000004 2015-11-22 19:28:11,303 DEBUG [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(421)) - Sending signal to pid 30192 as user ubuntu for container container_1448220478671_0001_01_000004 2015-11-22 19:28:11,303 DEBUG [AsyncDispatcher event handler] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:signalContainer(417)) - Sending signal 15 to pid 30192 as user ubuntu 2015-11-22 19:28:11,303 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - REQUEST /ws/v1/timeline/ on org.mortbay.jetty.HttpConnection@5ecc9c8f 2015-11-22 19:28:11,304 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - sessionManager=org.mortbay.jetty.servlet.HashSessionManager@3966697d 2015-11-22 19:28:11,304 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - session=null 2015-11-22 19:28:11,304 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - servlet=org.mortbay.jetty.servlet.DefaultServlet-55176785 2015-11-22 19:28:11,304 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - chain=NoCacheFilter->NoCacheFilter->safety->Timeline Authentication Filter->RMAuthenticationFilter->static_user_filter->guice->org.mortbay.jetty.servlet.DefaultServlet-55176785 2015-11-22 19:28:11,304 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - servlet holder=org.mortbay.jetty.servlet.DefaultServlet-55176785 2015-11-22 19:28:11,304 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter NoCacheFilter 2015-11-22 19:28:11,305 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter NoCacheFilter 2015-11-22 19:28:11,305 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter safety 2015-11-22 19:28:11,305 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter Timeline Authentication Filter 2015-11-22 19:28:11,305 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:28:11,305 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(576)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:41381/ws/v1/timeline/] user [ubuntu] authenticated 2015-11-22 19:28:11,305 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 2 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ], ContainerStatus: [ContainerId: container_1448220478671_0001_01_000004, State: RUNNING, Capability: , Diagnostics: , ExitStatus: 0, ]] 2015-11-22 19:28:11,305 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter RMAuthenticationFilter 2015-11-22 19:28:11,306 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:11,306 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:28:11,306 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:28:11,306 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:28:11,306 DEBUG [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(424)) - Processing container_1448220478671_0001_01_000004 of type LAUNCHED 2015-11-22 19:28:11,306 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220478671_0001_01_000004 Container Transitioned from ACQUIRED to RUNNING 2015-11-22 19:28:11,306 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:28:11,306 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:28:11,307 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(408)) - Trying to assign containers to child-queue of root 2015-11-22 19:28:11,307 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:printChildQueues(617)) - printChildQueues - queue: root child-queues: root.defaultusedCapacity=(0.15625), label=(*) 2015-11-22 19:28:11,307 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(564)) - Trying to assign to queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.15625, absoluteUsedCapacity=0.15625, numApps=1, numContainers=2 2015-11-22 19:28:11,307 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:assignContainers(817)) - assignContainers: node=ip-172-31-20-42.ap-northeast-1.compute.internal #applications=1 2015-11-22 19:28:11,307 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2015-11-22 19:28:11,307 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimitAndSetHeadroom(984)) - Headroom calculation for user ubuntu: userLimit= queueMaxAvailRes= consumed= headroom= 2015-11-22 19:28:11,307 DEBUG [ResourceManager Event Processor] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:assignContainers(477)) - pre-assignContainers for application application_1448220478671_0001 2015-11-22 19:28:11,307 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2015-11-22 19:28:11,307 DEBUG [ResourceManager Event Processor] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(470)) - showRequests: application=application_1448220478671_0001 headRoom= currentConsumption=640 2015-11-22 19:28:11,307 DEBUG [ResourceManager Event Processor] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(474)) - showRequests: application=application_1448220478671_0001 request={Priority: 0, Capability: , # Containers: 1, Location: *, Relax Locality: true, Node Label Expression: } 2015-11-22 19:28:11,307 DEBUG [ResourceManager Event Processor] allocator.IncreaseContainerAllocator (IncreaseContainerAllocator.java:assignContainers(184)) - Skip allocating increase request since we don't have any increase request on this node=ip-172-31-20-42.ap-northeast-1.compute.internal:50663 2015-11-22 19:28:11,308 DEBUG [ResourceManager Event Processor] allocator.RegularContainerAllocator (RegularContainerAllocator.java:assignContainer(407)) - assignContainers: node=ip-172-31-20-42.ap-northeast-1.compute.internal application=application_1448220478671_0001 priority=0 request={Priority: 0, Capability: , # Containers: 1, Location: *, Relax Locality: true, Node Label Expression: } type=OFF_SWITCH 2015-11-22 19:28:11,308 DEBUG [ResourceManager Event Processor] scheduler.ActiveUsersManager (ActiveUsersManager.java:deactivateApplication(94)) - User ubuntu removed from activeUsers, currently: 0 2015-11-22 19:28:11,308 DEBUG [ResourceManager Event Processor] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:allocate(594)) - allocate: applicationId=application_1448220478671_0001 container=container_1448220478671_0001_01_000005 host=ip-172-31-20-42.ap-northeast-1.compute.internal:50663 user=ubuntu resource= type=OFF_SWITCH 2015-11-22 19:28:11,308 DEBUG [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(424)) - Processing container_1448220478671_0001_01_000005 of type START 2015-11-22 19:28:11,308 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220478671_0001_01_000005 Container Transitioned from NEW to ALLOCATED 2015-11-22 19:28:11,308 DEBUG [ResourceManager Event Processor] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:allocate(208)) - allocate: applicationAttemptId=appattempt_1448220478671_0001_000001 container=container_1448220478671_0001_01_000005 host=ip-172-31-20-42.ap-northeast-1.compute.internal type=OFF_SWITCH 2015-11-22 19:28:11,308 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220478671_0001 CONTAINERID=container_1448220478671_0001_01_000005 2015-11-22 19:28:11,308 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(154)) - Assigned container container_1448220478671_0001_01_000005 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:50663, which has 3 containers, used and available after allocation 2015-11-22 19:28:11,308 INFO [ResourceManager Event Processor] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(89)) - assignedContainer application attempt=appattempt_1448220478671_0001_000001 container=container_1448220478671_0001_01_000005 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@33e07bad clusterResource= type=OFF_SWITCH 2015-11-22 19:28:11,308 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:assignContainers(881)) - post-assignContainers for application application_1448220478671_0001 2015-11-22 19:28:11,309 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:11,309 DEBUG [ResourceManager Event Processor] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(470)) - showRequests: application=application_1448220478671_0001 headRoom= currentConsumption=768 2015-11-22 19:28:11,309 DEBUG [ResourceManager Event Processor] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(474)) - showRequests: application=application_1448220478671_0001 request={Priority: 0, Capability: , # Containers: 0, Location: *, Relax Locality: true, Node Label Expression: } 2015-11-22 19:28:11,309 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:11,309 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:11,309 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:11,309 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:allocateResource(1277)) - default user=ubuntu used= numContainers=3 headroom = user-resources= 2015-11-22 19:28:11,309 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(575)) - Assigned to queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.1875, absoluteUsedCapacity=0.1875, numApps=1, numContainers=3 --> , OFF_SWITCH 2015-11-22 19:28:11,309 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(589)) - Re-sorting assigned queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.1875, absoluteUsedCapacity=0.1875, numApps=1, numContainers=3 2015-11-22 19:28:11,309 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:printChildQueues(617)) - printChildQueues - queue: root child-queues: root.defaultusedCapacity=(0.1875), label=(*) 2015-11-22 19:28:11,309 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(461)) - assignedContainer queue=root usedCapacity=0.1875 absoluteUsedCapacity=0.1875 used= cluster= 2015-11-22 19:28:11,309 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(473)) - ParentQ=root assignedSoFarInThisIteration= usedCapacity=0.1875 absoluteUsedCapacity=0.1875 2015-11-22 19:28:11,309 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(484)) - Not assigning more than one off-switch container, assignments so far: resource:; type:OFF_SWITCH; excessReservation:null; applicationid:null; skipped:false; fulfilled reservation:false; allocations(count/resource):1/; reservations(count/resource):0/ 2015-11-22 19:28:11,306 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(576)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:41381/ws/v1/timeline/] user [ubuntu] authenticated 2015-11-22 19:28:11,310 DEBUG [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(437)) - Sent signal SIGTERM to pid 30192 as user ubuntu for container container_1448220478671_0001_01_000004, result=failed 2015-11-22 19:28:11,308 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: CONTAINER_ALLOCATED 2015-11-22 19:28:11,316 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type CONTAINER_ALLOCATED 2015-11-22 19:28:11,316 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter static_user_filter 2015-11-22 19:28:11,316 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter guice 2015-11-22 19:28:11,316 DEBUG [AsyncDispatcher event handler] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.fs.FileContext.getAbstractFileSystem(FileContext.java:339) 2015-11-22 19:28:11,317 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] security.TimelineACLsManager (TimelineACLsManager.java:checkAccess(106)) - Verifying the access of ubuntu on the timeline entity { id: container_1448220478671_0001_01_000004, type: DS_CONTAINER } 2015-11-22 19:28:11,320 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - RESPONSE /ws/v1/timeline/ 200 2015-11-22 19:28:11,333 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.ContainerLocalizationCleanupEvent.EventType: CLEANUP_CONTAINER_RESOURCES 2015-11-22 19:28:11,333 DEBUG [DeletionService #2] nodemanager.DeletionService (DeletionService.java:run(266)) - FileDeletionTask : user : ubuntu subDir : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220478671_0001/container_1448220478671_0001_01_000004 baseDir : null 2015-11-22 19:28:11,334 DEBUG [DeletionService #1] nodemanager.DeletionService (DeletionService.java:run(266)) - FileDeletionTask : user : null subDir : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/application_1448220478671_0001/container_1448220478671_0001_01_000004 baseDir : null 2015-11-22 19:28:11,334 DEBUG [DeletionService #2] nodemanager.DeletionService (DeletionService.java:run(292)) - Deleting path: [/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220478671_0001/container_1448220478671_0001_01_000004] as user: [ubuntu] 2015-11-22 19:28:11,334 INFO [DeletionService #2] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(480)) - Deleting absolute path : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220478671_0001/container_1448220478671_0001_01_000004 2015-11-22 19:28:11,333 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerEvent.EventType: CONTAINER_RESOURCES_CLEANEDUP 2015-11-22 19:28:11,334 DEBUG [DeletionService #1] nodemanager.DeletionService (DeletionService.java:run(271)) - NM deleting absolute path : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/application_1448220478671_0001/container_1448220478671_0001_01_000004 2015-11-22 19:28:11,334 DEBUG [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1131)) - Processing container_1448220478671_0001_01_000004 of type CONTAINER_RESOURCES_CLEANEDUP 2015-11-22 19:28:11,334 INFO [AsyncDispatcher event handler] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu OPERATION=Container Finished - Succeeded TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1448220478671_0001 CONTAINERID=container_1448220478671_0001_01_000004 2015-11-22 19:28:11,334 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220478671_0001_01_000004 transitioned from EXITED_WITH_SUCCESS to DONE 2015-11-22 19:28:11,335 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationContainerFinishedEvent.EventType: APPLICATION_CONTAINER_FINISHED 2015-11-22 19:28:11,335 DEBUG [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(452)) - Processing application_1448220478671_0001 of type APPLICATION_CONTAINER_FINISHED 2015-11-22 19:28:11,335 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(346)) - Removing container_1448220478671_0001_01_000004 from application application_1448220478671_0001 2015-11-22 19:28:11,335 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerStopMonitoringEvent.EventType: STOP_MONITORING_CONTAINER 2015-11-22 19:28:11,335 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.event.LogHandlerContainerFinishedEvent.EventType: CONTAINER_FINISHED 2015-11-22 19:28:11,335 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServicesEvent.EventType: CONTAINER_STOP 2015-11-22 19:28:11,335 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_STOP for appId application_1448220478671_0001 2015-11-22 19:28:11,570 DEBUG [Task killer for 30192] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:signalContainer(417)) - Sending signal 9 to pid 30192 as user ubuntu 2015-11-22 19:28:11,809 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #832 2015-11-22 19:28:11,809 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #832 2015-11-22 19:28:11,810 DEBUG [IPC Server handler 16 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 16 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#832 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:11,810 DEBUG [IPC Server handler 16 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:11,810 DEBUG [IPC Server handler 16 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:28:11,810 DEBUG [IPC Server handler 16 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 1 procesingTime= 0 2015-11-22 19:28:11,810 DEBUG [IPC Server handler 16 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 16 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#832 Retry#0 2015-11-22 19:28:11,810 DEBUG [IPC Server handler 16 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 16 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#832 Retry#0 Wrote 359 bytes. 2015-11-22 19:28:11,811 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #832 2015-11-22 19:28:11,811 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:28:11,811 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:28:12,139 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #11 2015-11-22 19:28:12,139 DEBUG [IPC Server handler 7 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 7 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#11 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:12,140 DEBUG [IPC Server handler 7 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:12,140 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:12,140 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:28:12,141 DEBUG [IPC Server handler 7 on 46284] security.BaseContainerTokenSecretManager (BaseContainerTokenSecretManager.java:createPassword(90)) - Creating password for container_1448220478671_0001_01_000005 for user container_1448220478671_0001_01_000005 (auth:SIMPLE) to be run on NM ip-172-31-20-42.ap-northeast-1.compute.internal:50663 2015-11-22 19:28:12,141 DEBUG [IPC Server handler 7 on 46284] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(197)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 1 cluster_timestamp: 1448220478671 } attemptId: 1 } id: 5 } nmHostAddr: "ip-172-31-20-42.ap-northeast-1.compute.internal:50663" appSubmitter: "ubuntu" resource { memory: 128 virtual_cores: 1 } expiryTimeStamp: 1448221092140 masterKeyId: -550760804 rmIdentifier: 1448220478671 priority { priority: 0 } creationTime: 1448220491308 nodeLabelExpression: "" containerType: TASK 2015-11-22 19:28:12,141 DEBUG [IPC Server handler 7 on 46284] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(197)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 1 cluster_timestamp: 1448220478671 } attemptId: 1 } id: 5 } nmHostAddr: "ip-172-31-20-42.ap-northeast-1.compute.internal:50663" appSubmitter: "ubuntu" resource { memory: 128 virtual_cores: 1 } expiryTimeStamp: 1448221092140 masterKeyId: -550760804 rmIdentifier: 1448220478671 priority { priority: 0 } creationTime: 1448220491308 nodeLabelExpression: "" containerType: TASK 2015-11-22 19:28:12,141 DEBUG [IPC Server handler 7 on 46284] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(424)) - Processing container_1448220478671_0001_01_000005 of type ACQUIRED 2015-11-22 19:28:12,141 INFO [IPC Server handler 7 on 46284] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220478671_0001_01_000005 Container Transitioned from ALLOCATED to ACQUIRED 2015-11-22 19:28:12,141 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppRunningOnNodeEvent.EventType: APP_RUNNING_ON_NODE 2015-11-22 19:28:12,142 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(777)) - Processing event for application_1448220478671_0001 of type APP_RUNNING_ON_NODE 2015-11-22 19:28:12,142 DEBUG [IPC Server handler 7 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:12,142 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:12,142 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:12,142 DEBUG [IPC Server handler 7 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 1 procesingTime= 2 2015-11-22 19:28:12,143 DEBUG [IPC Server handler 7 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 7 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#11 Retry#0 2015-11-22 19:28:12,143 DEBUG [IPC Server handler 7 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 7 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#11 Retry#0 Wrote 384 bytes. 2015-11-22 19:28:12,148 DEBUG [IPC Server listener on 50663] ipc.Server (Server.java:register(2883)) - Server connection from 172.31.20.42:35362; # active connections: 2; # queued calls: 0 2015-11-22 19:28:12,148 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processOneRpc(1897)) - got #-33 2015-11-22 19:28:12,148 DEBUG [Socket Reader #1 for port 50663] security.SaslRpcServer (SaslRpcServer.java:create(175)) - Created SASL server with mechanism = DIGEST-MD5 2015-11-22 19:28:12,149 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processResponse(1099)) - Socket Reader #1 for port 50663: responding to null from 172.31.20.42:35362 Call#-33 Retry#-1 2015-11-22 19:28:12,149 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processResponse(1118)) - Socket Reader #1 for port 50663: responding to null from 172.31.20.42:35362 Call#-33 Retry#-1 Wrote 166 bytes. 2015-11-22 19:28:12,152 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processOneRpc(1897)) - got #-33 2015-11-22 19:28:12,153 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processSaslToken(1529)) - Have read input token of size 350 for processing by saslServer.evaluateResponse() 2015-11-22 19:28:12,153 DEBUG [Socket Reader #1 for port 50663] security.BaseNMTokenSecretManager (BaseNMTokenSecretManager.java:retrivePasswordInternal(103)) - creating password for appattempt_1448220478671_0001_000001 for user ubuntu to run on NM ip-172-31-20-42.ap-northeast-1.compute.internal:50663 2015-11-22 19:28:12,153 DEBUG [Socket Reader #1 for port 50663] security.NMTokenIdentifier (NMTokenIdentifier.java:write(96)) - Writing NMTokenIdentifier to RPC layer: appAttemptId { application_id { id: 1 cluster_timestamp: 1448220478671 } attemptId: 1 } nodeId { host: "ip-172-31-20-42.ap-northeast-1.compute.internal" port: 50663 } appSubmitter: "ubuntu" keyId: 744666633 2015-11-22 19:28:12,153 DEBUG [Socket Reader #1 for port 50663] security.NMTokenSecretManagerInNM (NMTokenSecretManagerInNM.java:retrievePassword(180)) - NMToken password retrieved successfully!! 2015-11-22 19:28:12,153 DEBUG [Socket Reader #1 for port 50663] security.SaslRpcServer (SaslRpcServer.java:handle(310)) - SASL server DIGEST-MD5 callback: setting password for client: appattempt_1448220478671_0001_000001 (auth:SIMPLE) 2015-11-22 19:28:12,154 DEBUG [Socket Reader #1 for port 50663] security.SaslRpcServer (SaslRpcServer.java:handle(328)) - SASL server DIGEST-MD5 callback: setting canonicalized client ID: appattempt_1448220478671_0001_000001 2015-11-22 19:28:12,154 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:buildSaslResponse(1546)) - Will send SUCCESS token of size 40 from saslServer. 2015-11-22 19:28:12,154 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:saslProcess(1434)) - SASL server context established. Negotiated QoP is auth 2015-11-22 19:28:12,154 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:saslProcess(1439)) - SASL server successfully authenticated client: appattempt_1448220478671_0001_000001 (auth:SIMPLE) 2015-11-22 19:28:12,154 INFO [Socket Reader #1 for port 50663] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220478671_0001_000001 (auth:SIMPLE) 2015-11-22 19:28:12,154 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processResponse(1099)) - Socket Reader #1 for port 50663: responding to null from 172.31.20.42:35362 Call#-33 Retry#-1 2015-11-22 19:28:12,154 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processResponse(1118)) - Socket Reader #1 for port 50663: responding to null from 172.31.20.42:35362 Call#-33 Retry#-1 Wrote 64 bytes. 2015-11-22 19:28:12,156 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processOneRpc(1897)) - got #-3 2015-11-22 19:28:12,157 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:authorizeConnection(2097)) - Successfully authorized userInfo { } protocol: "org.apache.hadoop.yarn.api.ContainerManagementProtocolPB" 2015-11-22 19:28:12,157 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processOneRpc(1897)) - got #12 2015-11-22 19:28:12,157 DEBUG [IPC Server handler 0 on 50663] ipc.Server (Server.java:run(2185)) - IPC Server handler 0 on 50663: org.apache.hadoop.yarn.api.ContainerManagementProtocolPB.startContainers from 172.31.20.42:35362 Call#12 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:12,157 DEBUG [IPC Server handler 0 on 50663] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:12,158 DEBUG [IPC Server handler 0 on 50663] security.BaseContainerTokenSecretManager (BaseContainerTokenSecretManager.java:retrievePasswordInternal(118)) - Retrieving password for container_1448220478671_0001_01_000005 for user container_1448220478671_0001_01_000005 (auth:SIMPLE) to be run on NM ip-172-31-20-42.ap-northeast-1.compute.internal:50663 2015-11-22 19:28:12,158 DEBUG [IPC Server handler 0 on 50663] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(197)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 1 cluster_timestamp: 1448220478671 } attemptId: 1 } id: 5 } nmHostAddr: "ip-172-31-20-42.ap-northeast-1.compute.internal:50663" appSubmitter: "ubuntu" resource { memory: 128 virtual_cores: 1 } expiryTimeStamp: 1448221092140 masterKeyId: -550760804 rmIdentifier: 1448220478671 priority { priority: 0 } creationTime: 1448220491308 nodeLabelExpression: "" containerType: TASK 2015-11-22 19:28:12,158 INFO [IPC Server handler 0 on 50663] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:startContainerInternal(889)) - Start request for container_1448220478671_0001_01_000005 by user ubuntu 2015-11-22 19:28:12,159 DEBUG [IPC Server handler 0 on 50663] utils.YarnServerSecurityUtils (YarnServerSecurityUtils.java:parseCredentials(135)) - = Kind: YARN_AM_RM_TOKEN, Service: , Ident: (appAttemptId { application_id { id: 1 cluster_timestamp: 1448220478671 } attemptId: 1 } keyId: -1778540031) 2015-11-22 19:28:12,159 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationContainerInitEvent.EventType: INIT_CONTAINER 2015-11-22 19:28:12,159 INFO [IPC Server handler 0 on 50663] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu IP=172.31.20.42 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1448220478671_0001 CONTAINERID=container_1448220478671_0001_01_000005 2015-11-22 19:28:12,159 DEBUG [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(452)) - Processing application_1448220478671_0001 of type INIT_CONTAINER 2015-11-22 19:28:12,160 DEBUG [IPC Server handler 0 on 50663] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: startContainers queueTime= 1 procesingTime= 2 2015-11-22 19:28:12,160 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(303)) - Adding container_1448220478671_0001_01_000005 to application application_1448220478671_0001 2015-11-22 19:28:12,160 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerInitEvent.EventType: INIT_CONTAINER 2015-11-22 19:28:12,160 DEBUG [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1131)) - Processing container_1448220478671_0001_01_000005 of type INIT_CONTAINER 2015-11-22 19:28:12,160 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220478671_0001_01_000005 transitioned from NEW to LOCALIZED 2015-11-22 19:28:12,160 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServicesEvent.EventType: CONTAINER_INIT 2015-11-22 19:28:12,160 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_INIT for appId application_1448220478671_0001 2015-11-22 19:28:12,160 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncherEvent.EventType: LAUNCH_CONTAINER 2015-11-22 19:28:12,161 DEBUG [ContainersLauncher #1] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.fs.FileContext.getAbstractFileSystem(FileContext.java:339) 2015-11-22 19:28:12,161 DEBUG [IPC Server handler 0 on 50663] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 0 on 50663: responding to org.apache.hadoop.yarn.api.ContainerManagementProtocolPB.startContainers from 172.31.20.42:35362 Call#12 Retry#0 2015-11-22 19:28:12,161 DEBUG [IPC Server handler 0 on 50663] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 0 on 50663: responding to org.apache.hadoop.yarn.api.ContainerManagementProtocolPB.startContainers from 172.31.20.42:35362 Call#12 Retry#0 Wrote 51 bytes. 2015-11-22 19:28:12,162 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:close(2894)) - Socket Reader #1 for port 50663: disconnecting client 172.31.20.42:35362. Number of active connections: 1 2015-11-22 19:28:12,172 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - REQUEST /ws/v1/timeline/ on org.mortbay.jetty.HttpConnection@5ecc9c8f 2015-11-22 19:28:12,173 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - sessionManager=org.mortbay.jetty.servlet.HashSessionManager@3966697d 2015-11-22 19:28:12,173 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - session=null 2015-11-22 19:28:12,173 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - servlet=org.mortbay.jetty.servlet.DefaultServlet-55176785 2015-11-22 19:28:12,173 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - chain=NoCacheFilter->NoCacheFilter->safety->Timeline Authentication Filter->RMAuthenticationFilter->static_user_filter->guice->org.mortbay.jetty.servlet.DefaultServlet-55176785 2015-11-22 19:28:12,173 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - servlet holder=org.mortbay.jetty.servlet.DefaultServlet-55176785 2015-11-22 19:28:12,173 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter NoCacheFilter 2015-11-22 19:28:12,174 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter NoCacheFilter 2015-11-22 19:28:12,174 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter safety 2015-11-22 19:28:12,174 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter Timeline Authentication Filter 2015-11-22 19:28:12,174 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(557)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:41381/ws/v1/timeline/?user.name=ubuntu] triggering authentication 2015-11-22 19:28:12,174 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(576)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:41381/ws/v1/timeline/?user.name=ubuntu] user [ubuntu] authenticated 2015-11-22 19:28:12,175 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter RMAuthenticationFilter 2015-11-22 19:28:12,175 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(557)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:41381/ws/v1/timeline/?user.name=ubuntu] triggering authentication 2015-11-22 19:28:12,175 DEBUG [IPC Server listener on 50663] ipc.Server (Server.java:register(2883)) - Server connection from 172.31.20.42:35363; # active connections: 2; # queued calls: 0 2015-11-22 19:28:12,175 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(576)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:41381/ws/v1/timeline/?user.name=ubuntu] user [ubuntu] authenticated 2015-11-22 19:28:12,176 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter static_user_filter 2015-11-22 19:28:12,176 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter guice 2015-11-22 19:28:12,179 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processOneRpc(1897)) - got #-33 2015-11-22 19:28:12,179 DEBUG [Socket Reader #1 for port 50663] security.SaslRpcServer (SaslRpcServer.java:create(175)) - Created SASL server with mechanism = DIGEST-MD5 2015-11-22 19:28:12,179 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processResponse(1099)) - Socket Reader #1 for port 50663: responding to null from 172.31.20.42:35363 Call#-33 Retry#-1 2015-11-22 19:28:12,179 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processResponse(1118)) - Socket Reader #1 for port 50663: responding to null from 172.31.20.42:35363 Call#-33 Retry#-1 Wrote 166 bytes. 2015-11-22 19:28:12,182 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - getResource(META-INF/services/javax.xml.bind.JAXBContext)=jar:file:/home/ubuntu/.m2/repository/com/sun/xml/bind/jaxb-impl/2.2.3-1/jaxb-impl-2.2.3-1.jar!/META-INF/services/javax.xml.bind.JAXBContext 2015-11-22 19:28:12,183 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.xml.bind.v2.ContextFactory from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:28:12,186 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processOneRpc(1897)) - got #-33 2015-11-22 19:28:12,186 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processSaslToken(1529)) - Have read input token of size 350 for processing by saslServer.evaluateResponse() 2015-11-22 19:28:12,187 DEBUG [Socket Reader #1 for port 50663] security.BaseNMTokenSecretManager (BaseNMTokenSecretManager.java:retrivePasswordInternal(103)) - creating password for appattempt_1448220478671_0001_000001 for user ubuntu to run on NM ip-172-31-20-42.ap-northeast-1.compute.internal:50663 2015-11-22 19:28:12,187 DEBUG [Socket Reader #1 for port 50663] security.NMTokenIdentifier (NMTokenIdentifier.java:write(96)) - Writing NMTokenIdentifier to RPC layer: appAttemptId { application_id { id: 1 cluster_timestamp: 1448220478671 } attemptId: 1 } nodeId { host: "ip-172-31-20-42.ap-northeast-1.compute.internal" port: 50663 } appSubmitter: "ubuntu" keyId: 744666633 2015-11-22 19:28:12,187 DEBUG [Socket Reader #1 for port 50663] security.NMTokenSecretManagerInNM (NMTokenSecretManagerInNM.java:retrievePassword(180)) - NMToken password retrieved successfully!! 2015-11-22 19:28:12,187 DEBUG [Socket Reader #1 for port 50663] security.SaslRpcServer (SaslRpcServer.java:handle(310)) - SASL server DIGEST-MD5 callback: setting password for client: appattempt_1448220478671_0001_000001 (auth:SIMPLE) 2015-11-22 19:28:12,188 DEBUG [Socket Reader #1 for port 50663] security.SaslRpcServer (SaslRpcServer.java:handle(328)) - SASL server DIGEST-MD5 callback: setting canonicalized client ID: appattempt_1448220478671_0001_000001 2015-11-22 19:28:12,188 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:buildSaslResponse(1546)) - Will send SUCCESS token of size 40 from saslServer. 2015-11-22 19:28:12,188 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:saslProcess(1434)) - SASL server context established. Negotiated QoP is auth 2015-11-22 19:28:12,188 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:saslProcess(1439)) - SASL server successfully authenticated client: appattempt_1448220478671_0001_000001 (auth:SIMPLE) 2015-11-22 19:28:12,188 INFO [Socket Reader #1 for port 50663] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220478671_0001_000001 (auth:SIMPLE) 2015-11-22 19:28:12,188 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processResponse(1099)) - Socket Reader #1 for port 50663: responding to null from 172.31.20.42:35363 Call#-33 Retry#-1 2015-11-22 19:28:12,188 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processResponse(1118)) - Socket Reader #1 for port 50663: responding to null from 172.31.20.42:35363 Call#-33 Retry#-1 Wrote 64 bytes. 2015-11-22 19:28:12,197 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processOneRpc(1897)) - got #-3 2015-11-22 19:28:12,197 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:authorizeConnection(2097)) - Successfully authorized userInfo { } protocol: "org.apache.hadoop.yarn.api.ContainerManagementProtocolPB" 2015-11-22 19:28:12,197 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:processOneRpc(1897)) - got #13 2015-11-22 19:28:12,197 DEBUG [IPC Server handler 2 on 50663] ipc.Server (Server.java:run(2185)) - IPC Server handler 2 on 50663: org.apache.hadoop.yarn.api.ContainerManagementProtocolPB.getContainerStatuses from 172.31.20.42:35363 Call#13 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:12,204 DEBUG [IPC Server handler 2 on 50663] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:12,205 INFO [IPC Server handler 2 on 50663] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:getContainerStatusInternal(1221)) - Getting container-status for container_1448220478671_0001_01_000005 2015-11-22 19:28:12,205 INFO [IPC Server handler 2 on 50663] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:getContainerStatusInternal(1235)) - Returning ContainerStatus: [ContainerId: container_1448220478671_0001_01_000005, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ] 2015-11-22 19:28:12,205 DEBUG [IPC Server handler 2 on 50663] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getContainerStatuses queueTime= 8 procesingTime= 0 2015-11-22 19:28:12,206 DEBUG [IPC Server handler 2 on 50663] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 2 on 50663: responding to org.apache.hadoop.yarn.api.ContainerManagementProtocolPB.getContainerStatuses from 172.31.20.42:35363 Call#13 Retry#0 2015-11-22 19:28:12,206 DEBUG [IPC Server handler 2 on 50663] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 2 on 50663: responding to org.apache.hadoop.yarn.api.ContainerManagementProtocolPB.getContainerStatuses from 172.31.20.42:35363 Call#13 Retry#0 Wrote 75 bytes. 2015-11-22 19:28:12,207 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:close(2894)) - Socket Reader #1 for port 50663: disconnecting client 172.31.20.42:35363. Number of active connections: 1 2015-11-22 19:28:12,209 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerEvent.EventType: CONTAINER_LAUNCHED 2015-11-22 19:28:12,209 DEBUG [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1131)) - Processing container_1448220478671_0001_01_000005 of type CONTAINER_LAUNCHED 2015-11-22 19:28:12,209 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220478671_0001_01_000005 transitioned from LOCALIZED to RUNNING 2015-11-22 19:28:12,210 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerStartMonitoringEvent.EventType: START_MONITORING_CONTAINER 2015-11-22 19:28:12,222 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - RESPONSE /ws/v1/timeline/ 200 2015-11-22 19:28:12,225 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - REQUEST /ws/v1/timeline/ on org.mortbay.jetty.HttpConnection@369d14ce 2015-11-22 19:28:12,225 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - sessionManager=org.mortbay.jetty.servlet.HashSessionManager@3966697d 2015-11-22 19:28:12,225 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - session=null 2015-11-22 19:28:12,225 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - servlet=org.mortbay.jetty.servlet.DefaultServlet-55176785 2015-11-22 19:28:12,225 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - chain=NoCacheFilter->NoCacheFilter->safety->Timeline Authentication Filter->RMAuthenticationFilter->static_user_filter->guice->org.mortbay.jetty.servlet.DefaultServlet-55176785 2015-11-22 19:28:12,225 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - servlet holder=org.mortbay.jetty.servlet.DefaultServlet-55176785 2015-11-22 19:28:12,226 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter NoCacheFilter 2015-11-22 19:28:12,226 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter NoCacheFilter 2015-11-22 19:28:12,226 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter safety 2015-11-22 19:28:12,226 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter Timeline Authentication Filter 2015-11-22 19:28:12,227 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(576)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:41381/ws/v1/timeline/] user [ubuntu] authenticated 2015-11-22 19:28:12,227 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter RMAuthenticationFilter 2015-11-22 19:28:12,227 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(576)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:41381/ws/v1/timeline/] user [ubuntu] authenticated 2015-11-22 19:28:12,227 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter static_user_filter 2015-11-22 19:28:12,227 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter guice 2015-11-22 19:28:12,236 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] security.TimelineACLsManager (TimelineACLsManager.java:checkAccess(106)) - Verifying the access of ubuntu on the timeline entity { id: container_1448220478671_0001_01_000005, type: DS_CONTAINER } 2015-11-22 19:28:12,241 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - RESPONSE /ws/v1/timeline/ 200 2015-11-22 19:28:12,241 INFO [ContainersLauncher #1] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:buildCommandExecutor(281)) - launchContainer: [nice, -n, 0, bash, /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220478671_0001/container_1448220478671_0001_01_000005/default_container_executor.sh] 2015-11-22 19:28:12,251 DEBUG [ContainersLauncher #1] launcher.ContainerLaunch (ContainerLaunch.java:call(333)) - Container container_1448220478671_0001_01_000005 completed with exit code 0 2015-11-22 19:28:12,251 INFO [ContainersLauncher #1] launcher.ContainerLaunch (ContainerLaunch.java:call(356)) - Container container_1448220478671_0001_01_000005 succeeded 2015-11-22 19:28:12,251 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerEvent.EventType: CONTAINER_EXITED_WITH_SUCCESS 2015-11-22 19:28:12,251 DEBUG [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1131)) - Processing container_1448220478671_0001_01_000005 of type CONTAINER_EXITED_WITH_SUCCESS 2015-11-22 19:28:12,251 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220478671_0001_01_000005 transitioned from RUNNING to EXITED_WITH_SUCCESS 2015-11-22 19:28:12,251 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncherEvent.EventType: CLEANUP_CONTAINER 2015-11-22 19:28:12,251 INFO [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(380)) - Cleaning up container container_1448220478671_0001_01_000005 2015-11-22 19:28:12,251 DEBUG [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(397)) - Marking container container_1448220478671_0001_01_000005 as inactive 2015-11-22 19:28:12,251 DEBUG [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(403)) - Getting pid for container container_1448220478671_0001_01_000005 to kill from pid file /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/application_1448220478671_0001/container_1448220478671_0001_01_000005/container_1448220478671_0001_01_000005.pid 2015-11-22 19:28:12,251 DEBUG [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:getContainerPid(569)) - Accessing pid for container container_1448220478671_0001_01_000005 from pid file /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/application_1448220478671_0001/container_1448220478671_0001_01_000005/container_1448220478671_0001_01_000005.pid 2015-11-22 19:28:12,251 DEBUG [AsyncDispatcher event handler] util.ProcessIdFileReader (ProcessIdFileReader.java:getProcessId(51)) - Accessing pid from pid file /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/application_1448220478671_0001/container_1448220478671_0001_01_000005/container_1448220478671_0001_01_000005.pid 2015-11-22 19:28:12,251 DEBUG [AsyncDispatcher event handler] util.ProcessIdFileReader (ProcessIdFileReader.java:getProcessId(99)) - Got pid 30244 from path /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/application_1448220478671_0001/container_1448220478671_0001_01_000005/container_1448220478671_0001_01_000005.pid 2015-11-22 19:28:12,251 DEBUG [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:getContainerPid(579)) - Got pid 30244 for container container_1448220478671_0001_01_000005 2015-11-22 19:28:12,251 DEBUG [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(421)) - Sending signal to pid 30244 as user ubuntu for container container_1448220478671_0001_01_000005 2015-11-22 19:28:12,252 DEBUG [AsyncDispatcher event handler] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:signalContainer(417)) - Sending signal 15 to pid 30244 as user ubuntu 2015-11-22 19:28:12,254 DEBUG [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(437)) - Sent signal SIGTERM to pid 30244 as user ubuntu for container container_1448220478671_0001_01_000005, result=failed 2015-11-22 19:28:12,254 DEBUG [AsyncDispatcher event handler] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.fs.FileContext.getAbstractFileSystem(FileContext.java:339) 2015-11-22 19:28:12,265 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.ContainerLocalizationCleanupEvent.EventType: CLEANUP_CONTAINER_RESOURCES 2015-11-22 19:28:12,265 DEBUG [DeletionService #0] nodemanager.DeletionService (DeletionService.java:run(266)) - FileDeletionTask : user : ubuntu subDir : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220478671_0001/container_1448220478671_0001_01_000005 baseDir : null 2015-11-22 19:28:12,266 DEBUG [DeletionService #3] nodemanager.DeletionService (DeletionService.java:run(266)) - FileDeletionTask : user : null subDir : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/application_1448220478671_0001/container_1448220478671_0001_01_000005 baseDir : null 2015-11-22 19:28:12,266 DEBUG [DeletionService #3] nodemanager.DeletionService (DeletionService.java:run(271)) - NM deleting absolute path : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/application_1448220478671_0001/container_1448220478671_0001_01_000005 2015-11-22 19:28:12,266 DEBUG [DeletionService #0] nodemanager.DeletionService (DeletionService.java:run(292)) - Deleting path: [/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220478671_0001/container_1448220478671_0001_01_000005] as user: [ubuntu] 2015-11-22 19:28:12,266 INFO [DeletionService #0] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(480)) - Deleting absolute path : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220478671_0001/container_1448220478671_0001_01_000005 2015-11-22 19:28:12,266 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerEvent.EventType: CONTAINER_RESOURCES_CLEANEDUP 2015-11-22 19:28:12,267 DEBUG [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1131)) - Processing container_1448220478671_0001_01_000005 of type CONTAINER_RESOURCES_CLEANEDUP 2015-11-22 19:28:12,267 INFO [AsyncDispatcher event handler] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu OPERATION=Container Finished - Succeeded TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1448220478671_0001 CONTAINERID=container_1448220478671_0001_01_000005 2015-11-22 19:28:12,267 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220478671_0001_01_000005 transitioned from EXITED_WITH_SUCCESS to DONE 2015-11-22 19:28:12,267 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationContainerFinishedEvent.EventType: APPLICATION_CONTAINER_FINISHED 2015-11-22 19:28:12,267 DEBUG [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(452)) - Processing application_1448220478671_0001 of type APPLICATION_CONTAINER_FINISHED 2015-11-22 19:28:12,267 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(346)) - Removing container_1448220478671_0001_01_000005 from application application_1448220478671_0001 2015-11-22 19:28:12,267 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerStopMonitoringEvent.EventType: STOP_MONITORING_CONTAINER 2015-11-22 19:28:12,267 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.event.LogHandlerContainerFinishedEvent.EventType: CONTAINER_FINISHED 2015-11-22 19:28:12,267 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServicesEvent.EventType: CONTAINER_STOP 2015-11-22 19:28:12,267 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_STOP for appId application_1448220478671_0001 2015-11-22 19:28:12,306 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:28:12,306 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 3 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ], ContainerStatus: [ContainerId: container_1448220478671_0001_01_000004, State: COMPLETE, Capability: , Diagnostics: , ExitStatus: 0, ], ContainerStatus: [ContainerId: container_1448220478671_0001_01_000005, State: COMPLETE, Capability: , Diagnostics: , ExitStatus: 0, ]] 2015-11-22 19:28:12,306 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:12,306 INFO [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:removeOrTrackCompletedContainersFromContext(605)) - Removed completed containers from NM context: [container_1448220478671_0001_01_000003, container_1448220478671_0001_01_000002] 2015-11-22 19:28:12,306 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:28:12,307 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:28:12,307 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:28:12,307 DEBUG [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(424)) - Processing container_1448220478671_0001_01_000004 of type FINISHED 2015-11-22 19:28:12,307 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220478671_0001_01_000004 Container Transitioned from RUNNING to COMPLETED 2015-11-22 19:28:12,307 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptContainerFinishedEvent.EventType: CONTAINER_FINISHED 2015-11-22 19:28:12,307 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220478671_0001 CONTAINERID=container_1448220478671_0001_01_000004 2015-11-22 19:28:12,307 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type CONTAINER_FINISHED 2015-11-22 19:28:12,308 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:releaseContainer(248)) - Released container container_1448220478671_0001_01_000004 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:50663, which currently has 2 containers, used and available, release resources=true 2015-11-22 19:28:12,308 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:12,308 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:releaseResource(1313)) - default used= numContainers=2 user=ubuntu user-resources= 2015-11-22 19:28:12,308 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:internalReleaseResource(630)) - completedContainer root: numChildQueue= 1, capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.15625, numApps=1, numContainers=2, cluster= 2015-11-22 19:28:12,308 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:internalReleaseResource(643)) - Re-sorting completed queue: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.15625, absoluteUsedCapacity=0.15625, numApps=1, numContainers=2 2015-11-22 19:28:12,308 DEBUG [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(424)) - Processing container_1448220478671_0001_01_000005 of type FINISHED 2015-11-22 19:28:12,308 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220478671_0001_01_000005 Container Transitioned from ACQUIRED to COMPLETED 2015-11-22 19:28:12,308 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptContainerFinishedEvent.EventType: CONTAINER_FINISHED 2015-11-22 19:28:12,308 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220478671_0001 CONTAINERID=container_1448220478671_0001_01_000005 2015-11-22 19:28:12,308 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type CONTAINER_FINISHED 2015-11-22 19:28:12,308 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:releaseContainer(248)) - Released container container_1448220478671_0001_01_000005 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:50663, which currently has 1 containers, used and available, release resources=true 2015-11-22 19:28:12,309 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:12,309 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:releaseResource(1313)) - default used= numContainers=1 user=ubuntu user-resources= 2015-11-22 19:28:12,309 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:internalReleaseResource(630)) - completedContainer root: numChildQueue= 1, capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.125, numApps=1, numContainers=1, cluster= 2015-11-22 19:28:12,309 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:internalReleaseResource(643)) - Re-sorting completed queue: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.125, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 2015-11-22 19:28:12,309 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:28:12,309 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:28:12,309 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:28:12,504 DEBUG [Task killer for 30244] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:signalContainer(417)) - Sending signal 9 to pid 30244 as user ubuntu 2015-11-22 19:28:12,811 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #833 2015-11-22 19:28:12,811 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #833 2015-11-22 19:28:12,812 DEBUG [IPC Server handler 15 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 15 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#833 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:12,812 DEBUG [IPC Server handler 15 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:12,812 DEBUG [IPC Server handler 15 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:28:12,812 DEBUG [IPC Server handler 15 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 1 procesingTime= 0 2015-11-22 19:28:12,813 DEBUG [IPC Server handler 15 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 15 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#833 Retry#0 2015-11-22 19:28:12,813 DEBUG [IPC Server handler 15 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 15 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#833 Retry#0 Wrote 359 bytes. 2015-11-22 19:28:12,813 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #833 2015-11-22 19:28:12,813 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:28:12,813 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:28:13,144 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #14 2015-11-22 19:28:13,144 DEBUG [IPC Server handler 9 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 9 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#14 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:13,144 DEBUG [IPC Server handler 9 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:13,145 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:13,145 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:28:13,145 DEBUG [IPC Server handler 9 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:13,145 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:13,145 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:13,145 DEBUG [IPC Server handler 9 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 1 2015-11-22 19:28:13,146 DEBUG [IPC Server handler 9 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 9 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#14 Retry#0 2015-11-22 19:28:13,146 DEBUG [IPC Server handler 9 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 9 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#14 Retry#0 Wrote 115 bytes. 2015-11-22 19:28:13,148 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - REQUEST /ws/v1/timeline/ on org.mortbay.jetty.HttpConnection@369d14ce 2015-11-22 19:28:13,149 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - sessionManager=org.mortbay.jetty.servlet.HashSessionManager@3966697d 2015-11-22 19:28:13,149 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - session=null 2015-11-22 19:28:13,149 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - servlet=org.mortbay.jetty.servlet.DefaultServlet-55176785 2015-11-22 19:28:13,149 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - chain=NoCacheFilter->NoCacheFilter->safety->Timeline Authentication Filter->RMAuthenticationFilter->static_user_filter->guice->org.mortbay.jetty.servlet.DefaultServlet-55176785 2015-11-22 19:28:13,149 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - servlet holder=org.mortbay.jetty.servlet.DefaultServlet-55176785 2015-11-22 19:28:13,149 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter NoCacheFilter 2015-11-22 19:28:13,149 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter NoCacheFilter 2015-11-22 19:28:13,149 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter safety 2015-11-22 19:28:13,149 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter Timeline Authentication Filter 2015-11-22 19:28:13,150 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(557)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:41381/ws/v1/timeline/?user.name=ubuntu] triggering authentication 2015-11-22 19:28:13,150 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(576)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:41381/ws/v1/timeline/?user.name=ubuntu] user [ubuntu] authenticated 2015-11-22 19:28:13,150 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter RMAuthenticationFilter 2015-11-22 19:28:13,150 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(557)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:41381/ws/v1/timeline/?user.name=ubuntu] triggering authentication 2015-11-22 19:28:13,150 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(576)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:41381/ws/v1/timeline/?user.name=ubuntu] user [ubuntu] authenticated 2015-11-22 19:28:13,150 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter static_user_filter 2015-11-22 19:28:13,150 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter guice 2015-11-22 19:28:13,153 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - getResource(META-INF/services/javax.xml.bind.JAXBContext)=jar:file:/home/ubuntu/.m2/repository/com/sun/xml/bind/jaxb-impl/2.2.3-1/jaxb-impl-2.2.3-1.jar!/META-INF/services/javax.xml.bind.JAXBContext 2015-11-22 19:28:13,153 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.xml.bind.v2.ContextFactory from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:28:13,178 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - RESPONSE /ws/v1/timeline/ 200 2015-11-22 19:28:13,180 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - REQUEST /ws/v1/timeline/ on org.mortbay.jetty.HttpConnection@62b51514 2015-11-22 19:28:13,180 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - sessionManager=org.mortbay.jetty.servlet.HashSessionManager@3966697d 2015-11-22 19:28:13,180 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - session=null 2015-11-22 19:28:13,181 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - servlet=org.mortbay.jetty.servlet.DefaultServlet-55176785 2015-11-22 19:28:13,181 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - chain=NoCacheFilter->NoCacheFilter->safety->Timeline Authentication Filter->RMAuthenticationFilter->static_user_filter->guice->org.mortbay.jetty.servlet.DefaultServlet-55176785 2015-11-22 19:28:13,181 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - servlet holder=org.mortbay.jetty.servlet.DefaultServlet-55176785 2015-11-22 19:28:13,181 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter NoCacheFilter 2015-11-22 19:28:13,181 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter NoCacheFilter 2015-11-22 19:28:13,181 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter safety 2015-11-22 19:28:13,181 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter Timeline Authentication Filter 2015-11-22 19:28:13,181 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(576)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:41381/ws/v1/timeline/] user [ubuntu] authenticated 2015-11-22 19:28:13,182 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter RMAuthenticationFilter 2015-11-22 19:28:13,182 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(576)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:41381/ws/v1/timeline/] user [ubuntu] authenticated 2015-11-22 19:28:13,182 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter static_user_filter 2015-11-22 19:28:13,182 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter guice 2015-11-22 19:28:13,183 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] security.TimelineACLsManager (TimelineACLsManager.java:checkAccess(106)) - Verifying the access of ubuntu on the timeline entity { id: container_1448220478671_0001_01_000004, type: DS_CONTAINER } 2015-11-22 19:28:13,184 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - RESPONSE /ws/v1/timeline/ 200 2015-11-22 19:28:13,185 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - REQUEST /ws/v1/timeline/ on org.mortbay.jetty.HttpConnection@62b51514 2015-11-22 19:28:13,185 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - sessionManager=org.mortbay.jetty.servlet.HashSessionManager@3966697d 2015-11-22 19:28:13,185 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - session=null 2015-11-22 19:28:13,185 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - servlet=org.mortbay.jetty.servlet.DefaultServlet-55176785 2015-11-22 19:28:13,185 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - chain=NoCacheFilter->NoCacheFilter->safety->Timeline Authentication Filter->RMAuthenticationFilter->static_user_filter->guice->org.mortbay.jetty.servlet.DefaultServlet-55176785 2015-11-22 19:28:13,185 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - servlet holder=org.mortbay.jetty.servlet.DefaultServlet-55176785 2015-11-22 19:28:13,185 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter NoCacheFilter 2015-11-22 19:28:13,185 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter NoCacheFilter 2015-11-22 19:28:13,186 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter safety 2015-11-22 19:28:13,186 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter Timeline Authentication Filter 2015-11-22 19:28:13,186 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(557)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:41381/ws/v1/timeline/?user.name=ubuntu] triggering authentication 2015-11-22 19:28:13,186 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(576)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:41381/ws/v1/timeline/?user.name=ubuntu] user [ubuntu] authenticated 2015-11-22 19:28:13,186 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter RMAuthenticationFilter 2015-11-22 19:28:13,186 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(557)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:41381/ws/v1/timeline/?user.name=ubuntu] triggering authentication 2015-11-22 19:28:13,186 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(576)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:41381/ws/v1/timeline/?user.name=ubuntu] user [ubuntu] authenticated 2015-11-22 19:28:13,186 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter static_user_filter 2015-11-22 19:28:13,187 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter guice 2015-11-22 19:28:13,190 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - getResource(META-INF/services/javax.xml.bind.JAXBContext)=jar:file:/home/ubuntu/.m2/repository/com/sun/xml/bind/jaxb-impl/2.2.3-1/jaxb-impl-2.2.3-1.jar!/META-INF/services/javax.xml.bind.JAXBContext 2015-11-22 19:28:13,191 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.xml.bind.v2.ContextFactory from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:28:13,214 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - RESPONSE /ws/v1/timeline/ 200 2015-11-22 19:28:13,216 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - REQUEST /ws/v1/timeline/ on org.mortbay.jetty.HttpConnection@1a0fc832 2015-11-22 19:28:13,216 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - sessionManager=org.mortbay.jetty.servlet.HashSessionManager@3966697d 2015-11-22 19:28:13,216 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - session=null 2015-11-22 19:28:13,216 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - servlet=org.mortbay.jetty.servlet.DefaultServlet-55176785 2015-11-22 19:28:13,216 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - chain=NoCacheFilter->NoCacheFilter->safety->Timeline Authentication Filter->RMAuthenticationFilter->static_user_filter->guice->org.mortbay.jetty.servlet.DefaultServlet-55176785 2015-11-22 19:28:13,216 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - servlet holder=org.mortbay.jetty.servlet.DefaultServlet-55176785 2015-11-22 19:28:13,216 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter NoCacheFilter 2015-11-22 19:28:13,217 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter NoCacheFilter 2015-11-22 19:28:13,217 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter safety 2015-11-22 19:28:13,217 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter Timeline Authentication Filter 2015-11-22 19:28:13,217 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(576)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:41381/ws/v1/timeline/] user [ubuntu] authenticated 2015-11-22 19:28:13,217 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter RMAuthenticationFilter 2015-11-22 19:28:13,217 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(576)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:41381/ws/v1/timeline/] user [ubuntu] authenticated 2015-11-22 19:28:13,217 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter static_user_filter 2015-11-22 19:28:13,218 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter guice 2015-11-22 19:28:13,218 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] security.TimelineACLsManager (TimelineACLsManager.java:checkAccess(106)) - Verifying the access of ubuntu on the timeline entity { id: container_1448220478671_0001_01_000005, type: DS_CONTAINER } 2015-11-22 19:28:13,219 DEBUG [2048592079@qtp-847268847-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - RESPONSE /ws/v1/timeline/ 200 2015-11-22 19:28:13,307 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:28:13,307 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:28:13,307 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:13,307 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:28:13,308 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:28:13,308 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:28:13,308 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:28:13,308 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:28:13,308 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:28:13,813 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #834 2015-11-22 19:28:13,814 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #834 2015-11-22 19:28:13,814 DEBUG [IPC Server handler 14 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 14 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#834 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:13,814 DEBUG [IPC Server handler 14 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:13,814 DEBUG [IPC Server handler 14 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:28:13,814 DEBUG [IPC Server handler 14 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 0 2015-11-22 19:28:13,815 DEBUG [IPC Server handler 14 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 14 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#834 Retry#0 2015-11-22 19:28:13,815 DEBUG [IPC Server handler 14 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 14 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#834 Retry#0 Wrote 359 bytes. 2015-11-22 19:28:13,815 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #834 2015-11-22 19:28:13,815 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:28:13,815 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:28:14,147 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #15 2015-11-22 19:28:14,147 DEBUG [IPC Server handler 10 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 10 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#15 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:14,148 DEBUG [IPC Server handler 10 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:14,148 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:14,149 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:28:14,149 DEBUG [IPC Server handler 10 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:14,149 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:14,149 DEBUG [IPC Server handler 10 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 1 procesingTime= 1 2015-11-22 19:28:14,149 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:14,149 DEBUG [IPC Server handler 10 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 10 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#15 Retry#0 2015-11-22 19:28:14,149 DEBUG [IPC Server handler 10 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 10 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#15 Retry#0 Wrote 47 bytes. 2015-11-22 19:28:14,308 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:28:14,308 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:28:14,308 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:14,308 INFO [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:removeOrTrackCompletedContainersFromContext(605)) - Removed completed containers from NM context: [container_1448220478671_0001_01_000004, container_1448220478671_0001_01_000005] 2015-11-22 19:28:14,308 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:28:14,308 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:28:14,308 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:28:14,308 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:28:14,308 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:28:14,309 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:28:14,816 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #835 2015-11-22 19:28:14,816 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #835 2015-11-22 19:28:14,816 DEBUG [IPC Server handler 18 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 18 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#835 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:14,816 DEBUG [IPC Server handler 18 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:14,816 DEBUG [IPC Server handler 18 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:28:14,817 DEBUG [IPC Server handler 18 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 1 2015-11-22 19:28:14,817 DEBUG [IPC Server handler 18 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 18 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#835 Retry#0 2015-11-22 19:28:14,817 DEBUG [IPC Server handler 18 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 18 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#835 Retry#0 Wrote 359 bytes. 2015-11-22 19:28:14,817 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #835 2015-11-22 19:28:14,817 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:28:14,818 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:28:15,151 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #16 2015-11-22 19:28:15,151 DEBUG [IPC Server handler 1 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 1 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#16 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:15,151 DEBUG [IPC Server handler 1 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:15,152 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:15,152 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:28:15,152 DEBUG [IPC Server handler 1 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:15,152 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:15,152 DEBUG [IPC Server handler 1 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 1 2015-11-22 19:28:15,152 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:15,152 DEBUG [IPC Server handler 1 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 1 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#16 Retry#0 2015-11-22 19:28:15,152 DEBUG [IPC Server handler 1 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 1 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#16 Retry#0 Wrote 47 bytes. 2015-11-22 19:28:15,308 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:28:15,308 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:28:15,309 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:15,309 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:28:15,309 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:28:15,309 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:28:15,309 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:28:15,309 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:28:15,309 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:28:15,332 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:50663 from appattempt_1448220478671_0001_000001] ipc.Client (Client.java:close(1217)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:50663 from appattempt_1448220478671_0001_000001: closed 2015-11-22 19:28:15,332 DEBUG [Socket Reader #1 for port 50663] ipc.Server (Server.java:close(2894)) - Socket Reader #1 for port 50663: disconnecting client 172.31.20.42:35345. Number of active connections: 0 2015-11-22 19:28:15,332 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:50663 from appattempt_1448220478671_0001_000001] ipc.Client (Client.java:run(1018)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:50663 from appattempt_1448220478671_0001_000001: stopped, remaining connections 2 2015-11-22 19:28:15,818 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #836 2015-11-22 19:28:15,818 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #836 2015-11-22 19:28:15,818 DEBUG [IPC Server handler 19 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 19 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#836 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:15,819 DEBUG [IPC Server handler 19 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:15,819 DEBUG [IPC Server handler 19 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:28:15,819 DEBUG [IPC Server handler 19 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 1 procesingTime= 0 2015-11-22 19:28:15,819 DEBUG [IPC Server handler 19 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 19 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#836 Retry#0 2015-11-22 19:28:15,819 DEBUG [IPC Server handler 19 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 19 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#836 Retry#0 Wrote 359 bytes. 2015-11-22 19:28:15,820 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #836 2015-11-22 19:28:15,820 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:28:15,820 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:28:16,154 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #17 2015-11-22 19:28:16,154 DEBUG [IPC Server handler 4 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 4 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#17 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:16,154 DEBUG [IPC Server handler 4 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:16,155 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:16,155 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:28:16,155 DEBUG [IPC Server handler 4 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:16,155 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:16,155 DEBUG [IPC Server handler 4 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 1 2015-11-22 19:28:16,155 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:16,156 DEBUG [IPC Server handler 4 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 4 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#17 Retry#0 2015-11-22 19:28:16,156 DEBUG [IPC Server handler 4 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 4 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#17 Retry#0 Wrote 47 bytes. 2015-11-22 19:28:16,309 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:28:16,309 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:28:16,309 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:16,309 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:28:16,309 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:28:16,310 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:28:16,310 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:28:16,310 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:28:16,310 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:28:16,458 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:47651 from ubuntu] ipc.Client (Client.java:close(1217)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:47651 from ubuntu: closed 2015-11-22 19:28:16,458 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:47651 from ubuntu] ipc.Client (Client.java:run(1018)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:47651 from ubuntu: stopped, remaining connections 1 2015-11-22 19:28:16,458 DEBUG [Socket Reader #1 for port 47651] ipc.Server (Server.java:close(2894)) - Socket Reader #1 for port 47651: disconnecting client 172.31.20.42:40571. Number of active connections: 0 2015-11-22 19:28:16,820 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #837 2015-11-22 19:28:16,821 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #837 2015-11-22 19:28:16,821 DEBUG [IPC Server handler 20 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 20 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#837 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:16,821 DEBUG [IPC Server handler 20 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:16,821 DEBUG [IPC Server handler 20 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:28:16,821 DEBUG [IPC Server handler 20 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 0 2015-11-22 19:28:16,822 DEBUG [IPC Server handler 20 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 20 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#837 Retry#0 2015-11-22 19:28:16,822 DEBUG [IPC Server handler 20 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 20 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#837 Retry#0 Wrote 359 bytes. 2015-11-22 19:28:16,822 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #837 2015-11-22 19:28:16,822 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:28:16,822 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:28:17,157 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #18 2015-11-22 19:28:17,157 DEBUG [IPC Server handler 13 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 13 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#18 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:17,157 DEBUG [IPC Server handler 13 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:17,158 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:17,158 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:28:17,158 DEBUG [IPC Server handler 13 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:17,158 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:17,158 DEBUG [IPC Server handler 13 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 1 2015-11-22 19:28:17,158 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:17,159 DEBUG [IPC Server handler 13 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 13 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#18 Retry#0 2015-11-22 19:28:17,159 DEBUG [IPC Server handler 13 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 13 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#18 Retry#0 Wrote 47 bytes. 2015-11-22 19:28:17,309 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:28:17,310 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:28:17,310 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:17,310 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:28:17,310 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:28:17,310 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:28:17,310 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:28:17,310 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:28:17,310 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:28:17,823 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #838 2015-11-22 19:28:17,823 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #838 2015-11-22 19:28:17,823 DEBUG [IPC Server handler 21 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 21 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#838 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:17,823 DEBUG [IPC Server handler 21 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:17,823 DEBUG [IPC Server handler 21 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:28:17,824 DEBUG [IPC Server handler 21 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 1 2015-11-22 19:28:17,824 DEBUG [IPC Server handler 21 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 21 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#838 Retry#0 2015-11-22 19:28:17,824 DEBUG [IPC Server handler 21 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 21 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#838 Retry#0 Wrote 359 bytes. 2015-11-22 19:28:17,824 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #838 2015-11-22 19:28:17,824 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:28:17,824 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:28:18,160 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #19 2015-11-22 19:28:18,160 DEBUG [IPC Server handler 11 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 11 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#19 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:18,160 DEBUG [IPC Server handler 11 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:18,161 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:18,161 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:28:18,161 DEBUG [IPC Server handler 11 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:18,161 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:18,161 DEBUG [IPC Server handler 11 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 1 2015-11-22 19:28:18,161 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:18,162 DEBUG [IPC Server handler 11 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 11 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#19 Retry#0 2015-11-22 19:28:18,162 DEBUG [IPC Server handler 11 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 11 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#19 Retry#0 Wrote 47 bytes. 2015-11-22 19:28:18,310 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:28:18,310 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:28:18,310 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:18,310 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:28:18,311 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:28:18,311 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:28:18,311 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:28:18,311 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:28:18,311 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:28:18,678 DEBUG [IPC Server idle connection scanner for port 48629] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 48629: task running 2015-11-22 19:28:18,685 DEBUG [IPC Server idle connection scanner for port 46284] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 46284: task running 2015-11-22 19:28:18,692 DEBUG [IPC Server idle connection scanner for port 52028] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 52028: task running 2015-11-22 19:28:18,825 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #839 2015-11-22 19:28:18,825 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #839 2015-11-22 19:28:18,825 DEBUG [IPC Server handler 22 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 22 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#839 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:18,825 DEBUG [IPC Server handler 22 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:18,825 DEBUG [IPC Server handler 22 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:28:18,826 DEBUG [IPC Server handler 22 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 1 2015-11-22 19:28:18,826 DEBUG [IPC Server handler 22 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 22 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#839 Retry#0 2015-11-22 19:28:18,826 DEBUG [IPC Server handler 22 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 22 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#839 Retry#0 Wrote 359 bytes. 2015-11-22 19:28:18,826 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #839 2015-11-22 19:28:18,826 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:28:18,826 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:28:18,948 DEBUG [IPC Server idle connection scanner for port 53669] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 53669: task running 2015-11-22 19:28:19,163 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #20 2015-11-22 19:28:19,163 DEBUG [IPC Server handler 14 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 14 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#20 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:19,163 DEBUG [IPC Server handler 14 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:19,164 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:19,164 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:28:19,164 DEBUG [IPC Server handler 14 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:19,164 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:19,165 DEBUG [IPC Server handler 14 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 2 2015-11-22 19:28:19,165 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:19,165 DEBUG [IPC Server handler 14 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 14 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#20 Retry#0 2015-11-22 19:28:19,165 DEBUG [IPC Server handler 14 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 14 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#20 Retry#0 Wrote 47 bytes. 2015-11-22 19:28:19,311 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:28:19,311 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:28:19,311 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:19,311 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:28:19,311 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:28:19,311 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:28:19,311 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:28:19,311 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:28:19,311 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:28:19,827 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #840 2015-11-22 19:28:19,827 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #840 2015-11-22 19:28:19,827 DEBUG [IPC Server handler 23 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 23 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#840 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:19,827 DEBUG [IPC Server handler 23 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:19,827 DEBUG [IPC Server handler 23 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:28:19,828 DEBUG [IPC Server handler 23 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 1 2015-11-22 19:28:19,828 DEBUG [IPC Server handler 23 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 23 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#840 Retry#0 2015-11-22 19:28:19,828 DEBUG [IPC Server handler 23 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 23 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#840 Retry#0 Wrote 359 bytes. 2015-11-22 19:28:19,828 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #840 2015-11-22 19:28:19,828 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:28:19,829 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:28:20,166 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #21 2015-11-22 19:28:20,166 DEBUG [IPC Server handler 3 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 3 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#21 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:20,166 DEBUG [IPC Server handler 3 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:20,167 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:20,167 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:28:20,167 DEBUG [IPC Server handler 3 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:20,168 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:20,168 DEBUG [IPC Server handler 3 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 1 procesingTime= 1 2015-11-22 19:28:20,168 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:20,168 DEBUG [IPC Server handler 3 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 3 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#21 Retry#0 2015-11-22 19:28:20,168 DEBUG [IPC Server handler 3 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 3 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#21 Retry#0 Wrote 47 bytes. 2015-11-22 19:28:20,177 DEBUG [IPC Server idle connection scanner for port 50663] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 50663: task running 2015-11-22 19:28:20,183 DEBUG [IPC Server idle connection scanner for port 47651] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 47651: task running 2015-11-22 19:28:20,311 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:28:20,311 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:28:20,312 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:20,312 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:28:20,312 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:28:20,312 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:28:20,312 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:28:20,312 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:28:20,312 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:28:20,829 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #841 2015-11-22 19:28:20,829 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #841 2015-11-22 19:28:20,829 DEBUG [IPC Server handler 24 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 24 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#841 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:20,829 DEBUG [IPC Server handler 24 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:20,830 DEBUG [IPC Server handler 24 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:28:20,830 DEBUG [IPC Server handler 24 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 1 procesingTime= 0 2015-11-22 19:28:20,830 DEBUG [IPC Server handler 24 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 24 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#841 Retry#0 2015-11-22 19:28:20,830 DEBUG [IPC Server handler 24 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 24 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#841 Retry#0 Wrote 359 bytes. 2015-11-22 19:28:20,830 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #841 2015-11-22 19:28:20,830 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 1ms 2015-11-22 19:28:20,831 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:28:21,169 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #22 2015-11-22 19:28:21,170 DEBUG [IPC Server handler 15 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 15 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#22 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:21,170 DEBUG [IPC Server handler 15 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:21,171 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:21,171 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:28:21,171 DEBUG [IPC Server handler 15 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:21,171 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:21,171 DEBUG [IPC Server handler 15 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 1 2015-11-22 19:28:21,171 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:21,171 DEBUG [IPC Server handler 15 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 15 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#22 Retry#0 2015-11-22 19:28:21,171 DEBUG [IPC Server handler 15 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 15 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#22 Retry#0 Wrote 47 bytes. 2015-11-22 19:28:21,177 DEBUG [IPC Server idle connection scanner for port 32936] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 32936: task running 2015-11-22 19:28:21,312 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:28:21,312 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:28:21,312 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:21,312 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:28:21,312 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:28:21,312 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:28:21,313 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:28:21,313 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:28:21,313 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:28:21,831 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #842 2015-11-22 19:28:21,831 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #842 2015-11-22 19:28:21,831 DEBUG [IPC Server handler 25 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 25 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#842 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:21,832 DEBUG [IPC Server handler 25 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:21,832 DEBUG [IPC Server handler 25 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:28:21,832 DEBUG [IPC Server handler 25 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 1 procesingTime= 0 2015-11-22 19:28:21,832 DEBUG [IPC Server handler 25 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 25 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#842 Retry#0 2015-11-22 19:28:21,832 DEBUG [IPC Server handler 25 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 25 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#842 Retry#0 Wrote 359 bytes. 2015-11-22 19:28:21,832 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #842 2015-11-22 19:28:21,833 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:28:21,833 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:28:22,173 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #23 2015-11-22 19:28:22,173 DEBUG [IPC Server handler 12 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 12 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#23 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:22,173 DEBUG [IPC Server handler 12 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:22,174 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:22,174 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:28:22,174 DEBUG [IPC Server handler 12 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:22,174 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:22,174 DEBUG [IPC Server handler 12 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 1 2015-11-22 19:28:22,174 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:22,174 DEBUG [IPC Server handler 12 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 12 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#23 Retry#0 2015-11-22 19:28:22,175 DEBUG [IPC Server handler 12 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 12 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#23 Retry#0 Wrote 47 bytes. 2015-11-22 19:28:22,312 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:28:22,313 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:28:22,313 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:22,313 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:28:22,313 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:28:22,313 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:28:22,313 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:28:22,313 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:28:22,313 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:28:22,833 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #843 2015-11-22 19:28:22,833 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #843 2015-11-22 19:28:22,834 DEBUG [IPC Server handler 26 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 26 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#843 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:22,834 DEBUG [IPC Server handler 26 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:22,834 DEBUG [IPC Server handler 26 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:28:22,834 DEBUG [IPC Server handler 26 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 1 procesingTime= 0 2015-11-22 19:28:22,834 DEBUG [IPC Server handler 26 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 26 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#843 Retry#0 2015-11-22 19:28:22,834 DEBUG [IPC Server handler 26 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 26 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#843 Retry#0 Wrote 359 bytes. 2015-11-22 19:28:22,834 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #843 2015-11-22 19:28:22,835 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:28:22,835 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:28:22,836 DEBUG [2048592079@qtp-847268847-0] mortbay.log (Slf4jLog.java:debug(40)) - EOF 2015-11-22 19:28:23,177 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #24 2015-11-22 19:28:23,177 DEBUG [IPC Server handler 18 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 18 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#24 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:23,177 DEBUG [IPC Server handler 18 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:23,178 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:23,178 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:28:23,178 DEBUG [IPC Server handler 18 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:23,178 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:23,178 DEBUG [IPC Server handler 18 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 1 2015-11-22 19:28:23,178 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:23,179 DEBUG [IPC Server handler 18 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 18 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#24 Retry#0 2015-11-22 19:28:23,179 DEBUG [IPC Server handler 18 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 18 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#24 Retry#0 Wrote 47 bytes. 2015-11-22 19:28:23,313 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:28:23,313 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:28:23,313 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:23,313 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:28:23,313 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:28:23,313 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:28:23,314 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:28:23,314 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:28:23,314 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:28:23,835 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #844 2015-11-22 19:28:23,835 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #844 2015-11-22 19:28:23,836 DEBUG [IPC Server handler 28 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 28 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#844 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:23,836 DEBUG [IPC Server handler 28 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:23,836 DEBUG [IPC Server handler 28 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:28:23,836 DEBUG [IPC Server handler 28 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 0 2015-11-22 19:28:23,836 DEBUG [IPC Server handler 28 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 28 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#844 Retry#0 2015-11-22 19:28:23,837 DEBUG [IPC Server handler 28 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 28 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#844 Retry#0 Wrote 359 bytes. 2015-11-22 19:28:23,837 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #844 2015-11-22 19:28:23,837 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:28:23,837 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:28:24,180 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #25 2015-11-22 19:28:24,181 DEBUG [IPC Server handler 20 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 20 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#25 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:24,181 DEBUG [IPC Server handler 20 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:24,182 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:24,182 DEBUG [IPC Server handler 20 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:24,182 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:28:24,182 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:24,182 DEBUG [IPC Server handler 20 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 1 procesingTime= 1 2015-11-22 19:28:24,182 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:24,182 DEBUG [IPC Server handler 20 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 20 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#25 Retry#0 2015-11-22 19:28:24,183 DEBUG [IPC Server handler 20 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 20 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#25 Retry#0 Wrote 47 bytes. 2015-11-22 19:28:24,313 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:28:24,314 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:28:24,314 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:24,314 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:28:24,314 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:28:24,314 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:28:24,314 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:28:24,314 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:28:24,314 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:28:24,837 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #845 2015-11-22 19:28:24,838 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #845 2015-11-22 19:28:24,838 DEBUG [IPC Server handler 27 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 27 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#845 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:24,838 DEBUG [IPC Server handler 27 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:24,838 DEBUG [IPC Server handler 27 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:28:24,839 DEBUG [IPC Server handler 27 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 1 2015-11-22 19:28:24,839 DEBUG [IPC Server handler 27 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 27 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#845 Retry#0 2015-11-22 19:28:24,839 DEBUG [IPC Server handler 27 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 27 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#845 Retry#0 Wrote 359 bytes. 2015-11-22 19:28:24,839 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #845 2015-11-22 19:28:24,839 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:28:24,839 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:28:25,184 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #26 2015-11-22 19:28:25,184 DEBUG [IPC Server handler 17 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 17 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#26 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:25,184 DEBUG [IPC Server handler 17 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:25,185 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:25,185 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:28:25,185 DEBUG [IPC Server handler 17 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:25,185 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:25,185 DEBUG [IPC Server handler 17 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 1 2015-11-22 19:28:25,185 DEBUG [IPC Server handler 17 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 17 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#26 Retry#0 2015-11-22 19:28:25,186 DEBUG [IPC Server handler 17 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 17 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#26 Retry#0 Wrote 47 bytes. 2015-11-22 19:28:25,185 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:25,314 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:28:25,314 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:28:25,314 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:25,314 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:28:25,315 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:28:25,315 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:28:25,315 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:28:25,315 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:28:25,315 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:28:25,840 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #846 2015-11-22 19:28:25,840 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #846 2015-11-22 19:28:25,840 DEBUG [IPC Server handler 17 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 17 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#846 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:25,840 DEBUG [IPC Server handler 17 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:25,841 DEBUG [IPC Server handler 17 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:28:25,841 DEBUG [IPC Server handler 17 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 1 2015-11-22 19:28:25,841 DEBUG [IPC Server handler 17 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 17 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#846 Retry#0 2015-11-22 19:28:25,841 DEBUG [IPC Server handler 17 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 17 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#846 Retry#0 Wrote 359 bytes. 2015-11-22 19:28:25,841 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #846 2015-11-22 19:28:25,842 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:28:25,842 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:28:26,187 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #27 2015-11-22 19:28:26,187 DEBUG [IPC Server handler 19 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 19 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#27 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:26,187 DEBUG [IPC Server handler 19 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:26,188 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:26,188 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:28:26,188 DEBUG [IPC Server handler 19 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:26,188 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:26,189 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:26,189 DEBUG [IPC Server handler 19 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 1 procesingTime= 1 2015-11-22 19:28:26,189 DEBUG [IPC Server handler 19 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 19 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#27 Retry#0 2015-11-22 19:28:26,189 DEBUG [IPC Server handler 19 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 19 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#27 Retry#0 Wrote 47 bytes. 2015-11-22 19:28:26,315 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:28:26,315 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:28:26,315 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:26,315 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:28:26,315 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:28:26,315 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:28:26,315 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:28:26,315 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:28:26,315 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:28:26,842 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #847 2015-11-22 19:28:26,842 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #847 2015-11-22 19:28:26,843 DEBUG [IPC Server handler 30 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 30 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#847 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:26,843 DEBUG [IPC Server handler 30 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:26,843 DEBUG [IPC Server handler 30 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:28:26,843 DEBUG [IPC Server handler 30 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 0 2015-11-22 19:28:26,843 DEBUG [IPC Server handler 30 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 30 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#847 Retry#0 2015-11-22 19:28:26,844 DEBUG [IPC Server handler 30 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 30 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#847 Retry#0 Wrote 359 bytes. 2015-11-22 19:28:26,844 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #847 2015-11-22 19:28:26,844 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:28:26,844 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:28:27,190 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #28 2015-11-22 19:28:27,190 DEBUG [IPC Server handler 21 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 21 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#28 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:27,190 DEBUG [IPC Server handler 21 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:27,192 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:27,192 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:28:27,192 DEBUG [IPC Server handler 21 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:27,193 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:27,193 DEBUG [IPC Server handler 21 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 1 procesingTime= 2 2015-11-22 19:28:27,193 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:27,193 DEBUG [IPC Server handler 21 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 21 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#28 Retry#0 2015-11-22 19:28:27,193 DEBUG [IPC Server handler 21 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 21 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#28 Retry#0 Wrote 47 bytes. 2015-11-22 19:28:27,315 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:28:27,315 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:28:27,315 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:27,316 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:28:27,316 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:28:27,316 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:28:27,316 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:28:27,316 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:28:27,316 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:28:27,844 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #848 2015-11-22 19:28:27,845 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #848 2015-11-22 19:28:27,845 DEBUG [IPC Server handler 29 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 29 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#848 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:27,845 DEBUG [IPC Server handler 29 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:27,845 DEBUG [IPC Server handler 29 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:28:27,845 DEBUG [IPC Server handler 29 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 0 2015-11-22 19:28:27,846 DEBUG [IPC Server handler 29 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 29 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#848 Retry#0 2015-11-22 19:28:27,846 DEBUG [IPC Server handler 29 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 29 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#848 Retry#0 Wrote 359 bytes. 2015-11-22 19:28:27,846 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #848 2015-11-22 19:28:27,846 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:28:27,846 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:28:28,194 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #29 2015-11-22 19:28:28,194 DEBUG [IPC Server handler 22 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 22 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#29 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:28,194 DEBUG [IPC Server handler 22 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:28,195 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:28,195 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:28:28,195 DEBUG [IPC Server handler 22 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:28,196 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:28,196 DEBUG [IPC Server handler 22 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 1 procesingTime= 1 2015-11-22 19:28:28,196 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:28,196 DEBUG [IPC Server handler 22 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 22 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#29 Retry#0 2015-11-22 19:28:28,196 DEBUG [IPC Server handler 22 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 22 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#29 Retry#0 Wrote 47 bytes. 2015-11-22 19:28:28,316 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:28:28,316 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:28:28,316 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:28,316 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:28:28,316 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:28:28,316 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:28:28,316 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:28:28,316 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:28:28,317 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:28:28,678 DEBUG [IPC Server idle connection scanner for port 48629] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 48629: task running 2015-11-22 19:28:28,685 DEBUG [IPC Server idle connection scanner for port 46284] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 46284: task running 2015-11-22 19:28:28,692 DEBUG [IPC Server idle connection scanner for port 52028] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 52028: task running 2015-11-22 19:28:28,847 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #849 2015-11-22 19:28:28,847 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #849 2015-11-22 19:28:28,847 DEBUG [IPC Server handler 31 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 31 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#849 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:28,847 DEBUG [IPC Server handler 31 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:28,847 DEBUG [IPC Server handler 31 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:28:28,848 DEBUG [IPC Server handler 31 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 1 2015-11-22 19:28:28,848 DEBUG [IPC Server handler 31 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 31 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#849 Retry#0 2015-11-22 19:28:28,848 DEBUG [IPC Server handler 31 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 31 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#849 Retry#0 Wrote 359 bytes. 2015-11-22 19:28:28,848 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #849 2015-11-22 19:28:28,848 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:28:28,849 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:28:28,948 DEBUG [IPC Server idle connection scanner for port 53669] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 53669: task running 2015-11-22 19:28:29,197 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #30 2015-11-22 19:28:29,198 DEBUG [IPC Server handler 23 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 23 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#30 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:29,198 DEBUG [IPC Server handler 23 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:29,198 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:29,198 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:28:29,198 DEBUG [IPC Server handler 23 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:29,199 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:29,199 DEBUG [IPC Server handler 23 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 1 procesingTime= 1 2015-11-22 19:28:29,199 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:29,199 DEBUG [IPC Server handler 23 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 23 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#30 Retry#0 2015-11-22 19:28:29,199 DEBUG [IPC Server handler 23 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 23 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#30 Retry#0 Wrote 47 bytes. 2015-11-22 19:28:29,316 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:28:29,316 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:28:29,317 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:29,317 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:28:29,317 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:28:29,317 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:28:29,317 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:28:29,317 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:28:29,317 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:28:29,849 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #850 2015-11-22 19:28:29,849 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #850 2015-11-22 19:28:29,849 DEBUG [IPC Server handler 11 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 11 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#850 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:29,849 DEBUG [IPC Server handler 11 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:29,850 DEBUG [IPC Server handler 11 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:28:29,850 DEBUG [IPC Server handler 11 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 1 procesingTime= 0 2015-11-22 19:28:29,850 DEBUG [IPC Server handler 11 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 11 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#850 Retry#0 2015-11-22 19:28:29,850 DEBUG [IPC Server handler 11 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 11 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#850 Retry#0 Wrote 359 bytes. 2015-11-22 19:28:29,850 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #850 2015-11-22 19:28:29,850 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 1ms 2015-11-22 19:28:29,851 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:28:30,177 DEBUG [IPC Server idle connection scanner for port 50663] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 50663: task running 2015-11-22 19:28:30,184 DEBUG [IPC Server idle connection scanner for port 47651] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 47651: task running 2015-11-22 19:28:30,201 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #31 2015-11-22 19:28:30,201 DEBUG [IPC Server handler 24 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 24 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#31 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:30,201 DEBUG [IPC Server handler 24 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:30,201 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:30,201 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:28:30,202 DEBUG [IPC Server handler 24 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:30,202 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:30,202 DEBUG [IPC Server handler 24 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 1 2015-11-22 19:28:30,202 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:30,202 DEBUG [IPC Server handler 24 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 24 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#31 Retry#0 2015-11-22 19:28:30,202 DEBUG [IPC Server handler 24 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 24 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#31 Retry#0 Wrote 47 bytes. 2015-11-22 19:28:30,317 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:28:30,317 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:28:30,317 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:30,317 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:28:30,317 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:28:30,317 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:28:30,317 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:28:30,318 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:28:30,318 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:28:30,851 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #851 2015-11-22 19:28:30,851 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #851 2015-11-22 19:28:30,852 DEBUG [IPC Server handler 32 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 32 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#851 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:30,852 DEBUG [IPC Server handler 32 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:30,852 DEBUG [IPC Server handler 32 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:28:30,852 DEBUG [IPC Server handler 32 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 1 procesingTime= 0 2015-11-22 19:28:30,852 DEBUG [IPC Server handler 32 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 32 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#851 Retry#0 2015-11-22 19:28:30,853 DEBUG [IPC Server handler 32 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 32 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#851 Retry#0 Wrote 359 bytes. 2015-11-22 19:28:30,853 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #851 2015-11-22 19:28:30,853 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:28:30,853 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:28:31,178 DEBUG [IPC Server idle connection scanner for port 32936] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 32936: task running 2015-11-22 19:28:31,204 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #32 2015-11-22 19:28:31,204 DEBUG [IPC Server handler 25 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 25 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#32 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:31,204 DEBUG [IPC Server handler 25 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:31,205 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:31,205 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:28:31,205 DEBUG [IPC Server handler 25 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:31,205 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:31,205 DEBUG [IPC Server handler 25 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 1 2015-11-22 19:28:31,205 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:31,205 DEBUG [IPC Server handler 25 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 25 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#32 Retry#0 2015-11-22 19:28:31,205 DEBUG [IPC Server handler 25 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 25 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#32 Retry#0 Wrote 47 bytes. 2015-11-22 19:28:31,317 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:28:31,317 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:28:31,318 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:31,318 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:28:31,318 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:28:31,318 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:28:31,318 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:28:31,318 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:28:31,318 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:28:31,853 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #852 2015-11-22 19:28:31,854 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #852 2015-11-22 19:28:31,854 DEBUG [IPC Server handler 34 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 34 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#852 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:31,854 DEBUG [IPC Server handler 34 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:31,854 DEBUG [IPC Server handler 34 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:28:31,854 DEBUG [IPC Server handler 34 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 0 2015-11-22 19:28:31,855 DEBUG [IPC Server handler 34 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 34 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#852 Retry#0 2015-11-22 19:28:31,855 DEBUG [IPC Server handler 34 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 34 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#852 Retry#0 Wrote 359 bytes. 2015-11-22 19:28:31,855 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #852 2015-11-22 19:28:31,855 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:28:31,855 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:28:32,207 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #33 2015-11-22 19:28:32,207 DEBUG [IPC Server handler 26 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 26 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#33 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:32,207 DEBUG [IPC Server handler 26 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:32,208 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:32,208 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:28:32,208 DEBUG [IPC Server handler 26 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:32,208 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:32,208 DEBUG [IPC Server handler 26 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 1 2015-11-22 19:28:32,208 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:32,208 DEBUG [IPC Server handler 26 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 26 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#33 Retry#0 2015-11-22 19:28:32,209 DEBUG [IPC Server handler 26 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 26 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#33 Retry#0 Wrote 47 bytes. 2015-11-22 19:28:32,318 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:28:32,318 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:28:32,318 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:32,318 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:28:32,318 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:28:32,318 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:28:32,318 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:28:32,319 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:28:32,319 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:28:32,856 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #853 2015-11-22 19:28:32,856 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #853 2015-11-22 19:28:32,856 DEBUG [IPC Server handler 37 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 37 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#853 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:32,856 DEBUG [IPC Server handler 37 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:32,856 DEBUG [IPC Server handler 37 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:28:32,856 DEBUG [IPC Server handler 37 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 0 2015-11-22 19:28:32,857 DEBUG [IPC Server handler 37 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 37 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#853 Retry#0 2015-11-22 19:28:32,857 DEBUG [IPC Server handler 37 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 37 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#853 Retry#0 Wrote 359 bytes. 2015-11-22 19:28:32,857 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #853 2015-11-22 19:28:32,857 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:28:32,857 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:28:33,210 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #34 2015-11-22 19:28:33,211 DEBUG [IPC Server handler 27 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 27 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#34 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:33,211 DEBUG [IPC Server handler 27 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:33,212 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:33,212 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:28:33,212 DEBUG [IPC Server handler 27 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:33,212 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:33,212 DEBUG [IPC Server handler 27 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 1 procesingTime= 1 2015-11-22 19:28:33,212 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:33,212 DEBUG [IPC Server handler 27 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 27 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#34 Retry#0 2015-11-22 19:28:33,213 DEBUG [IPC Server handler 27 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 27 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#34 Retry#0 Wrote 47 bytes. 2015-11-22 19:28:33,318 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:28:33,318 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:28:33,319 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:33,319 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:28:33,319 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:28:33,319 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:28:33,319 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:28:33,319 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:28:33,319 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:28:33,857 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #854 2015-11-22 19:28:33,858 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #854 2015-11-22 19:28:33,858 DEBUG [IPC Server handler 38 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 38 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#854 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:33,858 DEBUG [IPC Server handler 38 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:33,858 DEBUG [IPC Server handler 38 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:28:33,859 DEBUG [IPC Server handler 38 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 1 2015-11-22 19:28:33,859 DEBUG [IPC Server handler 38 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 38 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#854 Retry#0 2015-11-22 19:28:33,859 DEBUG [IPC Server handler 38 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 38 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#854 Retry#0 Wrote 359 bytes. 2015-11-22 19:28:33,859 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #854 2015-11-22 19:28:33,859 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:28:33,859 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:28:34,214 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #35 2015-11-22 19:28:34,214 DEBUG [IPC Server handler 28 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 28 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#35 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:34,214 DEBUG [IPC Server handler 28 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:34,215 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:34,215 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:28:34,215 DEBUG [IPC Server handler 28 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:34,216 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:34,216 DEBUG [IPC Server handler 28 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 1 procesingTime= 1 2015-11-22 19:28:34,216 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:34,216 DEBUG [IPC Server handler 28 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 28 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#35 Retry#0 2015-11-22 19:28:34,216 DEBUG [IPC Server handler 28 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 28 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#35 Retry#0 Wrote 47 bytes. 2015-11-22 19:28:34,319 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:28:34,319 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:28:34,319 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:34,319 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:28:34,319 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:28:34,319 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:28:34,320 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:28:34,320 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:28:34,320 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:28:34,860 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #855 2015-11-22 19:28:34,860 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #855 2015-11-22 19:28:34,860 DEBUG [IPC Server handler 39 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 39 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#855 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:34,860 DEBUG [IPC Server handler 39 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:34,860 DEBUG [IPC Server handler 39 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:28:34,861 DEBUG [IPC Server handler 39 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 1 2015-11-22 19:28:34,861 DEBUG [IPC Server handler 39 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 39 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#855 Retry#0 2015-11-22 19:28:34,861 DEBUG [IPC Server handler 39 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 39 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#855 Retry#0 Wrote 359 bytes. 2015-11-22 19:28:34,861 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #855 2015-11-22 19:28:34,861 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:28:34,861 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:28:35,217 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #36 2015-11-22 19:28:35,218 DEBUG [IPC Server handler 32 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 32 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#36 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:35,218 DEBUG [IPC Server handler 32 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:35,219 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:35,219 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:28:35,219 DEBUG [IPC Server handler 32 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:35,219 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:35,219 DEBUG [IPC Server handler 32 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 1 2015-11-22 19:28:35,219 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:35,219 DEBUG [IPC Server handler 32 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 32 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#36 Retry#0 2015-11-22 19:28:35,220 DEBUG [IPC Server handler 32 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 32 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#36 Retry#0 Wrote 47 bytes. 2015-11-22 19:28:35,319 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:28:35,320 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:28:35,320 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:35,320 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:28:35,320 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:28:35,320 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:28:35,320 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:28:35,320 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:28:35,320 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:28:35,862 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #856 2015-11-22 19:28:35,862 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #856 2015-11-22 19:28:35,862 DEBUG [IPC Server handler 40 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 40 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#856 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:35,862 DEBUG [IPC Server handler 40 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:35,862 DEBUG [IPC Server handler 40 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:28:35,863 DEBUG [IPC Server handler 40 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 1 2015-11-22 19:28:35,863 DEBUG [IPC Server handler 40 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 40 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#856 Retry#0 2015-11-22 19:28:35,863 DEBUG [IPC Server handler 40 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 40 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#856 Retry#0 Wrote 359 bytes. 2015-11-22 19:28:35,863 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #856 2015-11-22 19:28:35,863 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 1ms 2015-11-22 19:28:35,863 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:28:36,221 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #37 2015-11-22 19:28:36,221 DEBUG [IPC Server handler 29 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 29 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#37 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:36,221 DEBUG [IPC Server handler 29 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:36,222 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:36,222 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:28:36,223 DEBUG [IPC Server handler 29 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:36,223 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:36,223 DEBUG [IPC Server handler 29 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 2 2015-11-22 19:28:36,223 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:36,223 DEBUG [IPC Server handler 29 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 29 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#37 Retry#0 2015-11-22 19:28:36,223 DEBUG [IPC Server handler 29 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 29 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#37 Retry#0 Wrote 47 bytes. 2015-11-22 19:28:36,320 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:28:36,320 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:28:36,320 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:36,320 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:28:36,320 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:28:36,321 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:28:36,321 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:28:36,321 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:28:36,321 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:28:36,864 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #857 2015-11-22 19:28:36,864 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #857 2015-11-22 19:28:36,864 DEBUG [IPC Server handler 41 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 41 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#857 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:36,864 DEBUG [IPC Server handler 41 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:36,864 DEBUG [IPC Server handler 41 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:28:36,865 DEBUG [IPC Server handler 41 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 1 2015-11-22 19:28:36,865 DEBUG [IPC Server handler 41 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 41 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#857 Retry#0 2015-11-22 19:28:36,865 DEBUG [IPC Server handler 41 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 41 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#857 Retry#0 Wrote 360 bytes. 2015-11-22 19:28:36,865 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #857 2015-11-22 19:28:36,865 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 1ms 2015-11-22 19:28:36,866 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:28:37,225 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #38 2015-11-22 19:28:37,225 DEBUG [IPC Server handler 16 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 16 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#38 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:37,225 DEBUG [IPC Server handler 16 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:37,225 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:37,226 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:28:37,226 DEBUG [IPC Server handler 16 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:37,226 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:37,226 DEBUG [IPC Server handler 16 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 1 2015-11-22 19:28:37,226 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:37,226 DEBUG [IPC Server handler 16 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 16 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#38 Retry#0 2015-11-22 19:28:37,226 DEBUG [IPC Server handler 16 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 16 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#38 Retry#0 Wrote 47 bytes. 2015-11-22 19:28:37,321 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:28:37,321 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:28:37,321 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:37,321 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:28:37,321 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:28:37,321 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:28:37,321 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:28:37,321 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:28:37,321 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:28:37,866 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #858 2015-11-22 19:28:37,866 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #858 2015-11-22 19:28:37,866 DEBUG [IPC Server handler 42 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 42 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#858 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:37,866 DEBUG [IPC Server handler 42 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:37,867 DEBUG [IPC Server handler 42 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:28:37,867 DEBUG [IPC Server handler 42 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 1 2015-11-22 19:28:37,867 DEBUG [IPC Server handler 42 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 42 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#858 Retry#0 2015-11-22 19:28:37,867 DEBUG [IPC Server handler 42 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 42 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#858 Retry#0 Wrote 360 bytes. 2015-11-22 19:28:37,867 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #858 2015-11-22 19:28:37,867 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 1ms 2015-11-22 19:28:37,868 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:28:38,228 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #39 2015-11-22 19:28:38,228 DEBUG [IPC Server handler 31 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 31 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#39 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:38,228 DEBUG [IPC Server handler 31 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:38,228 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:38,229 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:28:38,229 DEBUG [IPC Server handler 31 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:38,229 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:38,229 DEBUG [IPC Server handler 31 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 1 2015-11-22 19:28:38,229 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:38,229 DEBUG [IPC Server handler 31 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 31 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#39 Retry#0 2015-11-22 19:28:38,229 DEBUG [IPC Server handler 31 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 31 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#39 Retry#0 Wrote 47 bytes. 2015-11-22 19:28:38,321 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:28:38,321 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:28:38,321 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:38,322 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:28:38,322 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:28:38,322 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:28:38,322 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:28:38,322 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:28:38,322 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:28:38,679 DEBUG [IPC Server idle connection scanner for port 48629] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 48629: task running 2015-11-22 19:28:38,685 DEBUG [IPC Server idle connection scanner for port 46284] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 46284: task running 2015-11-22 19:28:38,692 DEBUG [IPC Server idle connection scanner for port 52028] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 52028: task running 2015-11-22 19:28:38,868 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #859 2015-11-22 19:28:38,868 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #859 2015-11-22 19:28:38,868 DEBUG [IPC Server handler 43 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 43 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#859 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:38,869 DEBUG [IPC Server handler 43 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:38,869 DEBUG [IPC Server handler 43 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:28:38,869 DEBUG [IPC Server handler 43 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 1 procesingTime= 0 2015-11-22 19:28:38,869 DEBUG [IPC Server handler 43 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 43 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#859 Retry#0 2015-11-22 19:28:38,870 DEBUG [IPC Server handler 43 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 43 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#859 Retry#0 Wrote 360 bytes. 2015-11-22 19:28:38,870 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #859 2015-11-22 19:28:38,870 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:28:38,870 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:28:38,949 DEBUG [IPC Server idle connection scanner for port 53669] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 53669: task running 2015-11-22 19:28:39,231 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #40 2015-11-22 19:28:39,231 DEBUG [IPC Server handler 30 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 30 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#40 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:39,231 DEBUG [IPC Server handler 30 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:39,232 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:39,232 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:28:39,232 DEBUG [IPC Server handler 30 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:39,232 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:39,232 DEBUG [IPC Server handler 30 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 1 2015-11-22 19:28:39,232 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:39,232 DEBUG [IPC Server handler 30 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 30 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#40 Retry#0 2015-11-22 19:28:39,233 DEBUG [IPC Server handler 30 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 30 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#40 Retry#0 Wrote 47 bytes. 2015-11-22 19:28:39,322 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:28:39,322 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:28:39,322 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:39,322 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:28:39,322 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:28:39,322 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:28:39,322 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:28:39,322 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:28:39,322 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:28:39,870 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #860 2015-11-22 19:28:39,870 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #860 2015-11-22 19:28:39,871 DEBUG [IPC Server handler 45 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 45 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#860 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:39,871 DEBUG [IPC Server handler 45 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:39,871 DEBUG [IPC Server handler 45 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:28:39,871 DEBUG [IPC Server handler 45 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 0 2015-11-22 19:28:39,872 DEBUG [IPC Server handler 45 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 45 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#860 Retry#0 2015-11-22 19:28:39,872 DEBUG [IPC Server handler 45 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 45 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#860 Retry#0 Wrote 360 bytes. 2015-11-22 19:28:39,872 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #860 2015-11-22 19:28:39,872 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:28:39,872 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:28:40,177 DEBUG [IPC Server idle connection scanner for port 50663] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 50663: task running 2015-11-22 19:28:40,184 DEBUG [IPC Server idle connection scanner for port 47651] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 47651: task running 2015-11-22 19:28:40,234 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #41 2015-11-22 19:28:40,234 DEBUG [IPC Server handler 33 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 33 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#41 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:40,234 DEBUG [IPC Server handler 33 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:40,235 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:40,235 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:28:40,235 DEBUG [IPC Server handler 33 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:40,235 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:40,235 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:40,235 DEBUG [IPC Server handler 33 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 1 2015-11-22 19:28:40,235 DEBUG [IPC Server handler 33 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 33 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#41 Retry#0 2015-11-22 19:28:40,236 DEBUG [IPC Server handler 33 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 33 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#41 Retry#0 Wrote 47 bytes. 2015-11-22 19:28:40,322 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:28:40,322 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:28:40,323 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:40,323 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:28:40,323 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:28:40,323 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:28:40,323 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:28:40,323 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:28:40,323 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:28:40,872 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #861 2015-11-22 19:28:40,873 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #861 2015-11-22 19:28:40,873 DEBUG [IPC Server handler 44 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 44 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#861 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:40,873 DEBUG [IPC Server handler 44 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:40,873 DEBUG [IPC Server handler 44 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:28:40,873 DEBUG [IPC Server handler 44 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 0 2015-11-22 19:28:40,874 DEBUG [IPC Server handler 44 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 44 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#861 Retry#0 2015-11-22 19:28:40,874 DEBUG [IPC Server handler 44 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 44 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#861 Retry#0 Wrote 360 bytes. 2015-11-22 19:28:40,874 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #861 2015-11-22 19:28:40,874 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:28:40,874 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:28:41,178 DEBUG [IPC Server idle connection scanner for port 32936] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 32936: task running 2015-11-22 19:28:41,237 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #42 2015-11-22 19:28:41,237 DEBUG [IPC Server handler 36 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 36 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#42 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:41,237 DEBUG [IPC Server handler 36 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:41,238 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:41,238 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:28:41,238 DEBUG [IPC Server handler 36 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:41,239 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:41,239 DEBUG [IPC Server handler 36 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 2 2015-11-22 19:28:41,239 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:41,239 DEBUG [IPC Server handler 36 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 36 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#42 Retry#0 2015-11-22 19:28:41,239 DEBUG [IPC Server handler 36 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 36 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#42 Retry#0 Wrote 47 bytes. 2015-11-22 19:28:41,323 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:28:41,323 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:28:41,323 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:41,323 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:28:41,323 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:28:41,323 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:28:41,323 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:28:41,324 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:28:41,324 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:28:41,874 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #862 2015-11-22 19:28:41,875 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #862 2015-11-22 19:28:41,875 DEBUG [IPC Server handler 47 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 47 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#862 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:41,875 DEBUG [IPC Server handler 47 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:41,875 DEBUG [IPC Server handler 47 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:28:41,875 DEBUG [IPC Server handler 47 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 0 2015-11-22 19:28:41,876 DEBUG [IPC Server handler 47 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 47 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#862 Retry#0 2015-11-22 19:28:41,876 DEBUG [IPC Server handler 47 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 47 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#862 Retry#0 Wrote 360 bytes. 2015-11-22 19:28:41,876 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #862 2015-11-22 19:28:41,876 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:28:41,876 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:28:42,240 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #43 2015-11-22 19:28:42,240 DEBUG [IPC Server handler 35 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 35 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#43 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:42,241 DEBUG [IPC Server handler 35 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:42,241 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:42,241 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:28:42,242 DEBUG [IPC Server handler 35 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:42,242 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:42,242 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:42,242 DEBUG [IPC Server handler 35 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 1 procesingTime= 1 2015-11-22 19:28:42,242 DEBUG [IPC Server handler 35 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 35 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#43 Retry#0 2015-11-22 19:28:42,243 DEBUG [IPC Server handler 35 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 35 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#43 Retry#0 Wrote 47 bytes. 2015-11-22 19:28:42,323 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:28:42,323 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:28:42,324 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:42,324 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:28:42,324 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:28:42,324 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:28:42,324 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:28:42,324 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:28:42,324 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:28:42,876 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #863 2015-11-22 19:28:42,877 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #863 2015-11-22 19:28:42,877 DEBUG [IPC Server handler 46 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 46 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#863 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:42,877 DEBUG [IPC Server handler 46 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:42,877 DEBUG [IPC Server handler 46 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:28:42,877 DEBUG [IPC Server handler 46 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 0 2015-11-22 19:28:42,878 DEBUG [IPC Server handler 46 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 46 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#863 Retry#0 2015-11-22 19:28:42,878 DEBUG [IPC Server handler 46 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 46 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#863 Retry#0 Wrote 360 bytes. 2015-11-22 19:28:42,878 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #863 2015-11-22 19:28:42,878 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:28:42,878 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:28:43,243 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #44 2015-11-22 19:28:43,244 DEBUG [IPC Server handler 34 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 34 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#44 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:43,244 DEBUG [IPC Server handler 34 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:43,245 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:43,245 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:28:43,245 DEBUG [IPC Server handler 34 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:43,245 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:43,245 DEBUG [IPC Server handler 34 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 1 2015-11-22 19:28:43,245 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:43,245 DEBUG [IPC Server handler 34 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 34 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#44 Retry#0 2015-11-22 19:28:43,245 DEBUG [IPC Server handler 34 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 34 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#44 Retry#0 Wrote 47 bytes. 2015-11-22 19:28:43,324 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:28:43,324 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:28:43,324 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:43,324 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:28:43,324 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:28:43,325 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:28:43,325 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:28:43,325 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:28:43,325 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:28:43,878 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #864 2015-11-22 19:28:43,879 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #864 2015-11-22 19:28:43,879 DEBUG [IPC Server handler 48 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 48 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#864 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:43,879 DEBUG [IPC Server handler 48 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:43,879 DEBUG [IPC Server handler 48 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:28:43,879 DEBUG [IPC Server handler 48 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 0 2015-11-22 19:28:43,880 DEBUG [IPC Server handler 48 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 48 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#864 Retry#0 2015-11-22 19:28:43,880 DEBUG [IPC Server handler 48 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 48 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#864 Retry#0 Wrote 360 bytes. 2015-11-22 19:28:43,880 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #864 2015-11-22 19:28:43,880 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:28:43,880 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:28:44,247 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #45 2015-11-22 19:28:44,247 DEBUG [IPC Server handler 38 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 38 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#45 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:44,247 DEBUG [IPC Server handler 38 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:44,248 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:44,248 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:28:44,248 DEBUG [IPC Server handler 38 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:44,248 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:44,248 DEBUG [IPC Server handler 38 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 1 2015-11-22 19:28:44,248 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:44,248 DEBUG [IPC Server handler 38 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 38 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#45 Retry#0 2015-11-22 19:28:44,249 DEBUG [IPC Server handler 38 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 38 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#45 Retry#0 Wrote 47 bytes. 2015-11-22 19:28:44,324 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:28:44,325 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:28:44,325 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:44,325 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:28:44,325 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:28:44,325 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:28:44,325 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:28:44,325 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:28:44,325 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:28:44,880 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #865 2015-11-22 19:28:44,881 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #865 2015-11-22 19:28:44,881 DEBUG [IPC Server handler 49 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 49 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#865 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:44,881 DEBUG [IPC Server handler 49 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:44,881 DEBUG [IPC Server handler 49 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:28:44,882 DEBUG [IPC Server handler 49 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 1 2015-11-22 19:28:44,882 DEBUG [IPC Server handler 49 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 49 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#865 Retry#0 2015-11-22 19:28:44,882 DEBUG [IPC Server handler 49 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 49 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#865 Retry#0 Wrote 360 bytes. 2015-11-22 19:28:44,882 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #865 2015-11-22 19:28:44,882 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:28:44,882 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:28:45,250 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #46 2015-11-22 19:28:45,250 DEBUG [IPC Server handler 39 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 39 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#46 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:45,250 DEBUG [IPC Server handler 39 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:45,251 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:45,251 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:28:45,251 DEBUG [IPC Server handler 39 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:45,251 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:45,251 DEBUG [IPC Server handler 39 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 1 2015-11-22 19:28:45,251 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:45,251 DEBUG [IPC Server handler 39 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 39 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#46 Retry#0 2015-11-22 19:28:45,252 DEBUG [IPC Server handler 39 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 39 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#46 Retry#0 Wrote 47 bytes. 2015-11-22 19:28:45,325 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:28:45,325 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:28:45,325 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:45,325 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:28:45,326 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:28:45,326 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:28:45,326 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:28:45,326 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:28:45,326 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:28:45,883 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #866 2015-11-22 19:28:45,883 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #866 2015-11-22 19:28:45,883 DEBUG [IPC Server handler 33 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 33 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#866 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:45,883 DEBUG [IPC Server handler 33 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:45,883 DEBUG [IPC Server handler 33 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:28:45,884 DEBUG [IPC Server handler 33 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 1 2015-11-22 19:28:45,884 DEBUG [IPC Server handler 33 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 33 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#866 Retry#0 2015-11-22 19:28:45,884 DEBUG [IPC Server handler 33 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 33 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#866 Retry#0 Wrote 360 bytes. 2015-11-22 19:28:45,884 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #866 2015-11-22 19:28:45,884 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:28:45,884 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:28:46,253 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #47 2015-11-22 19:28:46,253 DEBUG [IPC Server handler 40 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 40 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#47 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:46,253 DEBUG [IPC Server handler 40 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:46,254 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:46,254 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:28:46,254 DEBUG [IPC Server handler 40 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:46,254 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:46,254 DEBUG [IPC Server handler 40 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 1 2015-11-22 19:28:46,254 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:46,254 DEBUG [IPC Server handler 40 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 40 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#47 Retry#0 2015-11-22 19:28:46,255 DEBUG [IPC Server handler 40 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 40 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#47 Retry#0 Wrote 47 bytes. 2015-11-22 19:28:46,326 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:28:46,326 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:28:46,326 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:46,326 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:28:46,326 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:28:46,326 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:28:46,326 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:28:46,326 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:28:46,326 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:28:46,885 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #867 2015-11-22 19:28:46,885 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #867 2015-11-22 19:28:46,885 DEBUG [IPC Server handler 35 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 35 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#867 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:46,885 DEBUG [IPC Server handler 35 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:46,885 DEBUG [IPC Server handler 35 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:28:46,886 DEBUG [IPC Server handler 35 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 1 2015-11-22 19:28:46,886 DEBUG [IPC Server handler 35 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 35 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#867 Retry#0 2015-11-22 19:28:46,886 DEBUG [IPC Server handler 35 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 35 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#867 Retry#0 Wrote 360 bytes. 2015-11-22 19:28:46,886 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #867 2015-11-22 19:28:46,886 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 1ms 2015-11-22 19:28:46,886 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:28:47,256 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #48 2015-11-22 19:28:47,256 DEBUG [IPC Server handler 41 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 41 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#48 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:47,256 DEBUG [IPC Server handler 41 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:47,258 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:47,258 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:28:47,258 DEBUG [IPC Server handler 41 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:47,258 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:47,258 DEBUG [IPC Server handler 41 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 2 2015-11-22 19:28:47,258 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:47,259 DEBUG [IPC Server handler 41 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 41 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#48 Retry#0 2015-11-22 19:28:47,259 DEBUG [IPC Server handler 41 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 41 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#48 Retry#0 Wrote 47 bytes. 2015-11-22 19:28:47,326 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:28:47,326 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:28:47,326 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:47,326 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:28:47,327 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:28:47,327 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:28:47,327 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:28:47,327 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:28:47,327 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:28:47,887 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #868 2015-11-22 19:28:47,887 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #868 2015-11-22 19:28:47,887 DEBUG [IPC Server handler 36 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 36 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#868 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:47,887 DEBUG [IPC Server handler 36 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:47,888 DEBUG [IPC Server handler 36 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:28:47,888 DEBUG [IPC Server handler 36 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 1 2015-11-22 19:28:47,888 DEBUG [IPC Server handler 36 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 36 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#868 Retry#0 2015-11-22 19:28:47,888 DEBUG [IPC Server handler 36 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 36 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#868 Retry#0 Wrote 360 bytes. 2015-11-22 19:28:47,888 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #868 2015-11-22 19:28:47,888 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 1ms 2015-11-22 19:28:47,889 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:28:48,260 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #49 2015-11-22 19:28:48,260 DEBUG [IPC Server handler 43 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 43 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#49 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:48,260 DEBUG [IPC Server handler 43 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:48,260 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:48,260 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:28:48,261 DEBUG [IPC Server handler 43 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:48,261 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:48,261 DEBUG [IPC Server handler 43 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 1 2015-11-22 19:28:48,261 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:48,261 DEBUG [IPC Server handler 43 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 43 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#49 Retry#0 2015-11-22 19:28:48,261 DEBUG [IPC Server handler 43 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 43 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#49 Retry#0 Wrote 47 bytes. 2015-11-22 19:28:48,327 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:28:48,327 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:28:48,327 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:48,327 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:28:48,327 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:28:48,327 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:28:48,327 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:28:48,327 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:28:48,327 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:28:48,679 DEBUG [IPC Server idle connection scanner for port 48629] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 48629: task running 2015-11-22 19:28:48,686 DEBUG [IPC Server idle connection scanner for port 46284] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 46284: task running 2015-11-22 19:28:48,692 DEBUG [IPC Server idle connection scanner for port 52028] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 52028: task running 2015-11-22 19:28:48,889 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #869 2015-11-22 19:28:48,889 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #869 2015-11-22 19:28:48,889 DEBUG [IPC Server handler 0 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 0 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#869 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:48,889 DEBUG [IPC Server handler 0 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:48,890 DEBUG [IPC Server handler 0 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:28:48,890 DEBUG [IPC Server handler 0 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 1 2015-11-22 19:28:48,890 DEBUG [IPC Server handler 0 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 0 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#869 Retry#0 2015-11-22 19:28:48,890 DEBUG [IPC Server handler 0 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 0 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#869 Retry#0 Wrote 360 bytes. 2015-11-22 19:28:48,890 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #869 2015-11-22 19:28:48,890 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 1ms 2015-11-22 19:28:48,891 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:28:48,949 DEBUG [IPC Server idle connection scanner for port 53669] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 53669: task running 2015-11-22 19:28:49,262 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #50 2015-11-22 19:28:49,263 DEBUG [IPC Server handler 44 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 44 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#50 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:49,263 DEBUG [IPC Server handler 44 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:49,263 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:49,263 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:28:49,263 DEBUG [IPC Server handler 44 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:49,263 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:49,263 DEBUG [IPC Server handler 44 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 1 procesingTime= 0 2015-11-22 19:28:49,263 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:49,263 DEBUG [IPC Server handler 44 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 44 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#50 Retry#0 2015-11-22 19:28:49,264 DEBUG [IPC Server handler 44 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 44 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#50 Retry#0 Wrote 47 bytes. 2015-11-22 19:28:49,327 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:28:49,327 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:28:49,327 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:49,328 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:28:49,328 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:28:49,328 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:28:49,328 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:28:49,328 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:28:49,328 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:28:49,891 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #870 2015-11-22 19:28:49,891 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #870 2015-11-22 19:28:49,891 DEBUG [IPC Server handler 1 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 1 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#870 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:49,891 DEBUG [IPC Server handler 1 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:49,892 DEBUG [IPC Server handler 1 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:28:49,892 DEBUG [IPC Server handler 1 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 1 procesingTime= 0 2015-11-22 19:28:49,892 DEBUG [IPC Server handler 1 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 1 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#870 Retry#0 2015-11-22 19:28:49,892 DEBUG [IPC Server handler 1 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 1 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#870 Retry#0 Wrote 360 bytes. 2015-11-22 19:28:49,892 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #870 2015-11-22 19:28:49,892 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 1ms 2015-11-22 19:28:49,893 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:28:50,177 DEBUG [IPC Server idle connection scanner for port 50663] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 50663: task running 2015-11-22 19:28:50,184 DEBUG [IPC Server idle connection scanner for port 47651] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 47651: task running 2015-11-22 19:28:50,265 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #51 2015-11-22 19:28:50,265 DEBUG [IPC Server handler 45 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 45 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#51 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:50,265 DEBUG [IPC Server handler 45 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:50,265 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:50,266 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:28:50,266 DEBUG [IPC Server handler 45 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:50,266 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:50,266 DEBUG [IPC Server handler 45 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 1 2015-11-22 19:28:50,266 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:50,266 DEBUG [IPC Server handler 45 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 45 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#51 Retry#0 2015-11-22 19:28:50,266 DEBUG [IPC Server handler 45 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 45 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#51 Retry#0 Wrote 47 bytes. 2015-11-22 19:28:50,328 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:28:50,328 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:28:50,328 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:50,328 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:28:50,328 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:28:50,328 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:28:50,328 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:28:50,328 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:28:50,329 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:28:50,893 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #871 2015-11-22 19:28:50,893 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #871 2015-11-22 19:28:50,893 DEBUG [IPC Server handler 4 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 4 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#871 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:50,893 DEBUG [IPC Server handler 4 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:50,894 DEBUG [IPC Server handler 4 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:28:50,894 DEBUG [IPC Server handler 4 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 1 procesingTime= 0 2015-11-22 19:28:50,894 DEBUG [IPC Server handler 4 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 4 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#871 Retry#0 2015-11-22 19:28:50,894 DEBUG [IPC Server handler 4 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 4 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#871 Retry#0 Wrote 360 bytes. 2015-11-22 19:28:50,894 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #871 2015-11-22 19:28:50,895 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:28:50,895 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:28:51,178 DEBUG [IPC Server idle connection scanner for port 32936] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 32936: task running 2015-11-22 19:28:51,267 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #52 2015-11-22 19:28:51,268 DEBUG [IPC Server handler 46 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 46 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#52 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:51,268 DEBUG [IPC Server handler 46 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:51,268 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:51,268 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:28:51,268 DEBUG [IPC Server handler 46 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:51,268 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:51,268 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:51,268 DEBUG [IPC Server handler 46 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 0 2015-11-22 19:28:51,269 DEBUG [IPC Server handler 46 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 46 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#52 Retry#0 2015-11-22 19:28:51,269 DEBUG [IPC Server handler 46 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 46 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#52 Retry#0 Wrote 47 bytes. 2015-11-22 19:28:51,328 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:28:51,328 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:28:51,328 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:51,329 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:28:51,329 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:28:51,329 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:28:51,329 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:28:51,329 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:28:51,329 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:28:51,895 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #872 2015-11-22 19:28:51,895 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #872 2015-11-22 19:28:51,896 DEBUG [IPC Server handler 2 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 2 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#872 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:51,896 DEBUG [IPC Server handler 2 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:51,896 DEBUG [IPC Server handler 2 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:28:51,896 DEBUG [IPC Server handler 2 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 1 procesingTime= 0 2015-11-22 19:28:51,896 DEBUG [IPC Server handler 2 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 2 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#872 Retry#0 2015-11-22 19:28:51,897 DEBUG [IPC Server handler 2 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 2 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#872 Retry#0 Wrote 360 bytes. 2015-11-22 19:28:51,897 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #872 2015-11-22 19:28:51,897 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:28:51,897 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:28:52,270 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #53 2015-11-22 19:28:52,270 DEBUG [IPC Server handler 47 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 47 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#53 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:52,270 DEBUG [IPC Server handler 47 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:52,271 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:52,271 DEBUG [IPC Server handler 47 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:52,271 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:28:52,271 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:52,271 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:52,271 DEBUG [IPC Server handler 47 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 1 procesingTime= 0 2015-11-22 19:28:52,271 DEBUG [IPC Server handler 47 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 47 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#53 Retry#0 2015-11-22 19:28:52,272 DEBUG [IPC Server handler 47 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 47 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#53 Retry#0 Wrote 47 bytes. 2015-11-22 19:28:52,329 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:28:52,329 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:28:52,329 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:52,329 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:28:52,329 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:28:52,329 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:28:52,329 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:28:52,330 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:28:52,330 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:28:52,897 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #873 2015-11-22 19:28:52,898 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #873 2015-11-22 19:28:52,898 DEBUG [IPC Server handler 3 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 3 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#873 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:52,898 DEBUG [IPC Server handler 3 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:52,898 DEBUG [IPC Server handler 3 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:28:52,898 DEBUG [IPC Server handler 3 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 0 2015-11-22 19:28:52,898 DEBUG [IPC Server handler 3 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 3 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#873 Retry#0 2015-11-22 19:28:52,899 DEBUG [IPC Server handler 3 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 3 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#873 Retry#0 Wrote 360 bytes. 2015-11-22 19:28:52,899 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #873 2015-11-22 19:28:52,899 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:28:52,899 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:28:53,273 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #54 2015-11-22 19:28:53,273 DEBUG [IPC Server handler 48 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 48 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#54 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:53,273 DEBUG [IPC Server handler 48 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:53,273 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:53,274 DEBUG [IPC Server handler 48 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:53,274 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:28:53,274 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:53,274 DEBUG [IPC Server handler 48 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 1 2015-11-22 19:28:53,274 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:53,274 DEBUG [IPC Server handler 48 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 48 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#54 Retry#0 2015-11-22 19:28:53,274 DEBUG [IPC Server handler 48 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 48 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#54 Retry#0 Wrote 47 bytes. 2015-11-22 19:28:53,329 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:28:53,329 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:28:53,330 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:53,330 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:28:53,330 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:28:53,330 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:28:53,330 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:28:53,330 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:28:53,330 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:28:53,899 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #874 2015-11-22 19:28:53,900 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #874 2015-11-22 19:28:53,900 DEBUG [IPC Server handler 5 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 5 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#874 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:53,900 DEBUG [IPC Server handler 5 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:53,900 DEBUG [IPC Server handler 5 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:28:53,901 DEBUG [IPC Server handler 5 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 1 2015-11-22 19:28:53,901 DEBUG [IPC Server handler 5 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 5 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#874 Retry#0 2015-11-22 19:28:53,901 DEBUG [IPC Server handler 5 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 5 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#874 Retry#0 Wrote 360 bytes. 2015-11-22 19:28:53,901 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #874 2015-11-22 19:28:53,901 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:28:53,901 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:28:54,276 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #55 2015-11-22 19:28:54,276 DEBUG [IPC Server handler 49 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 49 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#55 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:54,276 DEBUG [IPC Server handler 49 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:54,276 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:54,276 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:28:54,276 DEBUG [IPC Server handler 49 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:54,277 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:54,277 DEBUG [IPC Server handler 49 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 1 2015-11-22 19:28:54,277 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:54,277 DEBUG [IPC Server handler 49 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 49 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#55 Retry#0 2015-11-22 19:28:54,277 DEBUG [IPC Server handler 49 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 49 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#55 Retry#0 Wrote 47 bytes. 2015-11-22 19:28:54,330 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:28:54,330 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:28:54,330 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:54,330 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:28:54,330 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:28:54,330 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:28:54,331 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:28:54,331 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:28:54,331 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:28:54,902 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #875 2015-11-22 19:28:54,902 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #875 2015-11-22 19:28:54,902 DEBUG [IPC Server handler 6 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 6 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#875 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:54,902 DEBUG [IPC Server handler 6 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:54,902 DEBUG [IPC Server handler 6 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:28:54,903 DEBUG [IPC Server handler 6 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 1 2015-11-22 19:28:54,903 DEBUG [IPC Server handler 6 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 6 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#875 Retry#0 2015-11-22 19:28:54,903 DEBUG [IPC Server handler 6 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 6 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#875 Retry#0 Wrote 360 bytes. 2015-11-22 19:28:54,903 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #875 2015-11-22 19:28:54,903 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 1ms 2015-11-22 19:28:54,903 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:28:55,278 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #56 2015-11-22 19:28:55,279 DEBUG [IPC Server handler 42 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 42 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#56 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:55,279 DEBUG [IPC Server handler 42 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:55,279 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:55,279 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:28:55,279 DEBUG [IPC Server handler 42 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:55,279 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:55,280 DEBUG [IPC Server handler 42 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 1 2015-11-22 19:28:55,280 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:55,280 DEBUG [IPC Server handler 42 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 42 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#56 Retry#0 2015-11-22 19:28:55,280 DEBUG [IPC Server handler 42 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 42 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#56 Retry#0 Wrote 47 bytes. 2015-11-22 19:28:55,330 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:28:55,330 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:28:55,331 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:55,331 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:28:55,331 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:28:55,331 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:28:55,331 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:28:55,331 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:28:55,331 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:28:55,904 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #876 2015-11-22 19:28:55,904 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #876 2015-11-22 19:28:55,904 DEBUG [IPC Server handler 7 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 7 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#876 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:55,904 DEBUG [IPC Server handler 7 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:55,905 DEBUG [IPC Server handler 7 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:28:55,905 DEBUG [IPC Server handler 7 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 1 2015-11-22 19:28:55,905 DEBUG [IPC Server handler 7 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 7 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#876 Retry#0 2015-11-22 19:28:55,905 DEBUG [IPC Server handler 7 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 7 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#876 Retry#0 Wrote 360 bytes. 2015-11-22 19:28:55,905 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #876 2015-11-22 19:28:55,905 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 1ms 2015-11-22 19:28:55,906 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:28:56,281 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #57 2015-11-22 19:28:56,281 DEBUG [IPC Server handler 37 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 37 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#57 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:56,282 DEBUG [IPC Server handler 37 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:56,282 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:56,282 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:28:56,282 DEBUG [IPC Server handler 37 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:56,282 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:56,282 DEBUG [IPC Server handler 37 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 1 procesingTime= 0 2015-11-22 19:28:56,282 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:56,282 DEBUG [IPC Server handler 37 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 37 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#57 Retry#0 2015-11-22 19:28:56,283 DEBUG [IPC Server handler 37 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 37 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#57 Retry#0 Wrote 47 bytes. 2015-11-22 19:28:56,331 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:28:56,331 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:28:56,331 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:56,331 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:28:56,331 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:28:56,332 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:28:56,332 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:28:56,332 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:28:56,332 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:28:56,906 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #877 2015-11-22 19:28:56,906 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #877 2015-11-22 19:28:56,906 DEBUG [IPC Server handler 8 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 8 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#877 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:56,906 DEBUG [IPC Server handler 8 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:56,907 DEBUG [IPC Server handler 8 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:28:56,907 DEBUG [IPC Server handler 8 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 1 procesingTime= 0 2015-11-22 19:28:56,907 DEBUG [IPC Server handler 8 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 8 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#877 Retry#0 2015-11-22 19:28:56,908 DEBUG [IPC Server handler 8 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 8 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#877 Retry#0 Wrote 360 bytes. 2015-11-22 19:28:56,908 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #877 2015-11-22 19:28:56,908 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:28:56,908 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:28:57,284 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #58 2015-11-22 19:28:57,285 DEBUG [IPC Server handler 0 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 0 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#58 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:57,285 DEBUG [IPC Server handler 0 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:57,285 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:57,285 DEBUG [IPC Server handler 0 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:57,285 DEBUG [IPC Server handler 0 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 1 procesingTime= 0 2015-11-22 19:28:57,285 DEBUG [IPC Server handler 0 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 0 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#58 Retry#0 2015-11-22 19:28:57,286 DEBUG [IPC Server handler 0 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 0 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#58 Retry#0 Wrote 47 bytes. 2015-11-22 19:28:57,288 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:28:57,288 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:57,288 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:57,332 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:28:57,332 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:28:57,332 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:57,332 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:28:57,332 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:28:57,332 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:28:57,332 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:28:57,332 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:28:57,332 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:28:57,908 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #878 2015-11-22 19:28:57,909 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #878 2015-11-22 19:28:57,909 DEBUG [IPC Server handler 10 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 10 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#878 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:57,909 DEBUG [IPC Server handler 10 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:57,909 DEBUG [IPC Server handler 10 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:28:57,910 DEBUG [IPC Server handler 10 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 1 2015-11-22 19:28:57,910 DEBUG [IPC Server handler 10 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 10 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#878 Retry#0 2015-11-22 19:28:57,910 DEBUG [IPC Server handler 10 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 10 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#878 Retry#0 Wrote 360 bytes. 2015-11-22 19:28:57,910 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #878 2015-11-22 19:28:57,910 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:28:57,910 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:28:58,287 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #59 2015-11-22 19:28:58,287 DEBUG [IPC Server handler 2 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 2 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#59 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:58,287 DEBUG [IPC Server handler 2 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:58,287 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:58,288 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:28:58,288 DEBUG [IPC Server handler 2 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:58,288 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:58,288 DEBUG [IPC Server handler 2 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 1 2015-11-22 19:28:58,288 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:58,288 DEBUG [IPC Server handler 2 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 2 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#59 Retry#0 2015-11-22 19:28:58,288 DEBUG [IPC Server handler 2 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 2 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#59 Retry#0 Wrote 47 bytes. 2015-11-22 19:28:58,332 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:28:58,332 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:28:58,333 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:58,333 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:28:58,333 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:28:58,333 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:28:58,333 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:28:58,333 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:28:58,333 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:28:58,679 DEBUG [IPC Server idle connection scanner for port 48629] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 48629: task running 2015-11-22 19:28:58,686 DEBUG [IPC Server idle connection scanner for port 46284] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 46284: task running 2015-11-22 19:28:58,693 DEBUG [IPC Server idle connection scanner for port 52028] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 52028: task running 2015-11-22 19:28:58,911 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #879 2015-11-22 19:28:58,911 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #879 2015-11-22 19:28:58,911 DEBUG [IPC Server handler 9 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 9 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#879 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:58,911 DEBUG [IPC Server handler 9 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:58,911 DEBUG [IPC Server handler 9 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:28:58,912 DEBUG [IPC Server handler 9 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 1 2015-11-22 19:28:58,912 DEBUG [IPC Server handler 9 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 9 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#879 Retry#0 2015-11-22 19:28:58,912 DEBUG [IPC Server handler 9 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 9 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#879 Retry#0 Wrote 360 bytes. 2015-11-22 19:28:58,912 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #879 2015-11-22 19:28:58,912 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:28:58,913 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:28:58,949 DEBUG [IPC Server idle connection scanner for port 53669] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 53669: task running 2015-11-22 19:28:59,290 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #60 2015-11-22 19:28:59,290 DEBUG [IPC Server handler 5 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 5 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#60 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:59,290 DEBUG [IPC Server handler 5 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:59,290 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:59,291 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:28:59,291 DEBUG [IPC Server handler 5 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:28:59,291 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:59,291 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:28:59,291 DEBUG [IPC Server handler 5 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 1 2015-11-22 19:28:59,291 DEBUG [IPC Server handler 5 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 5 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#60 Retry#0 2015-11-22 19:28:59,291 DEBUG [IPC Server handler 5 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 5 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#60 Retry#0 Wrote 47 bytes. 2015-11-22 19:28:59,333 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:28:59,333 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:28:59,333 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:28:59,333 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:28:59,333 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:28:59,333 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:28:59,333 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:28:59,334 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:28:59,334 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:28:59,913 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #880 2015-11-22 19:28:59,913 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #880 2015-11-22 19:28:59,914 DEBUG [IPC Server handler 12 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 12 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#880 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:28:59,914 DEBUG [IPC Server handler 12 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:28:59,914 DEBUG [IPC Server handler 12 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:28:59,914 DEBUG [IPC Server handler 12 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 1 procesingTime= 0 2015-11-22 19:28:59,914 DEBUG [IPC Server handler 12 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 12 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#880 Retry#0 2015-11-22 19:28:59,915 DEBUG [IPC Server handler 12 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 12 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#880 Retry#0 Wrote 360 bytes. 2015-11-22 19:28:59,915 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #880 2015-11-22 19:28:59,915 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:28:59,915 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:29:00,178 DEBUG [IPC Server idle connection scanner for port 50663] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 50663: task running 2015-11-22 19:29:00,184 DEBUG [IPC Server idle connection scanner for port 47651] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 47651: task running 2015-11-22 19:29:00,293 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #61 2015-11-22 19:29:00,293 DEBUG [IPC Server handler 6 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 6 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#61 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:00,293 DEBUG [IPC Server handler 6 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:00,293 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:00,293 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:29:00,294 DEBUG [IPC Server handler 6 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:29:00,294 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:00,294 DEBUG [IPC Server handler 6 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 1 2015-11-22 19:29:00,294 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:00,294 DEBUG [IPC Server handler 6 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 6 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#61 Retry#0 2015-11-22 19:29:00,294 DEBUG [IPC Server handler 6 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 6 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#61 Retry#0 Wrote 47 bytes. 2015-11-22 19:29:00,333 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:29:00,333 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:29:00,334 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:00,334 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:29:00,334 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:29:00,334 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:29:00,334 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:29:00,334 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:29:00,334 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:29:00,915 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #881 2015-11-22 19:29:00,916 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #881 2015-11-22 19:29:00,916 DEBUG [IPC Server handler 13 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 13 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#881 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:00,916 DEBUG [IPC Server handler 13 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:00,916 DEBUG [IPC Server handler 13 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:29:00,917 DEBUG [IPC Server handler 13 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 1 2015-11-22 19:29:00,917 DEBUG [IPC Server handler 13 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 13 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#881 Retry#0 2015-11-22 19:29:00,917 DEBUG [IPC Server handler 13 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 13 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#881 Retry#0 Wrote 360 bytes. 2015-11-22 19:29:00,917 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #881 2015-11-22 19:29:00,917 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:29:00,917 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:29:01,178 DEBUG [IPC Server idle connection scanner for port 32936] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 32936: task running 2015-11-22 19:29:01,295 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #62 2015-11-22 19:29:01,296 DEBUG [IPC Server handler 8 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 8 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#62 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:01,296 DEBUG [IPC Server handler 8 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:01,296 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:01,296 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:29:01,296 DEBUG [IPC Server handler 8 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:29:01,296 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:01,297 DEBUG [IPC Server handler 8 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 1 2015-11-22 19:29:01,297 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:01,297 DEBUG [IPC Server handler 8 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 8 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#62 Retry#0 2015-11-22 19:29:01,297 DEBUG [IPC Server handler 8 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 8 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#62 Retry#0 Wrote 47 bytes. 2015-11-22 19:29:01,334 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:29:01,334 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:29:01,334 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:01,334 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:29:01,334 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:29:01,335 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:29:01,335 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:29:01,335 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:29:01,335 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:29:01,918 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #882 2015-11-22 19:29:01,918 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #882 2015-11-22 19:29:01,918 DEBUG [IPC Server handler 16 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 16 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#882 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:01,918 DEBUG [IPC Server handler 16 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:01,918 DEBUG [IPC Server handler 16 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:29:01,919 DEBUG [IPC Server handler 16 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 1 2015-11-22 19:29:01,919 DEBUG [IPC Server handler 16 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 16 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#882 Retry#0 2015-11-22 19:29:01,919 DEBUG [IPC Server handler 16 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 16 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#882 Retry#0 Wrote 360 bytes. 2015-11-22 19:29:01,919 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #882 2015-11-22 19:29:01,919 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:29:01,919 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:29:02,298 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #63 2015-11-22 19:29:02,298 DEBUG [IPC Server handler 7 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 7 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#63 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:02,298 DEBUG [IPC Server handler 7 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:02,299 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:02,299 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:29:02,299 DEBUG [IPC Server handler 7 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:29:02,299 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:02,299 DEBUG [IPC Server handler 7 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 1 procesingTime= 0 2015-11-22 19:29:02,299 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:02,299 DEBUG [IPC Server handler 7 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 7 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#63 Retry#0 2015-11-22 19:29:02,299 DEBUG [IPC Server handler 7 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 7 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#63 Retry#0 Wrote 47 bytes. 2015-11-22 19:29:02,334 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:29:02,335 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:29:02,335 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:02,335 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:29:02,335 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:29:02,335 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:29:02,335 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:29:02,335 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:29:02,335 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:29:02,920 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #883 2015-11-22 19:29:02,920 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #883 2015-11-22 19:29:02,920 DEBUG [IPC Server handler 15 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 15 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#883 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:02,920 DEBUG [IPC Server handler 15 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:02,920 DEBUG [IPC Server handler 15 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:29:02,921 DEBUG [IPC Server handler 15 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 1 2015-11-22 19:29:02,921 DEBUG [IPC Server handler 15 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 15 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#883 Retry#0 2015-11-22 19:29:02,921 DEBUG [IPC Server handler 15 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 15 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#883 Retry#0 Wrote 360 bytes. 2015-11-22 19:29:02,921 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #883 2015-11-22 19:29:02,921 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:29:02,921 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:29:03,301 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #64 2015-11-22 19:29:03,301 DEBUG [IPC Server handler 9 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 9 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#64 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:03,301 DEBUG [IPC Server handler 9 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:03,301 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:03,301 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:29:03,301 DEBUG [IPC Server handler 9 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:29:03,302 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:03,302 DEBUG [IPC Server handler 9 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 1 2015-11-22 19:29:03,302 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:03,302 DEBUG [IPC Server handler 9 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 9 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#64 Retry#0 2015-11-22 19:29:03,302 DEBUG [IPC Server handler 9 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 9 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#64 Retry#0 Wrote 47 bytes. 2015-11-22 19:29:03,335 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:29:03,335 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:29:03,335 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:03,335 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:29:03,336 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:29:03,336 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:29:03,336 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:29:03,336 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:29:03,336 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:29:03,922 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #884 2015-11-22 19:29:03,922 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #884 2015-11-22 19:29:03,922 DEBUG [IPC Server handler 14 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 14 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#884 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:03,922 DEBUG [IPC Server handler 14 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:03,922 DEBUG [IPC Server handler 14 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:29:03,923 DEBUG [IPC Server handler 14 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 1 2015-11-22 19:29:03,923 DEBUG [IPC Server handler 14 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 14 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#884 Retry#0 2015-11-22 19:29:03,923 DEBUG [IPC Server handler 14 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 14 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#884 Retry#0 Wrote 360 bytes. 2015-11-22 19:29:03,923 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #884 2015-11-22 19:29:03,923 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 1ms 2015-11-22 19:29:03,923 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:29:04,304 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #65 2015-11-22 19:29:04,304 DEBUG [IPC Server handler 10 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 10 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#65 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:04,304 DEBUG [IPC Server handler 10 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:04,304 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:04,304 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:29:04,304 DEBUG [IPC Server handler 10 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:29:04,304 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:04,304 DEBUG [IPC Server handler 10 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 0 2015-11-22 19:29:04,304 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:04,305 DEBUG [IPC Server handler 10 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 10 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#65 Retry#0 2015-11-22 19:29:04,305 DEBUG [IPC Server handler 10 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 10 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#65 Retry#0 Wrote 47 bytes. 2015-11-22 19:29:04,336 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:29:04,336 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:29:04,336 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:04,336 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:29:04,336 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:29:04,336 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:29:04,336 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:29:04,336 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:29:04,336 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:29:04,924 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #885 2015-11-22 19:29:04,924 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #885 2015-11-22 19:29:04,924 DEBUG [IPC Server handler 18 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 18 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#885 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:04,924 DEBUG [IPC Server handler 18 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:04,924 DEBUG [IPC Server handler 18 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:29:04,925 DEBUG [IPC Server handler 18 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 1 2015-11-22 19:29:04,925 DEBUG [IPC Server handler 18 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 18 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#885 Retry#0 2015-11-22 19:29:04,925 DEBUG [IPC Server handler 18 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 18 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#885 Retry#0 Wrote 360 bytes. 2015-11-22 19:29:04,925 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #885 2015-11-22 19:29:04,925 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:29:04,925 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:29:05,306 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #66 2015-11-22 19:29:05,306 DEBUG [IPC Server handler 1 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 1 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#66 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:05,307 DEBUG [IPC Server handler 1 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:05,307 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:05,307 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:29:05,307 DEBUG [IPC Server handler 1 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:29:05,307 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:05,307 DEBUG [IPC Server handler 1 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 1 procesingTime= 0 2015-11-22 19:29:05,307 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:05,308 DEBUG [IPC Server handler 1 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 1 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#66 Retry#0 2015-11-22 19:29:05,308 DEBUG [IPC Server handler 1 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 1 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#66 Retry#0 Wrote 47 bytes. 2015-11-22 19:29:05,336 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:29:05,336 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:29:05,336 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:05,337 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:29:05,337 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:29:05,337 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:29:05,337 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:29:05,337 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:29:05,337 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:29:05,926 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #886 2015-11-22 19:29:05,926 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #886 2015-11-22 19:29:05,926 DEBUG [IPC Server handler 19 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 19 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#886 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:05,926 DEBUG [IPC Server handler 19 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:05,926 DEBUG [IPC Server handler 19 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:29:05,927 DEBUG [IPC Server handler 19 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 1 2015-11-22 19:29:05,927 DEBUG [IPC Server handler 19 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 19 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#886 Retry#0 2015-11-22 19:29:05,927 DEBUG [IPC Server handler 19 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 19 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#886 Retry#0 Wrote 360 bytes. 2015-11-22 19:29:05,927 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #886 2015-11-22 19:29:05,927 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 1ms 2015-11-22 19:29:05,927 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:29:06,309 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #67 2015-11-22 19:29:06,309 DEBUG [IPC Server handler 4 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 4 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#67 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:06,309 DEBUG [IPC Server handler 4 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:06,310 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:06,310 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:29:06,310 DEBUG [IPC Server handler 4 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:29:06,310 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:06,310 DEBUG [IPC Server handler 4 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 1 2015-11-22 19:29:06,310 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:06,310 DEBUG [IPC Server handler 4 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 4 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#67 Retry#0 2015-11-22 19:29:06,310 DEBUG [IPC Server handler 4 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 4 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#67 Retry#0 Wrote 47 bytes. 2015-11-22 19:29:06,337 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:29:06,337 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:29:06,337 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:06,337 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:29:06,337 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:29:06,337 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:29:06,337 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:29:06,337 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:29:06,338 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:29:06,928 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #887 2015-11-22 19:29:06,928 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #887 2015-11-22 19:29:06,928 DEBUG [IPC Server handler 20 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 20 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#887 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:06,928 DEBUG [IPC Server handler 20 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:06,928 DEBUG [IPC Server handler 20 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:29:06,929 DEBUG [IPC Server handler 20 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 1 2015-11-22 19:29:06,929 DEBUG [IPC Server handler 20 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 20 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#887 Retry#0 2015-11-22 19:29:06,929 DEBUG [IPC Server handler 20 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 20 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#887 Retry#0 Wrote 360 bytes. 2015-11-22 19:29:06,929 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #887 2015-11-22 19:29:06,929 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 1ms 2015-11-22 19:29:06,929 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:29:07,312 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #68 2015-11-22 19:29:07,312 DEBUG [IPC Server handler 13 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 13 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#68 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:07,312 DEBUG [IPC Server handler 13 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:07,312 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:07,312 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:29:07,312 DEBUG [IPC Server handler 13 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:29:07,313 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:07,313 DEBUG [IPC Server handler 13 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 1 2015-11-22 19:29:07,313 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:07,313 DEBUG [IPC Server handler 13 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 13 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#68 Retry#0 2015-11-22 19:29:07,313 DEBUG [IPC Server handler 13 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 13 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#68 Retry#0 Wrote 47 bytes. 2015-11-22 19:29:07,337 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:29:07,337 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:29:07,338 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:07,338 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:29:07,338 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:29:07,338 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:29:07,338 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:29:07,338 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:29:07,338 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:29:07,930 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #888 2015-11-22 19:29:07,930 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #888 2015-11-22 19:29:07,930 DEBUG [IPC Server handler 21 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 21 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#888 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:07,930 DEBUG [IPC Server handler 21 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:07,930 DEBUG [IPC Server handler 21 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:29:07,931 DEBUG [IPC Server handler 21 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 1 2015-11-22 19:29:07,931 DEBUG [IPC Server handler 21 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 21 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#888 Retry#0 2015-11-22 19:29:07,931 DEBUG [IPC Server handler 21 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 21 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#888 Retry#0 Wrote 360 bytes. 2015-11-22 19:29:07,931 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #888 2015-11-22 19:29:07,931 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 1ms 2015-11-22 19:29:07,931 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:29:08,314 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #69 2015-11-22 19:29:08,314 DEBUG [IPC Server handler 11 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 11 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#69 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:08,314 DEBUG [IPC Server handler 11 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:08,315 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:08,315 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:29:08,315 DEBUG [IPC Server handler 11 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:29:08,315 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:08,315 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:08,315 DEBUG [IPC Server handler 11 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 1 2015-11-22 19:29:08,315 DEBUG [IPC Server handler 11 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 11 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#69 Retry#0 2015-11-22 19:29:08,315 DEBUG [IPC Server handler 11 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 11 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#69 Retry#0 Wrote 47 bytes. 2015-11-22 19:29:08,338 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:29:08,338 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:29:08,338 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:08,338 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:29:08,338 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:29:08,338 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:29:08,338 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:29:08,339 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:29:08,339 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:29:08,680 DEBUG [IPC Server idle connection scanner for port 48629] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 48629: task running 2015-11-22 19:29:08,686 DEBUG [IPC Server idle connection scanner for port 46284] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 46284: task running 2015-11-22 19:29:08,693 DEBUG [IPC Server idle connection scanner for port 52028] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 52028: task running 2015-11-22 19:29:08,931 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #889 2015-11-22 19:29:08,932 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #889 2015-11-22 19:29:08,932 DEBUG [IPC Server handler 22 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 22 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#889 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:08,932 DEBUG [IPC Server handler 22 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:08,932 DEBUG [IPC Server handler 22 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:29:08,932 DEBUG [IPC Server handler 22 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 0 2015-11-22 19:29:08,932 DEBUG [IPC Server handler 22 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 22 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#889 Retry#0 2015-11-22 19:29:08,933 DEBUG [IPC Server handler 22 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 22 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#889 Retry#0 Wrote 360 bytes. 2015-11-22 19:29:08,933 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #889 2015-11-22 19:29:08,933 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:29:08,933 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:29:08,949 DEBUG [IPC Server idle connection scanner for port 53669] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 53669: task running 2015-11-22 19:29:09,317 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #70 2015-11-22 19:29:09,317 DEBUG [IPC Server handler 14 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 14 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#70 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:09,317 DEBUG [IPC Server handler 14 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:09,317 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:09,317 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:29:09,317 DEBUG [IPC Server handler 14 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:29:09,317 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:09,318 DEBUG [IPC Server handler 14 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 1 2015-11-22 19:29:09,318 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:09,318 DEBUG [IPC Server handler 14 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 14 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#70 Retry#0 2015-11-22 19:29:09,318 DEBUG [IPC Server handler 14 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 14 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#70 Retry#0 Wrote 47 bytes. 2015-11-22 19:29:09,338 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:29:09,338 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:29:09,339 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:09,339 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:29:09,339 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:29:09,339 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:29:09,339 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:29:09,339 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:29:09,339 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:29:09,933 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #890 2015-11-22 19:29:09,933 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #890 2015-11-22 19:29:09,934 DEBUG [IPC Server handler 23 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 23 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#890 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:09,934 DEBUG [IPC Server handler 23 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:09,934 DEBUG [IPC Server handler 23 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:29:09,934 DEBUG [IPC Server handler 23 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 0 2015-11-22 19:29:09,934 DEBUG [IPC Server handler 23 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 23 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#890 Retry#0 2015-11-22 19:29:09,934 DEBUG [IPC Server handler 23 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 23 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#890 Retry#0 Wrote 360 bytes. 2015-11-22 19:29:09,935 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #890 2015-11-22 19:29:09,935 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:29:09,935 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:29:10,178 DEBUG [IPC Server idle connection scanner for port 50663] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 50663: task running 2015-11-22 19:29:10,184 DEBUG [IPC Server idle connection scanner for port 47651] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 47651: task running 2015-11-22 19:29:10,319 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #71 2015-11-22 19:29:10,319 DEBUG [IPC Server handler 3 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 3 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#71 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:10,319 DEBUG [IPC Server handler 3 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:10,320 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:10,320 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:29:10,320 DEBUG [IPC Server handler 3 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:29:10,320 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:10,320 DEBUG [IPC Server handler 3 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 1 2015-11-22 19:29:10,320 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:10,320 DEBUG [IPC Server handler 3 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 3 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#71 Retry#0 2015-11-22 19:29:10,320 DEBUG [IPC Server handler 3 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 3 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#71 Retry#0 Wrote 47 bytes. 2015-11-22 19:29:10,339 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:29:10,339 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:29:10,339 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:10,339 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:29:10,339 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:29:10,339 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:29:10,339 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:29:10,340 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:29:10,340 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:29:10,935 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #891 2015-11-22 19:29:10,935 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #891 2015-11-22 19:29:10,936 DEBUG [IPC Server handler 24 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 24 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#891 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:10,936 DEBUG [IPC Server handler 24 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:10,936 DEBUG [IPC Server handler 24 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:29:10,936 DEBUG [IPC Server handler 24 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 1 procesingTime= 0 2015-11-22 19:29:10,936 DEBUG [IPC Server handler 24 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 24 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#891 Retry#0 2015-11-22 19:29:10,936 DEBUG [IPC Server handler 24 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 24 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#891 Retry#0 Wrote 360 bytes. 2015-11-22 19:29:10,936 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #891 2015-11-22 19:29:10,937 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:29:10,937 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:29:11,179 DEBUG [IPC Server idle connection scanner for port 32936] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 32936: task running 2015-11-22 19:29:11,322 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #72 2015-11-22 19:29:11,322 DEBUG [IPC Server handler 15 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 15 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#72 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:11,322 DEBUG [IPC Server handler 15 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:11,322 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:11,322 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:29:11,322 DEBUG [IPC Server handler 15 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:29:11,322 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:11,322 DEBUG [IPC Server handler 15 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 0 2015-11-22 19:29:11,322 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:11,322 DEBUG [IPC Server handler 15 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 15 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#72 Retry#0 2015-11-22 19:29:11,323 DEBUG [IPC Server handler 15 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 15 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#72 Retry#0 Wrote 47 bytes. 2015-11-22 19:29:11,339 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:29:11,339 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:29:11,340 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:11,340 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:29:11,340 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:29:11,340 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:29:11,340 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:29:11,340 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:29:11,340 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:29:11,937 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #892 2015-11-22 19:29:11,937 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #892 2015-11-22 19:29:11,937 DEBUG [IPC Server handler 25 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 25 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#892 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:11,938 DEBUG [IPC Server handler 25 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:11,938 DEBUG [IPC Server handler 25 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:29:11,938 DEBUG [IPC Server handler 25 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 1 procesingTime= 0 2015-11-22 19:29:11,938 DEBUG [IPC Server handler 25 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 25 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#892 Retry#0 2015-11-22 19:29:11,938 DEBUG [IPC Server handler 25 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 25 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#892 Retry#0 Wrote 360 bytes. 2015-11-22 19:29:11,938 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #892 2015-11-22 19:29:11,939 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:29:11,939 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:29:12,324 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #73 2015-11-22 19:29:12,324 DEBUG [IPC Server handler 12 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 12 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#73 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:12,324 DEBUG [IPC Server handler 12 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:12,324 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:12,324 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:29:12,324 DEBUG [IPC Server handler 12 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:29:12,325 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:12,325 DEBUG [IPC Server handler 12 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 1 2015-11-22 19:29:12,325 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:12,325 DEBUG [IPC Server handler 12 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 12 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#73 Retry#0 2015-11-22 19:29:12,325 DEBUG [IPC Server handler 12 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 12 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#73 Retry#0 Wrote 47 bytes. 2015-11-22 19:29:12,340 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:29:12,340 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:29:12,340 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:12,340 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:29:12,340 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:29:12,340 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:29:12,340 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:29:12,341 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:29:12,341 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:29:12,939 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #893 2015-11-22 19:29:12,939 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #893 2015-11-22 19:29:12,939 DEBUG [IPC Server handler 26 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 26 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#893 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:12,940 DEBUG [IPC Server handler 26 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:12,940 DEBUG [IPC Server handler 26 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:29:12,940 DEBUG [IPC Server handler 26 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 1 procesingTime= 0 2015-11-22 19:29:12,940 DEBUG [IPC Server handler 26 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 26 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#893 Retry#0 2015-11-22 19:29:12,940 DEBUG [IPC Server handler 26 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 26 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#893 Retry#0 Wrote 360 bytes. 2015-11-22 19:29:12,940 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #893 2015-11-22 19:29:12,941 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:29:12,941 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:29:13,327 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #74 2015-11-22 19:29:13,327 DEBUG [IPC Server handler 18 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 18 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#74 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:13,327 DEBUG [IPC Server handler 18 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:13,327 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:13,327 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:29:13,327 DEBUG [IPC Server handler 18 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:29:13,327 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:13,327 DEBUG [IPC Server handler 18 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 0 2015-11-22 19:29:13,328 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:13,328 DEBUG [IPC Server handler 18 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 18 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#74 Retry#0 2015-11-22 19:29:13,328 DEBUG [IPC Server handler 18 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 18 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#74 Retry#0 Wrote 47 bytes. 2015-11-22 19:29:13,340 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:29:13,340 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:29:13,341 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:13,341 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:29:13,341 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:29:13,341 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:29:13,341 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:29:13,341 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:29:13,341 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:29:13,941 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #894 2015-11-22 19:29:13,941 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #894 2015-11-22 19:29:13,941 DEBUG [IPC Server handler 28 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 28 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#894 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:13,941 DEBUG [IPC Server handler 28 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:13,942 DEBUG [IPC Server handler 28 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:29:13,942 DEBUG [IPC Server handler 28 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 1 procesingTime= 0 2015-11-22 19:29:13,942 DEBUG [IPC Server handler 28 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 28 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#894 Retry#0 2015-11-22 19:29:13,942 DEBUG [IPC Server handler 28 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 28 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#894 Retry#0 Wrote 360 bytes. 2015-11-22 19:29:13,942 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #894 2015-11-22 19:29:13,942 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 1ms 2015-11-22 19:29:13,943 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:29:14,329 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #75 2015-11-22 19:29:14,329 DEBUG [IPC Server handler 20 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 20 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#75 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:14,329 DEBUG [IPC Server handler 20 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:14,329 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:14,330 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:29:14,330 DEBUG [IPC Server handler 20 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:29:14,330 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:14,330 DEBUG [IPC Server handler 20 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 1 2015-11-22 19:29:14,330 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:14,330 DEBUG [IPC Server handler 20 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 20 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#75 Retry#0 2015-11-22 19:29:14,330 DEBUG [IPC Server handler 20 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 20 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#75 Retry#0 Wrote 47 bytes. 2015-11-22 19:29:14,341 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:29:14,341 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:29:14,341 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:14,341 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:29:14,341 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:29:14,341 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:29:14,341 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:29:14,342 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:29:14,342 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:29:14,943 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #895 2015-11-22 19:29:14,943 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #895 2015-11-22 19:29:14,943 DEBUG [IPC Server handler 27 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 27 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#895 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:14,943 DEBUG [IPC Server handler 27 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:14,944 DEBUG [IPC Server handler 27 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:29:14,944 DEBUG [IPC Server handler 27 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 1 procesingTime= 0 2015-11-22 19:29:14,944 DEBUG [IPC Server handler 27 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 27 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#895 Retry#0 2015-11-22 19:29:14,945 DEBUG [IPC Server handler 27 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 27 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#895 Retry#0 Wrote 360 bytes. 2015-11-22 19:29:14,945 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #895 2015-11-22 19:29:14,945 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:29:14,945 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:29:15,332 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #76 2015-11-22 19:29:15,332 DEBUG [IPC Server handler 17 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 17 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#76 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:15,332 DEBUG [IPC Server handler 17 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:15,332 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:15,332 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:29:15,332 DEBUG [IPC Server handler 17 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:29:15,333 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:15,333 DEBUG [IPC Server handler 17 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 1 2015-11-22 19:29:15,333 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:15,333 DEBUG [IPC Server handler 17 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 17 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#76 Retry#0 2015-11-22 19:29:15,333 DEBUG [IPC Server handler 17 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 17 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#76 Retry#0 Wrote 47 bytes. 2015-11-22 19:29:15,341 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:29:15,341 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:29:15,342 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:15,342 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:29:15,342 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:29:15,342 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:29:15,342 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:29:15,342 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:29:15,342 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:29:15,945 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #896 2015-11-22 19:29:15,946 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #896 2015-11-22 19:29:15,946 DEBUG [IPC Server handler 17 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 17 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#896 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:15,946 DEBUG [IPC Server handler 17 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:15,946 DEBUG [IPC Server handler 17 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:29:15,947 DEBUG [IPC Server handler 17 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 1 2015-11-22 19:29:15,947 DEBUG [IPC Server handler 17 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 17 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#896 Retry#0 2015-11-22 19:29:15,947 DEBUG [IPC Server handler 17 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 17 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#896 Retry#0 Wrote 360 bytes. 2015-11-22 19:29:15,947 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #896 2015-11-22 19:29:15,947 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:29:15,947 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:29:16,334 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #77 2015-11-22 19:29:16,334 DEBUG [IPC Server handler 19 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 19 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#77 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:16,334 DEBUG [IPC Server handler 19 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:16,335 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:16,335 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:29:16,335 DEBUG [IPC Server handler 19 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:29:16,335 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:16,335 DEBUG [IPC Server handler 19 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 1 2015-11-22 19:29:16,335 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:16,335 DEBUG [IPC Server handler 19 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 19 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#77 Retry#0 2015-11-22 19:29:16,335 DEBUG [IPC Server handler 19 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 19 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#77 Retry#0 Wrote 47 bytes. 2015-11-22 19:29:16,342 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:29:16,342 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:29:16,342 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:16,342 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:29:16,342 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:29:16,343 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:29:16,343 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:29:16,343 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:29:16,343 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:29:16,948 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #897 2015-11-22 19:29:16,948 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #897 2015-11-22 19:29:16,948 DEBUG [IPC Server handler 30 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 30 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#897 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:16,948 DEBUG [IPC Server handler 30 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:16,948 DEBUG [IPC Server handler 30 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:29:16,949 DEBUG [IPC Server handler 30 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 1 2015-11-22 19:29:16,949 DEBUG [IPC Server handler 30 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 30 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#897 Retry#0 2015-11-22 19:29:16,949 DEBUG [IPC Server handler 30 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 30 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#897 Retry#0 Wrote 360 bytes. 2015-11-22 19:29:16,949 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #897 2015-11-22 19:29:16,949 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 1ms 2015-11-22 19:29:16,949 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:29:17,337 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #78 2015-11-22 19:29:17,337 DEBUG [IPC Server handler 21 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 21 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#78 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:17,337 DEBUG [IPC Server handler 21 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:17,337 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:17,337 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:29:17,337 DEBUG [IPC Server handler 21 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:29:17,338 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:17,338 DEBUG [IPC Server handler 21 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 1 2015-11-22 19:29:17,338 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:17,338 DEBUG [IPC Server handler 21 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 21 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#78 Retry#0 2015-11-22 19:29:17,338 DEBUG [IPC Server handler 21 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 21 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#78 Retry#0 Wrote 47 bytes. 2015-11-22 19:29:17,342 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:29:17,343 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:29:17,343 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:17,343 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:29:17,343 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:29:17,343 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:29:17,343 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:29:17,343 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:29:17,343 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:29:17,950 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #898 2015-11-22 19:29:17,950 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #898 2015-11-22 19:29:17,950 DEBUG [IPC Server handler 29 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 29 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#898 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:17,950 DEBUG [IPC Server handler 29 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:17,950 DEBUG [IPC Server handler 29 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:29:17,951 DEBUG [IPC Server handler 29 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 1 2015-11-22 19:29:17,951 DEBUG [IPC Server handler 29 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 29 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#898 Retry#0 2015-11-22 19:29:17,951 DEBUG [IPC Server handler 29 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 29 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#898 Retry#0 Wrote 360 bytes. 2015-11-22 19:29:17,951 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #898 2015-11-22 19:29:17,951 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:29:17,951 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:29:18,339 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #79 2015-11-22 19:29:18,339 DEBUG [IPC Server handler 22 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 22 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#79 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:18,339 DEBUG [IPC Server handler 22 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:18,340 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:18,340 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:29:18,340 DEBUG [IPC Server handler 22 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:29:18,340 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:18,340 DEBUG [IPC Server handler 22 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 1 procesingTime= 0 2015-11-22 19:29:18,340 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:18,340 DEBUG [IPC Server handler 22 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 22 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#79 Retry#0 2015-11-22 19:29:18,340 DEBUG [IPC Server handler 22 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 22 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#79 Retry#0 Wrote 47 bytes. 2015-11-22 19:29:18,343 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:29:18,343 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:29:18,343 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:18,343 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:29:18,343 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:29:18,344 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:29:18,344 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:29:18,344 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:29:18,344 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:29:18,680 DEBUG [IPC Server idle connection scanner for port 48629] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 48629: task running 2015-11-22 19:29:18,686 DEBUG [IPC Server idle connection scanner for port 46284] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 46284: task running 2015-11-22 19:29:18,693 DEBUG [IPC Server idle connection scanner for port 52028] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 52028: task running 2015-11-22 19:29:18,950 DEBUG [IPC Server idle connection scanner for port 53669] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 53669: task running 2015-11-22 19:29:18,952 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #899 2015-11-22 19:29:18,952 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #899 2015-11-22 19:29:18,952 DEBUG [IPC Server handler 31 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 31 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#899 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:18,952 DEBUG [IPC Server handler 31 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:18,952 DEBUG [IPC Server handler 31 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:29:18,953 DEBUG [IPC Server handler 31 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 1 2015-11-22 19:29:18,953 DEBUG [IPC Server handler 31 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 31 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#899 Retry#0 2015-11-22 19:29:18,953 DEBUG [IPC Server handler 31 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 31 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#899 Retry#0 Wrote 360 bytes. 2015-11-22 19:29:18,953 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #899 2015-11-22 19:29:18,953 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 1ms 2015-11-22 19:29:18,953 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:29:19,342 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #80 2015-11-22 19:29:19,342 DEBUG [IPC Server handler 23 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 23 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#80 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:19,342 DEBUG [IPC Server handler 23 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:19,342 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:19,342 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:29:19,342 DEBUG [IPC Server handler 23 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:29:19,343 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:19,343 DEBUG [IPC Server handler 23 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 1 2015-11-22 19:29:19,343 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:19,343 DEBUG [IPC Server handler 23 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 23 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#80 Retry#0 2015-11-22 19:29:19,343 DEBUG [IPC Server handler 23 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 23 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#80 Retry#0 Wrote 47 bytes. 2015-11-22 19:29:19,343 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:29:19,344 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:29:19,344 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:19,344 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:29:19,344 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:29:19,344 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:29:19,344 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:29:19,344 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:29:19,344 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:29:19,954 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #900 2015-11-22 19:29:19,954 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #900 2015-11-22 19:29:19,954 DEBUG [IPC Server handler 11 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 11 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#900 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:19,954 DEBUG [IPC Server handler 11 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:19,954 DEBUG [IPC Server handler 11 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:29:19,955 DEBUG [IPC Server handler 11 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 1 2015-11-22 19:29:19,955 DEBUG [IPC Server handler 11 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 11 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#900 Retry#0 2015-11-22 19:29:19,955 DEBUG [IPC Server handler 11 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 11 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#900 Retry#0 Wrote 360 bytes. 2015-11-22 19:29:19,955 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #900 2015-11-22 19:29:19,955 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 1ms 2015-11-22 19:29:19,955 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:29:20,178 DEBUG [IPC Server idle connection scanner for port 50663] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 50663: task running 2015-11-22 19:29:20,185 DEBUG [IPC Server idle connection scanner for port 47651] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 47651: task running 2015-11-22 19:29:20,344 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:29:20,344 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:29:20,344 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #81 2015-11-22 19:29:20,344 DEBUG [IPC Server handler 24 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 24 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#81 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:20,344 DEBUG [IPC Server handler 24 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:20,344 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:20,345 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:29:20,345 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:20,345 DEBUG [IPC Server handler 24 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:29:20,345 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:29:20,345 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:29:20,345 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:20,345 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:29:20,345 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:29:20,345 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:29:20,345 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:29:20,345 DEBUG [IPC Server handler 24 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 1 procesingTime= 0 2015-11-22 19:29:20,345 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:20,346 DEBUG [IPC Server handler 24 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 24 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#81 Retry#0 2015-11-22 19:29:20,346 DEBUG [IPC Server handler 24 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 24 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#81 Retry#0 Wrote 47 bytes. 2015-11-22 19:29:20,956 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #901 2015-11-22 19:29:20,956 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #901 2015-11-22 19:29:20,956 DEBUG [IPC Server handler 32 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 32 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#901 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:20,956 DEBUG [IPC Server handler 32 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:20,956 DEBUG [IPC Server handler 32 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:29:20,957 DEBUG [IPC Server handler 32 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 1 2015-11-22 19:29:20,957 DEBUG [IPC Server handler 32 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 32 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#901 Retry#0 2015-11-22 19:29:20,957 DEBUG [IPC Server handler 32 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 32 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#901 Retry#0 Wrote 360 bytes. 2015-11-22 19:29:20,957 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #901 2015-11-22 19:29:20,957 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:29:20,957 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:29:21,179 DEBUG [IPC Server idle connection scanner for port 32936] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 32936: task running 2015-11-22 19:29:21,345 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:29:21,345 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:29:21,345 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:21,345 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:29:21,345 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:29:21,345 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:29:21,345 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:29:21,345 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:29:21,346 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:29:21,347 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #82 2015-11-22 19:29:21,347 DEBUG [IPC Server handler 25 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 25 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#82 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:21,347 DEBUG [IPC Server handler 25 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:21,348 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:21,348 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:29:21,348 DEBUG [IPC Server handler 25 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:29:21,348 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:21,348 DEBUG [IPC Server handler 25 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 1 2015-11-22 19:29:21,348 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:21,348 DEBUG [IPC Server handler 25 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 25 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#82 Retry#0 2015-11-22 19:29:21,348 DEBUG [IPC Server handler 25 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 25 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#82 Retry#0 Wrote 47 bytes. 2015-11-22 19:29:21,958 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #902 2015-11-22 19:29:21,958 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #902 2015-11-22 19:29:21,958 DEBUG [IPC Server handler 34 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 34 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#902 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:21,958 DEBUG [IPC Server handler 34 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:21,958 DEBUG [IPC Server handler 34 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:29:21,959 DEBUG [IPC Server handler 34 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 1 2015-11-22 19:29:21,959 DEBUG [IPC Server handler 34 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 34 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#902 Retry#0 2015-11-22 19:29:21,959 DEBUG [IPC Server handler 34 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 34 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#902 Retry#0 Wrote 360 bytes. 2015-11-22 19:29:21,959 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #902 2015-11-22 19:29:21,959 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:29:21,959 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:29:22,345 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:29:22,345 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:29:22,346 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:22,346 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:29:22,346 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:29:22,346 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:29:22,346 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:29:22,346 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:29:22,346 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:29:22,349 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #83 2015-11-22 19:29:22,350 DEBUG [IPC Server handler 26 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 26 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#83 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:22,350 DEBUG [IPC Server handler 26 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:22,350 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:22,350 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:29:22,350 DEBUG [IPC Server handler 26 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:29:22,350 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:22,350 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:22,350 DEBUG [IPC Server handler 26 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 0 2015-11-22 19:29:22,351 DEBUG [IPC Server handler 26 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 26 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#83 Retry#0 2015-11-22 19:29:22,351 DEBUG [IPC Server handler 26 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 26 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#83 Retry#0 Wrote 47 bytes. 2015-11-22 19:29:22,960 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #903 2015-11-22 19:29:22,960 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #903 2015-11-22 19:29:22,960 DEBUG [IPC Server handler 37 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 37 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#903 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:22,960 DEBUG [IPC Server handler 37 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:22,960 DEBUG [IPC Server handler 37 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:29:22,961 DEBUG [IPC Server handler 37 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 1 2015-11-22 19:29:22,961 DEBUG [IPC Server handler 37 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 37 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#903 Retry#0 2015-11-22 19:29:22,961 DEBUG [IPC Server handler 37 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 37 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#903 Retry#0 Wrote 360 bytes. 2015-11-22 19:29:22,961 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #903 2015-11-22 19:29:22,961 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 1ms 2015-11-22 19:29:22,961 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:29:23,346 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:29:23,346 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:29:23,346 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:23,346 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:29:23,346 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:29:23,346 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:29:23,347 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:29:23,347 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:29:23,347 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:29:23,352 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #84 2015-11-22 19:29:23,352 DEBUG [IPC Server handler 27 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 27 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#84 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:23,352 DEBUG [IPC Server handler 27 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:23,352 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:23,353 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:29:23,353 DEBUG [IPC Server handler 27 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:29:23,353 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:23,353 DEBUG [IPC Server handler 27 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 1 2015-11-22 19:29:23,353 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:23,353 DEBUG [IPC Server handler 27 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 27 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#84 Retry#0 2015-11-22 19:29:23,353 DEBUG [IPC Server handler 27 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 27 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#84 Retry#0 Wrote 47 bytes. 2015-11-22 19:29:23,962 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #904 2015-11-22 19:29:23,962 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #904 2015-11-22 19:29:23,962 DEBUG [IPC Server handler 38 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 38 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#904 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:23,962 DEBUG [IPC Server handler 38 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:23,962 DEBUG [IPC Server handler 38 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:29:23,963 DEBUG [IPC Server handler 38 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 1 2015-11-22 19:29:23,963 DEBUG [IPC Server handler 38 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 38 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#904 Retry#0 2015-11-22 19:29:23,963 DEBUG [IPC Server handler 38 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 38 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#904 Retry#0 Wrote 360 bytes. 2015-11-22 19:29:23,963 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #904 2015-11-22 19:29:23,963 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 1ms 2015-11-22 19:29:23,963 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:29:24,346 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:29:24,347 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:29:24,347 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:24,347 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:29:24,347 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:29:24,347 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:29:24,347 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:29:24,347 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:29:24,347 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:29:24,354 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #85 2015-11-22 19:29:24,355 DEBUG [IPC Server handler 28 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 28 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#85 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:24,355 DEBUG [IPC Server handler 28 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:24,355 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:24,355 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:29:24,355 DEBUG [IPC Server handler 28 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:29:24,355 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:24,355 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:24,355 DEBUG [IPC Server handler 28 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 0 2015-11-22 19:29:24,356 DEBUG [IPC Server handler 28 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 28 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#85 Retry#0 2015-11-22 19:29:24,356 DEBUG [IPC Server handler 28 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 28 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#85 Retry#0 Wrote 47 bytes. 2015-11-22 19:29:24,964 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #905 2015-11-22 19:29:24,964 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #905 2015-11-22 19:29:24,964 DEBUG [IPC Server handler 39 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 39 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#905 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:24,964 DEBUG [IPC Server handler 39 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:24,965 DEBUG [IPC Server handler 39 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:29:24,965 DEBUG [IPC Server handler 39 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 1 2015-11-22 19:29:24,965 DEBUG [IPC Server handler 39 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 39 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#905 Retry#0 2015-11-22 19:29:24,965 DEBUG [IPC Server handler 39 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 39 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#905 Retry#0 Wrote 360 bytes. 2015-11-22 19:29:24,965 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #905 2015-11-22 19:29:24,965 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 1ms 2015-11-22 19:29:24,966 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:29:25,347 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:29:25,347 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:29:25,348 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:25,348 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:29:25,348 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:29:25,348 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:29:25,348 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:29:25,348 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:29:25,348 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:29:25,357 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #86 2015-11-22 19:29:25,357 DEBUG [IPC Server handler 32 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 32 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#86 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:25,357 DEBUG [IPC Server handler 32 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:25,358 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:25,358 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:29:25,358 DEBUG [IPC Server handler 32 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:29:25,358 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:25,358 DEBUG [IPC Server handler 32 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 1 2015-11-22 19:29:25,358 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:25,358 DEBUG [IPC Server handler 32 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 32 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#86 Retry#0 2015-11-22 19:29:25,358 DEBUG [IPC Server handler 32 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 32 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#86 Retry#0 Wrote 47 bytes. 2015-11-22 19:29:25,966 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #906 2015-11-22 19:29:25,966 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #906 2015-11-22 19:29:25,966 DEBUG [IPC Server handler 40 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 40 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#906 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:25,966 DEBUG [IPC Server handler 40 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:25,967 DEBUG [IPC Server handler 40 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:29:25,967 DEBUG [IPC Server handler 40 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 1 procesingTime= 0 2015-11-22 19:29:25,967 DEBUG [IPC Server handler 40 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 40 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#906 Retry#0 2015-11-22 19:29:25,967 DEBUG [IPC Server handler 40 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 40 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#906 Retry#0 Wrote 360 bytes. 2015-11-22 19:29:25,967 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #906 2015-11-22 19:29:25,967 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 1ms 2015-11-22 19:29:25,968 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:29:26,348 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:29:26,348 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:29:26,348 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:26,348 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:29:26,348 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:29:26,349 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:29:26,349 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:29:26,349 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:29:26,349 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:29:26,360 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #87 2015-11-22 19:29:26,360 DEBUG [IPC Server handler 29 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 29 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#87 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:26,360 DEBUG [IPC Server handler 29 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:26,360 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:26,360 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:29:26,360 DEBUG [IPC Server handler 29 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:29:26,360 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:26,360 DEBUG [IPC Server handler 29 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 0 2015-11-22 19:29:26,361 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:26,361 DEBUG [IPC Server handler 29 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 29 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#87 Retry#0 2015-11-22 19:29:26,361 DEBUG [IPC Server handler 29 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 29 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#87 Retry#0 Wrote 47 bytes. 2015-11-22 19:29:26,968 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #907 2015-11-22 19:29:26,968 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #907 2015-11-22 19:29:26,968 DEBUG [IPC Server handler 41 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 41 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#907 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:26,969 DEBUG [IPC Server handler 41 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:26,969 DEBUG [IPC Server handler 41 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:29:26,969 DEBUG [IPC Server handler 41 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 1 procesingTime= 0 2015-11-22 19:29:26,969 DEBUG [IPC Server handler 41 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 41 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#907 Retry#0 2015-11-22 19:29:26,969 DEBUG [IPC Server handler 41 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 41 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#907 Retry#0 Wrote 360 bytes. 2015-11-22 19:29:26,969 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #907 2015-11-22 19:29:26,970 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:29:26,970 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:29:27,348 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:29:27,349 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:29:27,349 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:27,349 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:29:27,349 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:29:27,349 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:29:27,349 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:29:27,349 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:29:27,349 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:29:27,362 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #88 2015-11-22 19:29:27,362 DEBUG [IPC Server handler 16 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 16 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#88 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:27,362 DEBUG [IPC Server handler 16 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:27,362 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:27,363 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:29:27,363 DEBUG [IPC Server handler 16 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:29:27,363 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:27,363 DEBUG [IPC Server handler 16 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 1 2015-11-22 19:29:27,363 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:27,363 DEBUG [IPC Server handler 16 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 16 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#88 Retry#0 2015-11-22 19:29:27,363 DEBUG [IPC Server handler 16 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 16 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#88 Retry#0 Wrote 47 bytes. 2015-11-22 19:29:27,970 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #908 2015-11-22 19:29:27,970 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #908 2015-11-22 19:29:27,970 DEBUG [IPC Server handler 42 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 42 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#908 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:27,970 DEBUG [IPC Server handler 42 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:27,971 DEBUG [IPC Server handler 42 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:29:27,971 DEBUG [IPC Server handler 42 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 1 procesingTime= 0 2015-11-22 19:29:27,971 DEBUG [IPC Server handler 42 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 42 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#908 Retry#0 2015-11-22 19:29:27,971 DEBUG [IPC Server handler 42 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 42 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#908 Retry#0 Wrote 360 bytes. 2015-11-22 19:29:27,971 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #908 2015-11-22 19:29:27,971 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 1ms 2015-11-22 19:29:27,972 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:29:28,349 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:29:28,349 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:29:28,349 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:28,349 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:29:28,350 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:29:28,350 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:29:28,350 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:29:28,350 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:29:28,350 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:29:28,364 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #89 2015-11-22 19:29:28,364 DEBUG [IPC Server handler 31 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 31 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#89 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:28,365 DEBUG [IPC Server handler 31 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:28,365 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:28,365 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:29:28,365 DEBUG [IPC Server handler 31 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:29:28,365 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:28,365 DEBUG [IPC Server handler 31 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 1 procesingTime= 0 2015-11-22 19:29:28,365 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:28,365 DEBUG [IPC Server handler 31 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 31 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#89 Retry#0 2015-11-22 19:29:28,365 DEBUG [IPC Server handler 31 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 31 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#89 Retry#0 Wrote 47 bytes. 2015-11-22 19:29:28,680 DEBUG [IPC Server idle connection scanner for port 48629] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 48629: task running 2015-11-22 19:29:28,686 DEBUG [IPC Server idle connection scanner for port 46284] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 46284: task running 2015-11-22 19:29:28,693 DEBUG [IPC Server idle connection scanner for port 52028] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 52028: task running 2015-11-22 19:29:28,950 DEBUG [IPC Server idle connection scanner for port 53669] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 53669: task running 2015-11-22 19:29:28,972 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #909 2015-11-22 19:29:28,972 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #909 2015-11-22 19:29:28,972 DEBUG [IPC Server handler 43 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 43 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#909 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:28,972 DEBUG [IPC Server handler 43 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:28,973 DEBUG [IPC Server handler 43 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:29:28,973 DEBUG [IPC Server handler 43 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 1 procesingTime= 0 2015-11-22 19:29:28,973 DEBUG [IPC Server handler 43 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 43 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#909 Retry#0 2015-11-22 19:29:28,973 DEBUG [IPC Server handler 43 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 43 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#909 Retry#0 Wrote 360 bytes. 2015-11-22 19:29:28,973 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #909 2015-11-22 19:29:28,974 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:29:28,974 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:29:29,350 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:29:29,350 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:29:29,350 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:29,350 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:29:29,350 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:29:29,350 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:29:29,350 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:29:29,351 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:29:29,351 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:29:29,367 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #90 2015-11-22 19:29:29,367 DEBUG [IPC Server handler 30 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 30 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#90 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:29,367 DEBUG [IPC Server handler 30 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:29,367 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:29,367 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:29:29,367 DEBUG [IPC Server handler 30 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:29:29,367 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:29,368 DEBUG [IPC Server handler 30 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 1 2015-11-22 19:29:29,368 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:29,368 DEBUG [IPC Server handler 30 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 30 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#90 Retry#0 2015-11-22 19:29:29,368 DEBUG [IPC Server handler 30 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 30 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#90 Retry#0 Wrote 47 bytes. 2015-11-22 19:29:29,974 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #910 2015-11-22 19:29:29,974 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #910 2015-11-22 19:29:29,975 DEBUG [IPC Server handler 45 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 45 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#910 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:29,975 DEBUG [IPC Server handler 45 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:29,975 DEBUG [IPC Server handler 45 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:29:29,975 DEBUG [IPC Server handler 45 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 0 2015-11-22 19:29:29,975 DEBUG [IPC Server handler 45 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 45 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#910 Retry#0 2015-11-22 19:29:29,976 DEBUG [IPC Server handler 45 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 45 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#910 Retry#0 Wrote 360 bytes. 2015-11-22 19:29:29,976 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #910 2015-11-22 19:29:29,976 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:29:29,976 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:29:30,179 DEBUG [IPC Server idle connection scanner for port 50663] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 50663: task running 2015-11-22 19:29:30,185 DEBUG [IPC Server idle connection scanner for port 47651] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 47651: task running 2015-11-22 19:29:30,350 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:29:30,350 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:29:30,351 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:30,351 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:29:30,351 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:29:30,351 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:29:30,351 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:29:30,351 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:29:30,351 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:29:30,369 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #91 2015-11-22 19:29:30,369 DEBUG [IPC Server handler 33 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 33 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#91 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:30,369 DEBUG [IPC Server handler 33 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:30,370 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:30,370 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:29:30,370 DEBUG [IPC Server handler 33 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:29:30,370 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:30,370 DEBUG [IPC Server handler 33 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 1 2015-11-22 19:29:30,370 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:30,370 DEBUG [IPC Server handler 33 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 33 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#91 Retry#0 2015-11-22 19:29:30,370 DEBUG [IPC Server handler 33 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 33 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#91 Retry#0 Wrote 47 bytes. 2015-11-22 19:29:30,976 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #911 2015-11-22 19:29:30,977 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #911 2015-11-22 19:29:30,977 DEBUG [IPC Server handler 44 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 44 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#911 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:30,977 DEBUG [IPC Server handler 44 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:30,977 DEBUG [IPC Server handler 44 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:29:30,977 DEBUG [IPC Server handler 44 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 0 2015-11-22 19:29:30,978 DEBUG [IPC Server handler 44 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 44 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#911 Retry#0 2015-11-22 19:29:30,978 DEBUG [IPC Server handler 44 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 44 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#911 Retry#0 Wrote 360 bytes. 2015-11-22 19:29:30,978 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #911 2015-11-22 19:29:30,978 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:29:30,978 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:29:31,179 DEBUG [IPC Server idle connection scanner for port 32936] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 32936: task running 2015-11-22 19:29:31,351 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:29:31,351 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:29:31,351 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:31,351 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:29:31,351 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:29:31,352 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:29:31,352 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:29:31,352 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:29:31,352 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:29:31,371 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #92 2015-11-22 19:29:31,372 DEBUG [IPC Server handler 36 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 36 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#92 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:31,372 DEBUG [IPC Server handler 36 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:31,372 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:31,372 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:29:31,372 DEBUG [IPC Server handler 36 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:29:31,372 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:31,372 DEBUG [IPC Server handler 36 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 0 2015-11-22 19:29:31,372 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:31,372 DEBUG [IPC Server handler 36 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 36 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#92 Retry#0 2015-11-22 19:29:31,373 DEBUG [IPC Server handler 36 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 36 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#92 Retry#0 Wrote 47 bytes. 2015-11-22 19:29:31,979 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #912 2015-11-22 19:29:31,979 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #912 2015-11-22 19:29:31,979 DEBUG [IPC Server handler 47 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 47 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#912 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:31,979 DEBUG [IPC Server handler 47 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:31,979 DEBUG [IPC Server handler 47 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:29:31,980 DEBUG [IPC Server handler 47 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 1 2015-11-22 19:29:31,980 DEBUG [IPC Server handler 47 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 47 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#912 Retry#0 2015-11-22 19:29:31,980 DEBUG [IPC Server handler 47 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 47 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#912 Retry#0 Wrote 360 bytes. 2015-11-22 19:29:31,980 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #912 2015-11-22 19:29:31,980 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:29:31,981 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:29:32,351 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:29:32,352 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:29:32,352 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:32,352 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:29:32,352 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:29:32,352 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:29:32,352 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:29:32,352 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:29:32,353 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:29:32,374 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #93 2015-11-22 19:29:32,374 DEBUG [IPC Server handler 35 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 35 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#93 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:32,374 DEBUG [IPC Server handler 35 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:32,374 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:32,374 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:29:32,375 DEBUG [IPC Server handler 35 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:29:32,375 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:32,375 DEBUG [IPC Server handler 35 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 1 2015-11-22 19:29:32,375 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:32,375 DEBUG [IPC Server handler 35 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 35 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#93 Retry#0 2015-11-22 19:29:32,375 DEBUG [IPC Server handler 35 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 35 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#93 Retry#0 Wrote 47 bytes. 2015-11-22 19:29:32,981 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #913 2015-11-22 19:29:32,981 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #913 2015-11-22 19:29:32,981 DEBUG [IPC Server handler 46 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 46 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#913 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:32,981 DEBUG [IPC Server handler 46 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:32,982 DEBUG [IPC Server handler 46 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:29:32,982 DEBUG [IPC Server handler 46 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 1 procesingTime= 0 2015-11-22 19:29:32,982 DEBUG [IPC Server handler 46 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 46 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#913 Retry#0 2015-11-22 19:29:32,982 DEBUG [IPC Server handler 46 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 46 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#913 Retry#0 Wrote 360 bytes. 2015-11-22 19:29:32,982 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #913 2015-11-22 19:29:32,982 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 1ms 2015-11-22 19:29:32,983 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:29:33,352 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:29:33,352 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:29:33,353 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:33,353 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:29:33,353 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:29:33,353 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:29:33,353 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:29:33,353 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:29:33,353 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:29:33,376 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #94 2015-11-22 19:29:33,377 DEBUG [IPC Server handler 34 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 34 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#94 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:33,377 DEBUG [IPC Server handler 34 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:33,377 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:33,377 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:29:33,377 DEBUG [IPC Server handler 34 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:29:33,377 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:33,377 DEBUG [IPC Server handler 34 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 0 2015-11-22 19:29:33,377 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:33,378 DEBUG [IPC Server handler 34 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 34 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#94 Retry#0 2015-11-22 19:29:33,378 DEBUG [IPC Server handler 34 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 34 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#94 Retry#0 Wrote 47 bytes. 2015-11-22 19:29:33,983 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #914 2015-11-22 19:29:33,983 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #914 2015-11-22 19:29:33,983 DEBUG [IPC Server handler 48 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 48 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#914 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:33,983 DEBUG [IPC Server handler 48 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:33,984 DEBUG [IPC Server handler 48 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:29:33,984 DEBUG [IPC Server handler 48 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 1 2015-11-22 19:29:33,984 DEBUG [IPC Server handler 48 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 48 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#914 Retry#0 2015-11-22 19:29:33,984 DEBUG [IPC Server handler 48 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 48 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#914 Retry#0 Wrote 360 bytes. 2015-11-22 19:29:33,984 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #914 2015-11-22 19:29:33,984 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 1ms 2015-11-22 19:29:33,985 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:29:34,353 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:29:34,353 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220478671_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:29:34,353 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:34,353 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type STATUS_UPDATE 2015-11-22 19:29:34,353 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:29:34,353 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:50663 clusterResources: 2015-11-22 19:29:34,354 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:50663 availableResource: 2015-11-22 19:29:34,354 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:29:34,354 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:29:34,379 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:processOneRpc(1897)) - got #95 2015-11-22 19:29:34,379 DEBUG [IPC Server handler 38 on 46284] ipc.Server (Server.java:run(2185)) - IPC Server handler 38 on 46284: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#95 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:34,379 DEBUG [IPC Server handler 38 on 46284] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220478671_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:34,379 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:34,379 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220478671_0001_000001 of type STATUS_UPDATE 2015-11-22 19:29:34,379 DEBUG [IPC Server handler 38 on 46284] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:29:34,380 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeFinishedContainersPulledByAMEvent.EventType: FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:34,380 DEBUG [IPC Server handler 38 on 46284] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 1 2015-11-22 19:29:34,380 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:50663 of type FINISHED_CONTAINERS_PULLED_BY_AM 2015-11-22 19:29:34,380 DEBUG [IPC Server handler 38 on 46284] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 38 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#95 Retry#0 2015-11-22 19:29:34,380 DEBUG [IPC Server handler 38 on 46284] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 38 on 46284: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:53059 Call#95 Retry#0 Wrote 47 bytes. 2015-11-22 19:29:34,696 DEBUG [Thread-3684] distributedshell.Client (Client.java:monitorApplication(736)) - Thread sleep in monitoring loop interrupted 2015-11-22 19:29:34,696 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #915 2015-11-22 19:29:34,696 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: TestDistributedShell entered state STOPPED 2015-11-22 19:29:34,697 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #915 2015-11-22 19:29:34,697 DEBUG [main] service.CompositeService (CompositeService.java:serviceStop(129)) - TestDistributedShell: stopping services, size=3 2015-11-22 19:29:34,697 DEBUG [IPC Server handler 49 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 49 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#915 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:34,697 DEBUG [IPC Server handler 49 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:34,697 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #2: Service org.apache.hadoop.yarn.server.MiniYARNCluster$ApplicationHistoryServerWrapper in state org.apache.hadoop.yarn.server.MiniYARNCluster$ApplicationHistoryServerWrapper: STARTED 2015-11-22 19:29:34,697 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.MiniYARNCluster$ApplicationHistoryServerWrapper entered state STOPPED 2015-11-22 19:29:34,697 DEBUG [IPC Server handler 49 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:29:34,697 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer entered state STOPPED 2015-11-22 19:29:34,698 DEBUG [IPC Server handler 49 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 1 2015-11-22 19:29:34,698 DEBUG [IPC Server handler 49 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 49 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#915 Retry#0 2015-11-22 19:29:34,698 DEBUG [IPC Server handler 49 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 49 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#915 Retry#0 Wrote 360 bytes. 2015-11-22 19:29:34,698 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #915 2015-11-22 19:29:34,698 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:29:34,698 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:29:34,701 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.mortbay.jetty.nio.SelectChannelConnector$1@a3d8174 2015-11-22 19:29:34,702 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping org.mortbay.jetty.webapp.WebAppContext@1ba9117e{/,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes/webapps/applicationhistory} 2015-11-22 19:29:34,702 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping SessionHandler@732c2a62 2015-11-22 19:29:34,702 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping SecurityHandler@272ed83b 2015-11-22 19:29:34,702 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping ServletHandler@41fecb8b 2015-11-22 19:29:34,702 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped guice 2015-11-22 19:29:34,702 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped Timeline Authentication Filter 2015-11-22 19:29:34,703 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped static_user_filter 2015-11-22 19:29:34,703 DEBUG [main] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:stopThreads(619)) - Stopping expired delegation token remover thread 2015-11-22 19:29:34,703 ERROR [Thread[Thread-3680,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(670)) - ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted 2015-11-22 19:29:34,703 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped RMAuthenticationFilter 2015-11-22 19:29:34,704 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped safety 2015-11-22 19:29:34,704 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped NoCacheFilter 2015-11-22 19:29:34,704 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped NoCacheFilter 2015-11-22 19:29:34,704 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped conf 2015-11-22 19:29:34,704 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped jmx 2015-11-22 19:29:34,704 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped metrics 2015-11-22 19:29:34,704 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped logLevel 2015-11-22 19:29:34,704 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped stacks 2015-11-22 19:29:34,705 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.mortbay.jetty.servlet.DefaultServlet$NIOResourceCache@120f102b 2015-11-22 19:29:34,705 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.mortbay.jetty.servlet.DefaultServlet-55176785 2015-11-22 19:29:34,705 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped ServletHandler@41fecb8b 2015-11-22 19:29:34,705 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped SecurityHandler@272ed83b 2015-11-22 19:29:34,705 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.mortbay.jetty.servlet.HashSessionManager@3966697d 2015-11-22 19:29:34,706 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped SessionHandler@732c2a62 2015-11-22 19:29:34,706 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping ErrorPageErrorHandler@625732 2015-11-22 19:29:34,706 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped ErrorPageErrorHandler@625732 2015-11-22 19:29:34,707 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@41fecb8b - guice as filter 2015-11-22 19:29:34,707 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@41fecb8b - Timeline Authentication Filter as filter 2015-11-22 19:29:34,707 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@41fecb8b - static_user_filter as filter 2015-11-22 19:29:34,707 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@41fecb8b - RMAuthenticationFilter as filter 2015-11-22 19:29:34,707 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@41fecb8b - safety as filter 2015-11-22 19:29:34,707 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@41fecb8b - NoCacheFilter as filter 2015-11-22 19:29:34,707 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@41fecb8b - NoCacheFilter as filter 2015-11-22 19:29:34,707 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@41fecb8b - (F=guice,[/*],[],15) as filterMapping 2015-11-22 19:29:34,708 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@41fecb8b - (F=static_user_filter,[/ws/*],[],15) as filterMapping 2015-11-22 19:29:34,708 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@41fecb8b - (F=RMAuthenticationFilter,[/ws/*],[],15) as filterMapping 2015-11-22 19:29:34,708 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@41fecb8b - (F=static_user_filter,[/applicationhistory/*],[],15) as filterMapping 2015-11-22 19:29:34,708 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@41fecb8b - (F=RMAuthenticationFilter,[/applicationhistory/*],[],15) as filterMapping 2015-11-22 19:29:34,708 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@41fecb8b - (F=static_user_filter,[/conf],[],15) as filterMapping 2015-11-22 19:29:34,708 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@41fecb8b - (F=RMAuthenticationFilter,[/conf],[],15) as filterMapping 2015-11-22 19:29:34,708 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@41fecb8b - (F=static_user_filter,[/jmx],[],15) as filterMapping 2015-11-22 19:29:34,708 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@41fecb8b - (F=RMAuthenticationFilter,[/jmx],[],15) as filterMapping 2015-11-22 19:29:34,709 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@41fecb8b - (F=static_user_filter,[/metrics],[],15) as filterMapping 2015-11-22 19:29:34,709 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@41fecb8b - (F=RMAuthenticationFilter,[/metrics],[],15) as filterMapping 2015-11-22 19:29:34,709 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@41fecb8b - (F=static_user_filter,[/logLevel],[],15) as filterMapping 2015-11-22 19:29:34,709 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@41fecb8b - (F=RMAuthenticationFilter,[/logLevel],[],15) as filterMapping 2015-11-22 19:29:34,709 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@41fecb8b - (F=static_user_filter,[/stacks],[],15) as filterMapping 2015-11-22 19:29:34,709 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@41fecb8b - (F=RMAuthenticationFilter,[/stacks],[],15) as filterMapping 2015-11-22 19:29:34,709 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@41fecb8b - (F=Timeline Authentication Filter,[/*],[],15) as filterMapping 2015-11-22 19:29:34,709 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@41fecb8b - (F=static_user_filter,[*.html, *.jsp],[],15) as filterMapping 2015-11-22 19:29:34,709 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@41fecb8b - (F=RMAuthenticationFilter,[*.html, *.jsp],[],15) as filterMapping 2015-11-22 19:29:34,710 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@41fecb8b - (F=safety,[/*],[],15) as filterMapping 2015-11-22 19:29:34,710 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@41fecb8b - (F=NoCacheFilter,[/*],[],15) as filterMapping 2015-11-22 19:29:34,710 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@41fecb8b - (F=NoCacheFilter,[/*],[],15) as filterMapping 2015-11-22 19:29:34,710 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap=null 2015-11-22 19:29:34,710 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=null 2015-11-22 19:29:34,710 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:34,710 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/jmx=jmx, /conf=conf, /stacks=stacks, /logLevel=logLevel, /metrics=metrics, /=org.mortbay.jetty.servlet.DefaultServlet-55176785} 2015-11-22 19:29:34,711 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={logLevel=logLevel, jmx=jmx, org.mortbay.jetty.servlet.DefaultServlet-55176785=org.mortbay.jetty.servlet.DefaultServlet-55176785, stacks=stacks, conf=conf, metrics=metrics} 2015-11-22 19:29:34,711 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@41fecb8b - conf as servlet 2015-11-22 19:29:34,711 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@41fecb8b - jmx as servlet 2015-11-22 19:29:34,711 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@41fecb8b - metrics as servlet 2015-11-22 19:29:34,711 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@41fecb8b - logLevel as servlet 2015-11-22 19:29:34,711 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@41fecb8b - stacks as servlet 2015-11-22 19:29:34,711 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@41fecb8b - org.mortbay.jetty.servlet.DefaultServlet-55176785 as servlet 2015-11-22 19:29:34,711 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@41fecb8b - (S=conf,[/conf]) as servletMapping 2015-11-22 19:29:34,711 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@41fecb8b - (S=jmx,[/jmx]) as servletMapping 2015-11-22 19:29:34,712 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@41fecb8b - (S=metrics,[/metrics]) as servletMapping 2015-11-22 19:29:34,712 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@41fecb8b - (S=logLevel,[/logLevel]) as servletMapping 2015-11-22 19:29:34,712 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@41fecb8b - (S=stacks,[/stacks]) as servletMapping 2015-11-22 19:29:34,712 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@41fecb8b - (S=org.mortbay.jetty.servlet.DefaultServlet-55176785,[/]) as servletMapping 2015-11-22 19:29:34,712 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap=null 2015-11-22 19:29:34,712 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=null 2015-11-22 19:29:34,712 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:34,712 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap=null 2015-11-22 19:29:34,713 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap=null 2015-11-22 19:29:34,713 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.mortbay.jetty.webapp.WebAppContext@1ba9117e{/,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes/webapps/applicationhistory} 2015-11-22 19:29:34,713 INFO [main] mortbay.log (Slf4jLog.java:info(67)) - Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:0 2015-11-22 19:29:34,713 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:0 2015-11-22 19:29:34,713 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping Server@6631f5ca 2015-11-22 19:29:34,714 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping ContextHandlerCollection@5ace1ed4 2015-11-22 19:29:34,714 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping org.mortbay.jetty.servlet.Context@11efdbd3{/static,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes/webapps/static/} 2015-11-22 19:29:34,714 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping ServletHandler@1613674b 2015-11-22 19:29:34,714 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.mortbay.jetty.servlet.DefaultServlet-553657135 2015-11-22 19:29:34,714 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped ServletHandler@1613674b 2015-11-22 19:29:34,715 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping ErrorHandler@27ff5d15 2015-11-22 19:29:34,715 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped ErrorHandler@27ff5d15 2015-11-22 19:29:34,715 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.mortbay.jetty.servlet.Context@11efdbd3{/static,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes/webapps/static/} 2015-11-22 19:29:34,715 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping org.mortbay.jetty.servlet.Context@6366111e{/logs,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/log} 2015-11-22 19:29:34,716 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping ServletHandler@4e096385 2015-11-22 19:29:34,716 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.apache.hadoop.http.AdminAuthorizedServlet-798054875 2015-11-22 19:29:34,716 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped ServletHandler@4e096385 2015-11-22 19:29:34,716 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping ErrorHandler@649bec2e 2015-11-22 19:29:34,716 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped ErrorHandler@649bec2e 2015-11-22 19:29:34,716 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.mortbay.jetty.servlet.Context@6366111e{/logs,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/log} 2015-11-22 19:29:34,717 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped ContextHandlerCollection@5ace1ed4 2015-11-22 19:29:34,717 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.mortbay.jetty.servlet.HashSessionIdManager@693fe6c9 2015-11-22 19:29:34,817 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.mortbay.thread.QueuedThreadPool@32804bef 2015-11-22 19:29:34,817 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped Server@6631f5ca 2015-11-22 19:29:34,818 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:shutdown(588)) - refCount=3 2015-11-22 19:29:34,818 DEBUG [main] service.CompositeService (CompositeService.java:serviceStop(129)) - org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer: stopping services, size=5 2015-11-22 19:29:34,818 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #4: Service org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryManagerImpl in state org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryManagerImpl: STARTED 2015-11-22 19:29:34,818 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryManagerImpl entered state STOPPED 2015-11-22 19:29:34,818 INFO [main] applicationhistoryservice.ApplicationHistoryManagerImpl (ApplicationHistoryManagerImpl.java:serviceStop(78)) - Stopping ApplicationHistory 2015-11-22 19:29:34,818 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.applicationhistoryservice.MemoryApplicationHistoryStore entered state STOPPED 2015-11-22 19:29:34,818 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #3: Service ApplicationHistoryClientService in state ApplicationHistoryClientService: STARTED 2015-11-22 19:29:34,818 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: ApplicationHistoryClientService entered state STOPPED 2015-11-22 19:29:34,819 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 32936 2015-11-22 19:29:34,819 DEBUG [IPC Server handler 0 on 32936] ipc.Server (Server.java:run(2290)) - IPC Server handler 0 on 32936: exiting 2015-11-22 19:29:34,819 DEBUG [IPC Server handler 3 on 32936] ipc.Server (Server.java:run(2290)) - IPC Server handler 3 on 32936: exiting 2015-11-22 19:29:34,819 DEBUG [IPC Server handler 2 on 32936] ipc.Server (Server.java:run(2290)) - IPC Server handler 2 on 32936: exiting 2015-11-22 19:29:34,819 DEBUG [IPC Server handler 1 on 32936] ipc.Server (Server.java:run(2290)) - IPC Server handler 1 on 32936: exiting 2015-11-22 19:29:34,819 DEBUG [IPC Server handler 4 on 32936] ipc.Server (Server.java:run(2290)) - IPC Server handler 4 on 32936: exiting 2015-11-22 19:29:34,820 DEBUG [IPC Server handler 7 on 32936] ipc.Server (Server.java:run(2290)) - IPC Server handler 7 on 32936: exiting 2015-11-22 19:29:34,820 DEBUG [IPC Server handler 8 on 32936] ipc.Server (Server.java:run(2290)) - IPC Server handler 8 on 32936: exiting 2015-11-22 19:29:34,819 DEBUG [IPC Server handler 6 on 32936] ipc.Server (Server.java:run(2290)) - IPC Server handler 6 on 32936: exiting 2015-11-22 19:29:34,820 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #2: Service org.apache.hadoop.yarn.server.timeline.TimelineDataManager in state org.apache.hadoop.yarn.server.timeline.TimelineDataManager: STARTED 2015-11-22 19:29:34,820 INFO [IPC Server listener on 32936] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 32936 2015-11-22 19:29:34,820 DEBUG [IPC Server handler 5 on 32936] ipc.Server (Server.java:run(2290)) - IPC Server handler 5 on 32936: exiting 2015-11-22 19:29:34,820 DEBUG [IPC Server handler 9 on 32936] ipc.Server (Server.java:run(2290)) - IPC Server handler 9 on 32936: exiting 2015-11-22 19:29:34,821 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.timeline.TimelineDataManager entered state STOPPED 2015-11-22 19:29:34,820 DEBUG [IPC Server Responder] ipc.Server (Server.java:doRunLoop(997)) - Checking for old call responses. 2015-11-22 19:29:34,822 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:29:34,822 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #1: Service org.apache.hadoop.yarn.server.timeline.security.TimelineDelegationTokenSecretManagerService in state org.apache.hadoop.yarn.server.timeline.security.TimelineDelegationTokenSecretManagerService: STARTED 2015-11-22 19:29:34,823 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.timeline.security.TimelineDelegationTokenSecretManagerService entered state STOPPED 2015-11-22 19:29:34,823 DEBUG [main] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:stopThreads(619)) - Stopping expired delegation token remover thread 2015-11-22 19:29:34,823 ERROR [Thread[Thread-3662,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(670)) - ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted 2015-11-22 19:29:34,823 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #0: Service org.apache.hadoop.yarn.server.timeline.MemoryTimelineStore in state org.apache.hadoop.yarn.server.timeline.MemoryTimelineStore: STARTED 2015-11-22 19:29:34,823 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.timeline.MemoryTimelineStore entered state STOPPED 2015-11-22 19:29:34,823 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #1: Service org.apache.hadoop.yarn.server.MiniYARNCluster$NodeManagerWrapper_0 in state org.apache.hadoop.yarn.server.MiniYARNCluster$NodeManagerWrapper_0: STARTED 2015-11-22 19:29:34,823 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.MiniYARNCluster$NodeManagerWrapper_0 entered state STOPPED 2015-11-22 19:29:34,824 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: NodeManager entered state STOPPED 2015-11-22 19:29:34,824 DEBUG [main] service.CompositeService (CompositeService.java:serviceStop(129)) - NodeManager: stopping services, size=7 2015-11-22 19:29:34,824 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #6: Service org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl in state org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: STARTED 2015-11-22 19:29:34,824 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl entered state STOPPED 2015-11-22 19:29:34,824 INFO [main] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:unRegisterNM(268)) - Successfully Unregistered the Node ip-172-31-20-42.ap-northeast-1.compute.internal:50663 with ResourceManager. 2015-11-22 19:29:34,824 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #5: Service Dispatcher in state Dispatcher: STARTED 2015-11-22 19:29:34,824 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: Dispatcher entered state STOPPED 2015-11-22 19:29:34,825 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #4: Service org.apache.hadoop.yarn.server.nodemanager.webapp.WebServer in state org.apache.hadoop.yarn.server.nodemanager.webapp.WebServer: STARTED 2015-11-22 19:29:34,825 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.nodemanager.webapp.WebServer entered state STOPPED 2015-11-22 19:29:34,825 DEBUG [main] webapp.WebServer (WebServer.java:serviceStop(99)) - Stopping webapp 2015-11-22 19:29:34,826 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.mortbay.jetty.nio.SelectChannelConnector$1@34f5090e 2015-11-22 19:29:34,827 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping org.mortbay.jetty.webapp.WebAppContext@31e5415e{/,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes/webapps/node} 2015-11-22 19:29:34,827 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping SessionHandler@a2431d0 2015-11-22 19:29:34,827 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping SecurityHandler@1cbb87f3 2015-11-22 19:29:34,828 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping ServletHandler@1a4013 2015-11-22 19:29:34,828 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped guice 2015-11-22 19:29:34,828 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped Timeline Authentication Filter 2015-11-22 19:29:34,828 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped static_user_filter 2015-11-22 19:29:34,828 DEBUG [main] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:stopThreads(619)) - Stopping expired delegation token remover thread 2015-11-22 19:29:34,829 ERROR [Thread[Thread-3655,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(670)) - ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted 2015-11-22 19:29:34,829 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped RMAuthenticationFilter 2015-11-22 19:29:34,829 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped safety 2015-11-22 19:29:34,829 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped NoCacheFilter 2015-11-22 19:29:34,829 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped NoCacheFilter 2015-11-22 19:29:34,829 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped conf 2015-11-22 19:29:34,829 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped jmx 2015-11-22 19:29:34,830 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped metrics 2015-11-22 19:29:34,830 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped logLevel 2015-11-22 19:29:34,830 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped stacks 2015-11-22 19:29:34,830 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.mortbay.jetty.servlet.DefaultServlet$NIOResourceCache@1b6e1eff 2015-11-22 19:29:34,830 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.mortbay.jetty.servlet.DefaultServlet-376093630 2015-11-22 19:29:34,830 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped ServletHandler@1a4013 2015-11-22 19:29:34,831 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped SecurityHandler@1cbb87f3 2015-11-22 19:29:34,831 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.mortbay.jetty.servlet.HashSessionManager@306f16f3 2015-11-22 19:29:34,831 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped SessionHandler@a2431d0 2015-11-22 19:29:34,831 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping ErrorPageErrorHandler@702b8b12 2015-11-22 19:29:34,831 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped ErrorPageErrorHandler@702b8b12 2015-11-22 19:29:34,832 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@1a4013 - guice as filter 2015-11-22 19:29:34,832 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@1a4013 - Timeline Authentication Filter as filter 2015-11-22 19:29:34,832 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@1a4013 - static_user_filter as filter 2015-11-22 19:29:34,832 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@1a4013 - RMAuthenticationFilter as filter 2015-11-22 19:29:34,832 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@1a4013 - safety as filter 2015-11-22 19:29:34,833 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@1a4013 - NoCacheFilter as filter 2015-11-22 19:29:34,833 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@1a4013 - NoCacheFilter as filter 2015-11-22 19:29:34,833 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@1a4013 - (F=guice,[/*],[],15) as filterMapping 2015-11-22 19:29:34,833 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@1a4013 - (F=static_user_filter,[/ws/*],[],15) as filterMapping 2015-11-22 19:29:34,833 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@1a4013 - (F=RMAuthenticationFilter,[/ws/*],[],15) as filterMapping 2015-11-22 19:29:34,833 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@1a4013 - (F=static_user_filter,[/node/*],[],15) as filterMapping 2015-11-22 19:29:34,833 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@1a4013 - (F=RMAuthenticationFilter,[/node/*],[],15) as filterMapping 2015-11-22 19:29:34,833 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@1a4013 - (F=static_user_filter,[/conf],[],15) as filterMapping 2015-11-22 19:29:34,834 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@1a4013 - (F=RMAuthenticationFilter,[/conf],[],15) as filterMapping 2015-11-22 19:29:34,834 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@1a4013 - (F=static_user_filter,[/jmx],[],15) as filterMapping 2015-11-22 19:29:34,834 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@1a4013 - (F=RMAuthenticationFilter,[/jmx],[],15) as filterMapping 2015-11-22 19:29:34,834 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@1a4013 - (F=static_user_filter,[/metrics],[],15) as filterMapping 2015-11-22 19:29:34,834 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@1a4013 - (F=RMAuthenticationFilter,[/metrics],[],15) as filterMapping 2015-11-22 19:29:34,834 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@1a4013 - (F=static_user_filter,[/logLevel],[],15) as filterMapping 2015-11-22 19:29:34,834 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@1a4013 - (F=RMAuthenticationFilter,[/logLevel],[],15) as filterMapping 2015-11-22 19:29:34,834 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@1a4013 - (F=static_user_filter,[/stacks],[],15) as filterMapping 2015-11-22 19:29:34,835 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@1a4013 - (F=RMAuthenticationFilter,[/stacks],[],15) as filterMapping 2015-11-22 19:29:34,835 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@1a4013 - (F=Timeline Authentication Filter,[/*],[],15) as filterMapping 2015-11-22 19:29:34,835 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@1a4013 - (F=static_user_filter,[*.html, *.jsp],[],15) as filterMapping 2015-11-22 19:29:34,835 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@1a4013 - (F=RMAuthenticationFilter,[*.html, *.jsp],[],15) as filterMapping 2015-11-22 19:29:34,835 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@1a4013 - (F=safety,[/*],[],15) as filterMapping 2015-11-22 19:29:34,835 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@1a4013 - (F=NoCacheFilter,[/*],[],15) as filterMapping 2015-11-22 19:29:34,835 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@1a4013 - (F=NoCacheFilter,[/*],[],15) as filterMapping 2015-11-22 19:29:34,835 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap=null 2015-11-22 19:29:34,836 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=null 2015-11-22 19:29:34,836 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:34,836 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/jmx=jmx, /conf=conf, /stacks=stacks, /logLevel=logLevel, /metrics=metrics, /=org.mortbay.jetty.servlet.DefaultServlet-376093630} 2015-11-22 19:29:34,836 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={logLevel=logLevel, jmx=jmx, stacks=stacks, conf=conf, metrics=metrics, org.mortbay.jetty.servlet.DefaultServlet-376093630=org.mortbay.jetty.servlet.DefaultServlet-376093630} 2015-11-22 19:29:34,836 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@1a4013 - conf as servlet 2015-11-22 19:29:34,836 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@1a4013 - jmx as servlet 2015-11-22 19:29:34,836 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@1a4013 - metrics as servlet 2015-11-22 19:29:34,836 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@1a4013 - logLevel as servlet 2015-11-22 19:29:34,837 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@1a4013 - stacks as servlet 2015-11-22 19:29:34,837 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@1a4013 - org.mortbay.jetty.servlet.DefaultServlet-376093630 as servlet 2015-11-22 19:29:34,837 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@1a4013 - (S=conf,[/conf]) as servletMapping 2015-11-22 19:29:34,837 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@1a4013 - (S=jmx,[/jmx]) as servletMapping 2015-11-22 19:29:34,837 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@1a4013 - (S=metrics,[/metrics]) as servletMapping 2015-11-22 19:29:34,837 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@1a4013 - (S=logLevel,[/logLevel]) as servletMapping 2015-11-22 19:29:34,837 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@1a4013 - (S=stacks,[/stacks]) as servletMapping 2015-11-22 19:29:34,837 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@1a4013 - (S=org.mortbay.jetty.servlet.DefaultServlet-376093630,[/]) as servletMapping 2015-11-22 19:29:34,838 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap=null 2015-11-22 19:29:34,838 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=null 2015-11-22 19:29:34,838 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:34,838 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap=null 2015-11-22 19:29:34,838 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap=null 2015-11-22 19:29:34,838 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.mortbay.jetty.webapp.WebAppContext@31e5415e{/,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes/webapps/node} 2015-11-22 19:29:34,839 INFO [main] mortbay.log (Slf4jLog.java:info(67)) - Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:0 2015-11-22 19:29:34,839 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:0 2015-11-22 19:29:34,839 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping Server@5f9b2141 2015-11-22 19:29:34,839 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping ContextHandlerCollection@247d8ae 2015-11-22 19:29:34,839 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping org.mortbay.jetty.servlet.Context@8ce6003{/static,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes/webapps/static/} 2015-11-22 19:29:34,840 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping ServletHandler@48974e45 2015-11-22 19:29:34,840 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.mortbay.jetty.servlet.DefaultServlet-78686207 2015-11-22 19:29:34,840 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped ServletHandler@48974e45 2015-11-22 19:29:34,840 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping ErrorHandler@6a84a97d 2015-11-22 19:29:34,840 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped ErrorHandler@6a84a97d 2015-11-22 19:29:34,841 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.mortbay.jetty.servlet.Context@8ce6003{/static,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes/webapps/static/} 2015-11-22 19:29:34,841 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping org.mortbay.jetty.servlet.Context@6f0fae81{/logs,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/log} 2015-11-22 19:29:34,841 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping ServletHandler@6c130c45 2015-11-22 19:29:34,842 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.apache.hadoop.http.AdminAuthorizedServlet-754669547 2015-11-22 19:29:34,842 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped ServletHandler@6c130c45 2015-11-22 19:29:34,842 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping ErrorHandler@50ad3bc1 2015-11-22 19:29:34,842 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped ErrorHandler@50ad3bc1 2015-11-22 19:29:34,842 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.mortbay.jetty.servlet.Context@6f0fae81{/logs,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/log} 2015-11-22 19:29:34,843 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped ContextHandlerCollection@247d8ae 2015-11-22 19:29:34,843 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.mortbay.jetty.servlet.HashSessionIdManager@223aa2f7 2015-11-22 19:29:34,943 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.mortbay.thread.QueuedThreadPool@4a98cd5f 2015-11-22 19:29:34,943 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped Server@5f9b2141 2015-11-22 19:29:34,943 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #3: Service org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl in state org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl: STARTED 2015-11-22 19:29:34,944 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl entered state STOPPED 2015-11-22 19:29:34,944 INFO [main] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:cleanUpApplicationsOnNMShutDown(556)) - Applications still running : [application_1448220478671_0001] 2015-11-22 19:29:34,944 INFO [main] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:cleanUpApplicationsOnNMShutDown(572)) - Waiting for Applications to be Finished 2015-11-22 19:29:34,944 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationFinishEvent.EventType: FINISH_APPLICATION 2015-11-22 19:29:34,944 DEBUG [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(452)) - Processing application_1448220478671_0001 of type FINISH_APPLICATION 2015-11-22 19:29:34,944 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(463)) - Application application_1448220478671_0001 transitioned from RUNNING to FINISHING_CONTAINERS_WAIT 2015-11-22 19:29:34,944 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerKillEvent.EventType: KILL_CONTAINER 2015-11-22 19:29:34,944 DEBUG [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1131)) - Processing container_1448220478671_0001_01_000001 of type KILL_CONTAINER 2015-11-22 19:29:34,944 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220478671_0001_01_000001 transitioned from RUNNING to KILLING 2015-11-22 19:29:34,944 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncherEvent.EventType: CLEANUP_CONTAINER 2015-11-22 19:29:34,944 INFO [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(380)) - Cleaning up container container_1448220478671_0001_01_000001 2015-11-22 19:29:34,945 DEBUG [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(397)) - Marking container container_1448220478671_0001_01_000001 as inactive 2015-11-22 19:29:34,945 DEBUG [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(403)) - Getting pid for container container_1448220478671_0001_01_000001 to kill from pid file /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/application_1448220478671_0001/container_1448220478671_0001_01_000001/container_1448220478671_0001_01_000001.pid 2015-11-22 19:29:34,945 DEBUG [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:getContainerPid(569)) - Accessing pid for container container_1448220478671_0001_01_000001 from pid file /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/application_1448220478671_0001/container_1448220478671_0001_01_000001/container_1448220478671_0001_01_000001.pid 2015-11-22 19:29:34,945 DEBUG [AsyncDispatcher event handler] util.ProcessIdFileReader (ProcessIdFileReader.java:getProcessId(51)) - Accessing pid from pid file /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/application_1448220478671_0001/container_1448220478671_0001_01_000001/container_1448220478671_0001_01_000001.pid 2015-11-22 19:29:34,945 DEBUG [AsyncDispatcher event handler] util.ProcessIdFileReader (ProcessIdFileReader.java:getProcessId(99)) - Got pid 29996 from path /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/application_1448220478671_0001/container_1448220478671_0001_01_000001/container_1448220478671_0001_01_000001.pid 2015-11-22 19:29:34,945 DEBUG [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:getContainerPid(579)) - Got pid 29996 for container container_1448220478671_0001_01_000001 2015-11-22 19:29:34,945 DEBUG [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(421)) - Sending signal to pid 29996 as user ubuntu for container container_1448220478671_0001_01_000001 2015-11-22 19:29:34,945 DEBUG [AsyncDispatcher event handler] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:signalContainer(417)) - Sending signal 15 to pid 29996 as user ubuntu 2015-11-22 19:29:34,948 DEBUG [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(437)) - Sent signal SIGTERM to pid 29996 as user ubuntu for container container_1448220478671_0001_01_000001, result=success 2015-11-22 19:29:34,949 DEBUG [AsyncDispatcher event handler] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.fs.FileContext.getAbstractFileSystem(FileContext.java:339) 2015-11-22 19:29:34,956 WARN [ContainersLauncher #0] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(237)) - Exit code from container container_1448220478671_0001_01_000001 is : 143 2015-11-22 19:29:34,957 DEBUG [ContainersLauncher #0] container.ContainerImpl (ContainerImpl.java:handle(1131)) - Processing container_1448220478671_0001_01_000001 of type UPDATE_DIAGNOSTICS_MSG 2015-11-22 19:29:34,957 DEBUG [ContainersLauncher #0] launcher.ContainerLaunch (ContainerLaunch.java:call(333)) - Container container_1448220478671_0001_01_000001 completed with exit code 143 2015-11-22 19:29:34,961 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerExitEvent.EventType: CONTAINER_KILLED_ON_REQUEST 2015-11-22 19:29:34,961 DEBUG [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1131)) - Processing container_1448220478671_0001_01_000001 of type CONTAINER_KILLED_ON_REQUEST 2015-11-22 19:29:34,961 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220478671_0001_01_000001 transitioned from KILLING to CONTAINER_CLEANEDUP_AFTER_KILL 2015-11-22 19:29:34,961 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.ContainerLocalizationCleanupEvent.EventType: CLEANUP_CONTAINER_RESOURCES 2015-11-22 19:29:34,961 DEBUG [AsyncDispatcher event handler] localizer.LocalizedResource (LocalizedResource.java:handle(193)) - Processing file:/home/ubuntu/DistributedShell/application_1448220478671_0001/AppMaster.jar of type RELEASE 2015-11-22 19:29:34,961 DEBUG [AsyncDispatcher event handler] localizer.LocalizedResource (LocalizedResource.java:handle(193)) - Processing file:/home/ubuntu/DistributedShell/application_1448220478671_0001/shellCommands of type RELEASE 2015-11-22 19:29:34,961 DEBUG [AsyncDispatcher event handler] localizer.LocalizedResource (LocalizedResource.java:handle(193)) - Processing file:/home/ubuntu/DistributedShell/application_1448220478671_0001/shellArgs of type RELEASE 2015-11-22 19:29:34,962 DEBUG [AsyncDispatcher event handler] localizer.LocalizedResource (LocalizedResource.java:handle(193)) - Processing file:/home/ubuntu/DistributedShell/application_1448220478671_0001/log4j.properties of type RELEASE 2015-11-22 19:29:34,962 DEBUG [DeletionService #1] nodemanager.DeletionService (DeletionService.java:run(266)) - FileDeletionTask : user : ubuntu subDir : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220478671_0001/container_1448220478671_0001_01_000001 baseDir : null 2015-11-22 19:29:34,962 DEBUG [DeletionService #2] nodemanager.DeletionService (DeletionService.java:run(266)) - FileDeletionTask : user : null subDir : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/application_1448220478671_0001/container_1448220478671_0001_01_000001 baseDir : null 2015-11-22 19:29:34,962 DEBUG [Socket Reader #1 for port 46284] ipc.Server (Server.java:close(2894)) - Socket Reader #1 for port 46284: disconnecting client 172.31.20.42:53059. Number of active connections: 0 2015-11-22 19:29:34,962 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerEvent.EventType: CONTAINER_RESOURCES_CLEANEDUP 2015-11-22 19:29:34,962 DEBUG [DeletionService #2] nodemanager.DeletionService (DeletionService.java:run(271)) - NM deleting absolute path : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/application_1448220478671_0001/container_1448220478671_0001_01_000001 2015-11-22 19:29:34,962 DEBUG [DeletionService #1] nodemanager.DeletionService (DeletionService.java:run(292)) - Deleting path: [/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220478671_0001/container_1448220478671_0001_01_000001] as user: [ubuntu] 2015-11-22 19:29:34,962 INFO [DeletionService #1] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(480)) - Deleting absolute path : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220478671_0001/container_1448220478671_0001_01_000001 2015-11-22 19:29:34,962 DEBUG [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1131)) - Processing container_1448220478671_0001_01_000001 of type CONTAINER_RESOURCES_CLEANEDUP 2015-11-22 19:29:34,963 INFO [AsyncDispatcher event handler] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu OPERATION=Container Finished - Killed TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1448220478671_0001 CONTAINERID=container_1448220478671_0001_01_000001 2015-11-22 19:29:34,963 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220478671_0001_01_000001 transitioned from CONTAINER_CLEANEDUP_AFTER_KILL to DONE 2015-11-22 19:29:34,963 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationContainerFinishedEvent.EventType: APPLICATION_CONTAINER_FINISHED 2015-11-22 19:29:34,963 DEBUG [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(452)) - Processing application_1448220478671_0001 of type APPLICATION_CONTAINER_FINISHED 2015-11-22 19:29:34,963 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(401)) - Removing container_1448220478671_0001_01_000001 from application application_1448220478671_0001 2015-11-22 19:29:34,963 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(463)) - Application application_1448220478671_0001 transitioned from FINISHING_CONTAINERS_WAIT to APPLICATION_RESOURCES_CLEANINGUP 2015-11-22 19:29:34,963 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerStopMonitoringEvent.EventType: STOP_MONITORING_CONTAINER 2015-11-22 19:29:34,963 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.event.LogHandlerContainerFinishedEvent.EventType: CONTAINER_FINISHED 2015-11-22 19:29:34,963 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServicesEvent.EventType: CONTAINER_STOP 2015-11-22 19:29:34,963 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_STOP for appId application_1448220478671_0001 2015-11-22 19:29:34,963 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.ApplicationLocalizationEvent.EventType: DESTROY_APPLICATION_RESOURCES 2015-11-22 19:29:34,964 DEBUG [DeletionService #3] nodemanager.DeletionService (DeletionService.java:run(266)) - FileDeletionTask : user : ubuntu subDir : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220478671_0001 baseDir : null 2015-11-22 19:29:34,964 DEBUG [DeletionService #3] nodemanager.DeletionService (DeletionService.java:run(292)) - Deleting path: [/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220478671_0001] as user: [ubuntu] 2015-11-22 19:29:34,964 INFO [DeletionService #3] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(480)) - Deleting absolute path : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220478671_0001 2015-11-22 19:29:34,964 DEBUG [DeletionService #0] nodemanager.DeletionService (DeletionService.java:run(266)) - FileDeletionTask : user : null subDir : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/application_1448220478671_0001 baseDir : null 2015-11-22 19:29:34,964 DEBUG [DeletionService #0] nodemanager.DeletionService (DeletionService.java:run(271)) - NM deleting absolute path : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/application_1448220478671_0001 2015-11-22 19:29:34,964 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServicesEvent.EventType: APPLICATION_STOP 2015-11-22 19:29:34,964 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event APPLICATION_STOP for appId application_1448220478671_0001 2015-11-22 19:29:34,964 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationEvent.EventType: APPLICATION_RESOURCES_CLEANEDUP 2015-11-22 19:29:34,964 DEBUG [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(452)) - Processing application_1448220478671_0001 of type APPLICATION_RESOURCES_CLEANEDUP 2015-11-22 19:29:34,964 DEBUG [AsyncDispatcher event handler] security.NMTokenSecretManagerInNM (NMTokenSecretManagerInNM.java:appFinished(191)) - Removing application attempts NMToken keys for application application_1448220478671_0001 2015-11-22 19:29:34,965 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(463)) - Application application_1448220478671_0001 transitioned from APPLICATION_RESOURCES_CLEANINGUP to FINISHED 2015-11-22 19:29:34,965 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.event.LogHandlerAppFinishedEvent.EventType: APPLICATION_FINISHED 2015-11-22 19:29:34,965 INFO [AsyncDispatcher event handler] loghandler.NonAggregatingLogHandler (NonAggregatingLogHandler.java:handle(167)) - Scheduling Log Deletion for application: application_1448220478671_0001, with delay of 10800 seconds 2015-11-22 19:29:35,199 DEBUG [Task killer for 29996] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:signalContainer(417)) - Sending signal 9 to pid 29996 as user ubuntu 2015-11-22 19:29:35,699 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #916 2015-11-22 19:29:35,699 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #916 2015-11-22 19:29:35,699 DEBUG [IPC Server handler 33 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 33 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#916 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:35,699 DEBUG [IPC Server handler 33 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:35,699 DEBUG [IPC Server handler 33 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:29:35,700 DEBUG [IPC Server handler 33 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 1 2015-11-22 19:29:35,700 DEBUG [IPC Server handler 33 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 33 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#916 Retry#0 2015-11-22 19:29:35,700 DEBUG [IPC Server handler 33 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 33 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#916 Retry#0 Wrote 360 bytes. 2015-11-22 19:29:35,700 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #916 2015-11-22 19:29:35,700 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 1ms 2015-11-22 19:29:35,701 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:29:36,701 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #917 2015-11-22 19:29:36,701 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #917 2015-11-22 19:29:36,701 DEBUG [IPC Server handler 35 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 35 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#917 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:36,701 DEBUG [IPC Server handler 35 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:36,702 DEBUG [IPC Server handler 35 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:29:36,702 DEBUG [IPC Server handler 35 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 1 procesingTime= 0 2015-11-22 19:29:36,702 DEBUG [IPC Server handler 35 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 35 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#917 Retry#0 2015-11-22 19:29:36,702 DEBUG [IPC Server handler 35 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 35 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#917 Retry#0 Wrote 360 bytes. 2015-11-22 19:29:36,702 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #917 2015-11-22 19:29:36,703 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:29:36,703 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:29:37,703 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #918 2015-11-22 19:29:37,703 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #918 2015-11-22 19:29:37,704 DEBUG [IPC Server handler 36 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 36 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#918 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:37,704 DEBUG [IPC Server handler 36 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:37,704 DEBUG [IPC Server handler 36 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:29:37,704 DEBUG [IPC Server handler 36 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 0 2015-11-22 19:29:37,704 DEBUG [IPC Server handler 36 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 36 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#918 Retry#0 2015-11-22 19:29:37,704 DEBUG [IPC Server handler 36 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 36 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#918 Retry#0 Wrote 360 bytes. 2015-11-22 19:29:37,705 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #918 2015-11-22 19:29:37,705 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:29:37,705 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:29:38,680 DEBUG [IPC Server idle connection scanner for port 48629] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 48629: task running 2015-11-22 19:29:38,687 DEBUG [IPC Server idle connection scanner for port 46284] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 46284: task running 2015-11-22 19:29:38,694 DEBUG [IPC Server idle connection scanner for port 52028] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 52028: task running 2015-11-22 19:29:38,705 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #919 2015-11-22 19:29:38,705 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #919 2015-11-22 19:29:38,706 DEBUG [IPC Server handler 0 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 0 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#919 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:38,706 DEBUG [IPC Server handler 0 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:38,706 DEBUG [IPC Server handler 0 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:29:38,706 DEBUG [IPC Server handler 0 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 0 2015-11-22 19:29:38,706 DEBUG [IPC Server handler 0 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 0 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#919 Retry#0 2015-11-22 19:29:38,707 DEBUG [IPC Server handler 0 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 0 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#919 Retry#0 Wrote 360 bytes. 2015-11-22 19:29:38,707 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #919 2015-11-22 19:29:38,707 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:29:38,707 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:29:38,944 INFO [main] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:cleanUpApplicationsOnNMShutDown(589)) - Done waiting for Applications to be Finished. Still alive: [application_1448220478671_0001] 2015-11-22 19:29:38,945 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 50663 2015-11-22 19:29:38,945 DEBUG [IPC Server handler 0 on 50663] ipc.Server (Server.java:run(2290)) - IPC Server handler 0 on 50663: exiting 2015-11-22 19:29:38,945 DEBUG [IPC Server handler 3 on 50663] ipc.Server (Server.java:run(2290)) - IPC Server handler 3 on 50663: exiting 2015-11-22 19:29:38,945 DEBUG [IPC Server handler 8 on 50663] ipc.Server (Server.java:run(2290)) - IPC Server handler 8 on 50663: exiting 2015-11-22 19:29:38,946 DEBUG [IPC Server handler 5 on 50663] ipc.Server (Server.java:run(2290)) - IPC Server handler 5 on 50663: exiting 2015-11-22 19:29:38,946 DEBUG [IPC Server handler 12 on 50663] ipc.Server (Server.java:run(2290)) - IPC Server handler 12 on 50663: exiting 2015-11-22 19:29:38,946 DEBUG [IPC Server handler 13 on 50663] ipc.Server (Server.java:run(2290)) - IPC Server handler 13 on 50663: exiting 2015-11-22 19:29:38,946 DEBUG [IPC Server handler 15 on 50663] ipc.Server (Server.java:run(2290)) - IPC Server handler 15 on 50663: exiting 2015-11-22 19:29:38,945 DEBUG [IPC Server handler 6 on 50663] ipc.Server (Server.java:run(2290)) - IPC Server handler 6 on 50663: exiting 2015-11-22 19:29:38,945 DEBUG [IPC Server handler 4 on 50663] ipc.Server (Server.java:run(2290)) - IPC Server handler 4 on 50663: exiting 2015-11-22 19:29:38,945 DEBUG [IPC Server handler 1 on 50663] ipc.Server (Server.java:run(2290)) - IPC Server handler 1 on 50663: exiting 2015-11-22 19:29:38,946 INFO [IPC Server listener on 50663] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 50663 2015-11-22 19:29:38,946 DEBUG [IPC Server handler 19 on 50663] ipc.Server (Server.java:run(2290)) - IPC Server handler 19 on 50663: exiting 2015-11-22 19:29:38,946 DEBUG [IPC Server handler 18 on 50663] ipc.Server (Server.java:run(2290)) - IPC Server handler 18 on 50663: exiting 2015-11-22 19:29:38,946 DEBUG [IPC Server handler 17 on 50663] ipc.Server (Server.java:run(2290)) - IPC Server handler 17 on 50663: exiting 2015-11-22 19:29:38,946 DEBUG [IPC Server handler 16 on 50663] ipc.Server (Server.java:run(2290)) - IPC Server handler 16 on 50663: exiting 2015-11-22 19:29:38,946 DEBUG [IPC Server handler 11 on 50663] ipc.Server (Server.java:run(2290)) - IPC Server handler 11 on 50663: exiting 2015-11-22 19:29:38,946 DEBUG [IPC Server handler 14 on 50663] ipc.Server (Server.java:run(2290)) - IPC Server handler 14 on 50663: exiting 2015-11-22 19:29:38,946 DEBUG [IPC Server handler 10 on 50663] ipc.Server (Server.java:run(2290)) - IPC Server handler 10 on 50663: exiting 2015-11-22 19:29:38,946 DEBUG [IPC Server handler 9 on 50663] ipc.Server (Server.java:run(2290)) - IPC Server handler 9 on 50663: exiting 2015-11-22 19:29:38,945 DEBUG [IPC Server handler 2 on 50663] ipc.Server (Server.java:run(2290)) - IPC Server handler 2 on 50663: exiting 2015-11-22 19:29:38,945 DEBUG [IPC Server handler 7 on 50663] ipc.Server (Server.java:run(2290)) - IPC Server handler 7 on 50663: exiting 2015-11-22 19:29:38,950 DEBUG [IPC Server idle connection scanner for port 53669] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 53669: task running 2015-11-22 19:29:38,950 DEBUG [main] service.CompositeService (CompositeService.java:serviceStop(129)) - org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl: stopping services, size=7 2015-11-22 19:29:38,950 DEBUG [IPC Server Responder] ipc.Server (Server.java:doRunLoop(997)) - Checking for old call responses. 2015-11-22 19:29:38,951 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #6: Service org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.sharedcache.SharedCacheUploadService in state org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.sharedcache.SharedCacheUploadService: STARTED 2015-11-22 19:29:38,951 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:29:38,951 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.sharedcache.SharedCacheUploadService entered state STOPPED 2015-11-22 19:29:38,951 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #5: Service org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.NonAggregatingLogHandler in state org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.NonAggregatingLogHandler: STARTED 2015-11-22 19:29:38,951 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.NonAggregatingLogHandler entered state STOPPED 2015-11-22 19:29:39,707 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #920 2015-11-22 19:29:39,708 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #920 2015-11-22 19:29:39,708 DEBUG [IPC Server handler 1 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 1 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#920 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:39,708 DEBUG [IPC Server handler 1 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:39,708 DEBUG [IPC Server handler 1 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:29:39,709 DEBUG [IPC Server handler 1 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 1 2015-11-22 19:29:39,709 DEBUG [IPC Server handler 1 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 1 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#920 Retry#0 2015-11-22 19:29:39,709 DEBUG [IPC Server handler 1 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 1 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#920 Retry#0 Wrote 360 bytes. 2015-11-22 19:29:39,709 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #920 2015-11-22 19:29:39,709 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:29:39,709 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:29:40,185 DEBUG [IPC Server idle connection scanner for port 47651] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 47651: task running 2015-11-22 19:29:40,710 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #921 2015-11-22 19:29:40,710 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #921 2015-11-22 19:29:40,710 DEBUG [IPC Server handler 4 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 4 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#921 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:40,710 DEBUG [IPC Server handler 4 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:40,710 DEBUG [IPC Server handler 4 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:29:40,710 DEBUG [IPC Server handler 4 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 0 2015-11-22 19:29:40,711 DEBUG [IPC Server handler 4 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 4 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#921 Retry#0 2015-11-22 19:29:40,711 DEBUG [IPC Server handler 4 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 4 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#921 Retry#0 Wrote 360 bytes. 2015-11-22 19:29:40,711 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #921 2015-11-22 19:29:40,711 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:29:40,711 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:29:41,712 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #922 2015-11-22 19:29:41,712 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #922 2015-11-22 19:29:41,712 DEBUG [IPC Server handler 2 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 2 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#922 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:41,712 DEBUG [IPC Server handler 2 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:41,712 DEBUG [IPC Server handler 2 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:29:41,712 DEBUG [IPC Server handler 2 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 0 2015-11-22 19:29:41,713 DEBUG [IPC Server handler 2 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 2 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#922 Retry#0 2015-11-22 19:29:41,713 DEBUG [IPC Server handler 2 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 2 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#922 Retry#0 Wrote 360 bytes. 2015-11-22 19:29:41,713 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #922 2015-11-22 19:29:41,713 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:29:41,713 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:29:42,713 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #923 2015-11-22 19:29:42,714 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #923 2015-11-22 19:29:42,714 DEBUG [IPC Server handler 3 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 3 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#923 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:42,714 DEBUG [IPC Server handler 3 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:42,714 DEBUG [IPC Server handler 3 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:29:42,714 DEBUG [IPC Server handler 3 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 0 2015-11-22 19:29:42,714 DEBUG [IPC Server handler 3 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 3 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#923 Retry#0 2015-11-22 19:29:42,715 DEBUG [IPC Server handler 3 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 3 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#923 Retry#0 Wrote 360 bytes. 2015-11-22 19:29:42,715 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #923 2015-11-22 19:29:42,715 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:29:42,715 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:29:43,715 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #924 2015-11-22 19:29:43,716 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #924 2015-11-22 19:29:43,716 DEBUG [IPC Server handler 5 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 5 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#924 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:43,716 DEBUG [IPC Server handler 5 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:43,716 DEBUG [IPC Server handler 5 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:29:43,716 DEBUG [IPC Server handler 5 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 0 2015-11-22 19:29:43,717 DEBUG [IPC Server handler 5 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 5 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#924 Retry#0 2015-11-22 19:29:43,717 DEBUG [IPC Server handler 5 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 5 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#924 Retry#0 Wrote 360 bytes. 2015-11-22 19:29:43,717 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #924 2015-11-22 19:29:43,717 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:29:43,717 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:29:44,718 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #925 2015-11-22 19:29:44,718 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #925 2015-11-22 19:29:44,718 DEBUG [IPC Server handler 6 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 6 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#925 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:44,718 DEBUG [IPC Server handler 6 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:44,718 DEBUG [IPC Server handler 6 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:29:44,719 DEBUG [IPC Server handler 6 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 1 2015-11-22 19:29:44,719 DEBUG [IPC Server handler 6 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 6 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#925 Retry#0 2015-11-22 19:29:44,719 DEBUG [IPC Server handler 6 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 6 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#925 Retry#0 Wrote 360 bytes. 2015-11-22 19:29:44,719 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #925 2015-11-22 19:29:44,719 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:29:44,719 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:29:45,720 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #926 2015-11-22 19:29:45,720 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #926 2015-11-22 19:29:45,720 DEBUG [IPC Server handler 7 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 7 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#926 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:45,720 DEBUG [IPC Server handler 7 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:45,720 DEBUG [IPC Server handler 7 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:29:45,721 DEBUG [IPC Server handler 7 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 1 2015-11-22 19:29:45,721 DEBUG [IPC Server handler 7 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 7 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#926 Retry#0 2015-11-22 19:29:45,721 DEBUG [IPC Server handler 7 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 7 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#926 Retry#0 Wrote 360 bytes. 2015-11-22 19:29:45,721 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #926 2015-11-22 19:29:45,721 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 1ms 2015-11-22 19:29:45,722 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:29:46,722 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #927 2015-11-22 19:29:46,722 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #927 2015-11-22 19:29:46,722 DEBUG [IPC Server handler 8 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 8 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#927 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:46,722 DEBUG [IPC Server handler 8 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:46,723 DEBUG [IPC Server handler 8 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:29:46,723 DEBUG [IPC Server handler 8 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 1 procesingTime= 0 2015-11-22 19:29:46,723 DEBUG [IPC Server handler 8 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 8 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#927 Retry#0 2015-11-22 19:29:46,723 DEBUG [IPC Server handler 8 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 8 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#927 Retry#0 Wrote 360 bytes. 2015-11-22 19:29:46,723 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #927 2015-11-22 19:29:46,723 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 1ms 2015-11-22 19:29:46,724 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:29:47,724 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #928 2015-11-22 19:29:47,724 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #928 2015-11-22 19:29:47,724 DEBUG [IPC Server handler 10 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 10 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#928 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:47,725 DEBUG [IPC Server handler 10 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:47,725 DEBUG [IPC Server handler 10 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:29:47,725 DEBUG [IPC Server handler 10 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 1 procesingTime= 0 2015-11-22 19:29:47,725 DEBUG [IPC Server handler 10 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 10 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#928 Retry#0 2015-11-22 19:29:47,725 DEBUG [IPC Server handler 10 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 10 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#928 Retry#0 Wrote 360 bytes. 2015-11-22 19:29:47,725 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #928 2015-11-22 19:29:47,726 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 1ms 2015-11-22 19:29:47,726 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:29:48,681 DEBUG [IPC Server idle connection scanner for port 48629] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 48629: task running 2015-11-22 19:29:48,687 DEBUG [IPC Server idle connection scanner for port 46284] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 46284: task running 2015-11-22 19:29:48,694 DEBUG [IPC Server idle connection scanner for port 52028] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 52028: task running 2015-11-22 19:29:48,726 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #929 2015-11-22 19:29:48,726 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #929 2015-11-22 19:29:48,726 DEBUG [IPC Server handler 9 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 9 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#929 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:48,726 DEBUG [IPC Server handler 9 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:48,727 DEBUG [IPC Server handler 9 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:29:48,727 DEBUG [IPC Server handler 9 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 1 procesingTime= 0 2015-11-22 19:29:48,727 DEBUG [IPC Server handler 9 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 9 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#929 Retry#0 2015-11-22 19:29:48,727 DEBUG [IPC Server handler 9 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 9 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#929 Retry#0 Wrote 360 bytes. 2015-11-22 19:29:48,727 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #929 2015-11-22 19:29:48,728 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:29:48,728 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:29:48,951 DEBUG [IPC Server idle connection scanner for port 53669] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 53669: task running 2015-11-22 19:29:48,952 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #4: Service Dispatcher in state Dispatcher: STARTED 2015-11-22 19:29:48,952 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: Dispatcher entered state STOPPED 2015-11-22 19:29:48,953 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #3: Service containers-monitor in state containers-monitor: STARTED 2015-11-22 19:29:48,953 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: containers-monitor entered state STOPPED 2015-11-22 19:29:48,953 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #2: Service org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices in state org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices: STARTED 2015-11-22 19:29:48,953 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices entered state STOPPED 2015-11-22 19:29:48,953 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #1: Service containers-launcher in state containers-launcher: STARTED 2015-11-22 19:29:48,953 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: containers-launcher entered state STOPPED 2015-11-22 19:29:48,953 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #0: Service org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService in state org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: STARTED 2015-11-22 19:29:48,954 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService entered state STOPPED 2015-11-22 19:29:48,954 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 47651 2015-11-22 19:29:48,954 DEBUG [IPC Server handler 0 on 47651] ipc.Server (Server.java:run(2290)) - IPC Server handler 0 on 47651: exiting 2015-11-22 19:29:48,954 DEBUG [IPC Server handler 2 on 47651] ipc.Server (Server.java:run(2290)) - IPC Server handler 2 on 47651: exiting 2015-11-22 19:29:48,954 DEBUG [IPC Server handler 4 on 47651] ipc.Server (Server.java:run(2290)) - IPC Server handler 4 on 47651: exiting 2015-11-22 19:29:48,954 DEBUG [IPC Server handler 3 on 47651] ipc.Server (Server.java:run(2290)) - IPC Server handler 3 on 47651: exiting 2015-11-22 19:29:48,954 DEBUG [IPC Server handler 1 on 47651] ipc.Server (Server.java:run(2290)) - IPC Server handler 1 on 47651: exiting 2015-11-22 19:29:48,955 INFO [IPC Server listener on 47651] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 47651 2015-11-22 19:29:48,955 DEBUG [IPC Server Responder] ipc.Server (Server.java:doRunLoop(997)) - Checking for old call responses. 2015-11-22 19:29:48,955 DEBUG [main] service.CompositeService (CompositeService.java:serviceStop(129)) - org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: stopping services, size=1 2015-11-22 19:29:48,955 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:29:48,956 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #0: Service org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerTracker in state org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerTracker: STARTED 2015-11-22 19:29:48,956 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerTracker entered state STOPPED 2015-11-22 19:29:48,956 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #2: Service org.apache.hadoop.yarn.server.nodemanager.NodeResourceMonitorImpl in state org.apache.hadoop.yarn.server.nodemanager.NodeResourceMonitorImpl: STARTED 2015-11-22 19:29:48,956 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.nodemanager.NodeResourceMonitorImpl entered state STOPPED 2015-11-22 19:29:48,956 INFO [Public Localizer] localizer.ResourceLocalizationService (ResourceLocalizationService.java:run(908)) - Public cache exiting 2015-11-22 19:29:48,957 WARN [Node Resource Monitor] nodemanager.NodeResourceMonitorImpl (NodeResourceMonitorImpl.java:run(154)) - org.apache.hadoop.yarn.server.nodemanager.NodeResourceMonitorImpl is interrupted. Exiting. 2015-11-22 19:29:48,957 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #1: Service org.apache.hadoop.yarn.server.nodemanager.NodeHealthCheckerService in state org.apache.hadoop.yarn.server.nodemanager.NodeHealthCheckerService: STARTED 2015-11-22 19:29:48,957 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.nodemanager.NodeHealthCheckerService entered state STOPPED 2015-11-22 19:29:48,957 DEBUG [main] service.CompositeService (CompositeService.java:serviceStop(129)) - org.apache.hadoop.yarn.server.nodemanager.NodeHealthCheckerService: stopping services, size=1 2015-11-22 19:29:48,957 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #0: Service org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService in state org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService: STARTED 2015-11-22 19:29:48,958 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService entered state STOPPED 2015-11-22 19:29:48,958 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #0: Service org.apache.hadoop.yarn.server.nodemanager.DeletionService in state org.apache.hadoop.yarn.server.nodemanager.DeletionService: STARTED 2015-11-22 19:29:48,958 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.nodemanager.DeletionService entered state STOPPED 2015-11-22 19:29:48,959 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:shutdown(588)) - refCount=2 2015-11-22 19:29:48,960 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.nodemanager.recovery.NMNullStateStoreService entered state STOPPED 2015-11-22 19:29:48,960 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #0: Service org.apache.hadoop.yarn.server.MiniYARNCluster$ResourceManagerWrapper_0 in state org.apache.hadoop.yarn.server.MiniYARNCluster$ResourceManagerWrapper_0: STARTED 2015-11-22 19:29:48,960 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.MiniYARNCluster$ResourceManagerWrapper_0 entered state STOPPED 2015-11-22 19:29:49,728 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #930 2015-11-22 19:29:49,728 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #930 2015-11-22 19:29:49,729 DEBUG [IPC Server handler 12 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 12 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#930 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:49,729 DEBUG [IPC Server handler 12 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:49,729 DEBUG [IPC Server handler 12 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:29:49,729 DEBUG [IPC Server handler 12 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 1 procesingTime= 0 2015-11-22 19:29:49,729 DEBUG [IPC Server handler 12 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 12 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#930 Retry#0 2015-11-22 19:29:49,729 DEBUG [IPC Server handler 12 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 12 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#930 Retry#0 Wrote 360 bytes. 2015-11-22 19:29:49,729 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #930 2015-11-22 19:29:49,730 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:29:49,730 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:29:50,730 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #931 2015-11-22 19:29:50,730 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #931 2015-11-22 19:29:50,730 DEBUG [IPC Server handler 13 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 13 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#931 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:50,731 DEBUG [IPC Server handler 13 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:50,731 DEBUG [IPC Server handler 13 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:29:50,731 DEBUG [IPC Server handler 13 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 1 procesingTime= 0 2015-11-22 19:29:50,731 DEBUG [IPC Server handler 13 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 13 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#931 Retry#0 2015-11-22 19:29:50,731 DEBUG [IPC Server handler 13 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 13 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#931 Retry#0 Wrote 360 bytes. 2015-11-22 19:29:50,731 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #931 2015-11-22 19:29:50,732 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:29:50,732 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:29:51,732 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #932 2015-11-22 19:29:51,732 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #932 2015-11-22 19:29:51,733 DEBUG [IPC Server handler 16 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 16 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#932 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:51,733 DEBUG [IPC Server handler 16 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:51,733 DEBUG [IPC Server handler 16 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:29:51,733 DEBUG [IPC Server handler 16 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 0 2015-11-22 19:29:51,734 DEBUG [IPC Server handler 16 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 16 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#932 Retry#0 2015-11-22 19:29:51,734 DEBUG [IPC Server handler 16 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 16 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#932 Retry#0 Wrote 360 bytes. 2015-11-22 19:29:51,734 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #932 2015-11-22 19:29:51,734 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:29:51,734 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:29:52,734 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #933 2015-11-22 19:29:52,735 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #933 2015-11-22 19:29:52,735 DEBUG [IPC Server handler 15 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 15 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#933 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:52,735 DEBUG [IPC Server handler 15 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:52,735 DEBUG [IPC Server handler 15 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:29:52,735 DEBUG [IPC Server handler 15 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 0 2015-11-22 19:29:52,736 DEBUG [IPC Server handler 15 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 15 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#933 Retry#0 2015-11-22 19:29:52,736 DEBUG [IPC Server handler 15 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 15 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#933 Retry#0 Wrote 360 bytes. 2015-11-22 19:29:52,736 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #933 2015-11-22 19:29:52,736 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:29:52,736 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:29:53,737 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #934 2015-11-22 19:29:53,737 DEBUG [Socket Reader #1 for port 52028] ipc.Server (Server.java:processOneRpc(1897)) - got #934 2015-11-22 19:29:53,737 DEBUG [IPC Server handler 14 on 52028] ipc.Server (Server.java:run(2185)) - IPC Server handler 14 on 52028: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#934 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:29:53,737 DEBUG [IPC Server handler 14 on 52028] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:29:53,737 DEBUG [IPC Server handler 14 on 52028] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220478671_0001 owned by ubuntu 2015-11-22 19:29:53,738 DEBUG [IPC Server handler 14 on 52028] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 1 2015-11-22 19:29:53,738 DEBUG [IPC Server handler 14 on 52028] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 14 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#934 Retry#0 2015-11-22 19:29:53,738 DEBUG [IPC Server handler 14 on 52028] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 14 on 52028: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:53762 Call#934 Retry#0 Wrote 360 bytes. 2015-11-22 19:29:53,738 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu got value #934 2015-11-22 19:29:53,738 DEBUG [Thread-3684] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:29:53,738 INFO [Thread-3684] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220484786, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40307/proxy/application_1448220478671_0001/, appUser=ubuntu 2015-11-22 19:29:53,961 WARN [main] server.MiniYARNCluster (MiniYARNCluster.java:waitForAppMastersToFinish(474)) - Stopping RM while some app masters are still alive 2015-11-22 19:29:53,961 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: ResourceManager entered state STOPPED 2015-11-22 19:29:53,963 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.mortbay.jetty.nio.SelectChannelConnector$1@6d3a388c 2015-11-22 19:29:53,964 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping org.mortbay.jetty.webapp.WebAppContext@193f604a{/,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes/webapps/cluster} 2015-11-22 19:29:53,964 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping SessionHandler@20f5239f 2015-11-22 19:29:53,965 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping SecurityHandler@57af006c 2015-11-22 19:29:53,965 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping ServletHandler@6bedbc4d 2015-11-22 19:29:53,965 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped guice 2015-11-22 19:29:53,965 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped Timeline Authentication Filter 2015-11-22 19:29:53,965 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped static_user_filter 2015-11-22 19:29:53,966 DEBUG [main] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:stopThreads(619)) - Stopping expired delegation token remover thread 2015-11-22 19:29:53,966 ERROR [Thread[Thread-3605,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(670)) - ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted 2015-11-22 19:29:53,966 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped RMAuthenticationFilter 2015-11-22 19:29:53,966 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped safety 2015-11-22 19:29:53,966 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped NoCacheFilter 2015-11-22 19:29:53,967 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped NoCacheFilter 2015-11-22 19:29:53,967 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped proxy 2015-11-22 19:29:53,967 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped conf 2015-11-22 19:29:53,967 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped jmx 2015-11-22 19:29:53,967 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped metrics 2015-11-22 19:29:53,967 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped logLevel 2015-11-22 19:29:53,967 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped stacks 2015-11-22 19:29:53,968 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.mortbay.jetty.servlet.DefaultServlet$NIOResourceCache@932bc4a 2015-11-22 19:29:53,968 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.mortbay.jetty.servlet.DefaultServlet-1589830287 2015-11-22 19:29:53,968 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped ServletHandler@6bedbc4d 2015-11-22 19:29:53,968 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped SecurityHandler@57af006c 2015-11-22 19:29:53,968 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.mortbay.jetty.servlet.HashSessionManager@d29f28 2015-11-22 19:29:53,969 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped SessionHandler@20f5239f 2015-11-22 19:29:53,969 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping ErrorPageErrorHandler@2fd1433e 2015-11-22 19:29:53,969 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped ErrorPageErrorHandler@2fd1433e 2015-11-22 19:29:53,970 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@6bedbc4d - guice as filter 2015-11-22 19:29:53,970 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@6bedbc4d - Timeline Authentication Filter as filter 2015-11-22 19:29:53,970 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@6bedbc4d - static_user_filter as filter 2015-11-22 19:29:53,970 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@6bedbc4d - RMAuthenticationFilter as filter 2015-11-22 19:29:53,970 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@6bedbc4d - safety as filter 2015-11-22 19:29:53,970 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@6bedbc4d - NoCacheFilter as filter 2015-11-22 19:29:53,970 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@6bedbc4d - NoCacheFilter as filter 2015-11-22 19:29:53,970 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@6bedbc4d - (F=guice,[/*],[],15) as filterMapping 2015-11-22 19:29:53,971 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@6bedbc4d - (F=static_user_filter,[/proxy/*],[],15) as filterMapping 2015-11-22 19:29:53,971 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@6bedbc4d - (F=RMAuthenticationFilter,[/proxy/*],[],15) as filterMapping 2015-11-22 19:29:53,971 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@6bedbc4d - (F=static_user_filter,[/ws/*],[],15) as filterMapping 2015-11-22 19:29:53,971 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@6bedbc4d - (F=RMAuthenticationFilter,[/ws/*],[],15) as filterMapping 2015-11-22 19:29:53,971 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@6bedbc4d - (F=static_user_filter,[/cluster/*],[],15) as filterMapping 2015-11-22 19:29:53,971 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@6bedbc4d - (F=RMAuthenticationFilter,[/cluster/*],[],15) as filterMapping 2015-11-22 19:29:53,971 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@6bedbc4d - (F=static_user_filter,[/conf],[],15) as filterMapping 2015-11-22 19:29:53,971 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@6bedbc4d - (F=RMAuthenticationFilter,[/conf],[],15) as filterMapping 2015-11-22 19:29:53,972 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@6bedbc4d - (F=static_user_filter,[/jmx],[],15) as filterMapping 2015-11-22 19:29:53,972 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@6bedbc4d - (F=RMAuthenticationFilter,[/jmx],[],15) as filterMapping 2015-11-22 19:29:53,972 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@6bedbc4d - (F=static_user_filter,[/metrics],[],15) as filterMapping 2015-11-22 19:29:53,972 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@6bedbc4d - (F=RMAuthenticationFilter,[/metrics],[],15) as filterMapping 2015-11-22 19:29:53,972 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@6bedbc4d - (F=static_user_filter,[/logLevel],[],15) as filterMapping 2015-11-22 19:29:53,972 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@6bedbc4d - (F=RMAuthenticationFilter,[/logLevel],[],15) as filterMapping 2015-11-22 19:29:53,972 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@6bedbc4d - (F=static_user_filter,[/stacks],[],15) as filterMapping 2015-11-22 19:29:53,972 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@6bedbc4d - (F=RMAuthenticationFilter,[/stacks],[],15) as filterMapping 2015-11-22 19:29:53,972 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@6bedbc4d - (F=Timeline Authentication Filter,[/*],[],15) as filterMapping 2015-11-22 19:29:53,973 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@6bedbc4d - (F=static_user_filter,[*.html, *.jsp],[],15) as filterMapping 2015-11-22 19:29:53,973 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@6bedbc4d - (F=RMAuthenticationFilter,[*.html, *.jsp],[],15) as filterMapping 2015-11-22 19:29:53,973 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@6bedbc4d - (F=safety,[/*],[],15) as filterMapping 2015-11-22 19:29:53,973 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@6bedbc4d - (F=NoCacheFilter,[/*],[],15) as filterMapping 2015-11-22 19:29:53,973 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@6bedbc4d - (F=NoCacheFilter,[/*],[],15) as filterMapping 2015-11-22 19:29:53,973 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap=null 2015-11-22 19:29:53,973 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=null 2015-11-22 19:29:53,973 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:53,974 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/jmx=jmx, /proxy/*=proxy, /conf=conf, /stacks=stacks, /logLevel=logLevel, /metrics=metrics, /=org.mortbay.jetty.servlet.DefaultServlet-1589830287} 2015-11-22 19:29:53,974 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={proxy=proxy, logLevel=logLevel, jmx=jmx, stacks=stacks, conf=conf, metrics=metrics, org.mortbay.jetty.servlet.DefaultServlet-1589830287=org.mortbay.jetty.servlet.DefaultServlet-1589830287} 2015-11-22 19:29:53,974 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@6bedbc4d - proxy as servlet 2015-11-22 19:29:53,974 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@6bedbc4d - conf as servlet 2015-11-22 19:29:53,974 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@6bedbc4d - jmx as servlet 2015-11-22 19:29:53,974 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@6bedbc4d - metrics as servlet 2015-11-22 19:29:53,974 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@6bedbc4d - logLevel as servlet 2015-11-22 19:29:53,974 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@6bedbc4d - stacks as servlet 2015-11-22 19:29:53,975 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@6bedbc4d - org.mortbay.jetty.servlet.DefaultServlet-1589830287 as servlet 2015-11-22 19:29:53,975 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@6bedbc4d - (S=proxy,[/proxy/*]) as servletMapping 2015-11-22 19:29:53,975 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@6bedbc4d - (S=conf,[/conf]) as servletMapping 2015-11-22 19:29:53,975 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@6bedbc4d - (S=jmx,[/jmx]) as servletMapping 2015-11-22 19:29:53,975 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@6bedbc4d - (S=metrics,[/metrics]) as servletMapping 2015-11-22 19:29:53,975 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@6bedbc4d - (S=logLevel,[/logLevel]) as servletMapping 2015-11-22 19:29:53,975 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@6bedbc4d - (S=stacks,[/stacks]) as servletMapping 2015-11-22 19:29:53,975 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@6bedbc4d - (S=org.mortbay.jetty.servlet.DefaultServlet-1589830287,[/]) as servletMapping 2015-11-22 19:29:53,975 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap=null 2015-11-22 19:29:53,975 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=null 2015-11-22 19:29:53,976 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:53,976 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap=null 2015-11-22 19:29:53,976 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap=null 2015-11-22 19:29:53,976 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.mortbay.jetty.webapp.WebAppContext@193f604a{/,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes/webapps/cluster} 2015-11-22 19:29:53,976 INFO [main] mortbay.log (Slf4jLog.java:info(67)) - Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:0 2015-11-22 19:29:53,976 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:0 2015-11-22 19:29:53,977 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping Server@33990a0c 2015-11-22 19:29:53,977 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping ContextHandlerCollection@4de4b452 2015-11-22 19:29:53,977 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping org.mortbay.jetty.servlet.Context@59fcb93d{/static,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes/webapps/static/} 2015-11-22 19:29:53,978 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping ServletHandler@50b5ac82 2015-11-22 19:29:53,978 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.mortbay.jetty.servlet.DefaultServlet-2030062083 2015-11-22 19:29:53,978 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped ServletHandler@50b5ac82 2015-11-22 19:29:53,978 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping ErrorHandler@101952da 2015-11-22 19:29:53,978 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped ErrorHandler@101952da 2015-11-22 19:29:53,979 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.mortbay.jetty.servlet.Context@59fcb93d{/static,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes/webapps/static/} 2015-11-22 19:29:53,979 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping org.mortbay.jetty.servlet.Context@300a1cfb{/logs,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/log} 2015-11-22 19:29:53,979 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping ServletHandler@6babf3bf 2015-11-22 19:29:53,979 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.apache.hadoop.http.AdminAuthorizedServlet-2039828444 2015-11-22 19:29:53,979 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped ServletHandler@6babf3bf 2015-11-22 19:29:53,979 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping ErrorHandler@3059cbc 2015-11-22 19:29:53,980 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped ErrorHandler@3059cbc 2015-11-22 19:29:53,980 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.mortbay.jetty.servlet.Context@300a1cfb{/logs,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/log} 2015-11-22 19:29:53,980 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped ContextHandlerCollection@4de4b452 2015-11-22 19:29:53,980 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.mortbay.jetty.servlet.HashSessionIdManager@7ea9e1e2 2015-11-22 19:29:54,081 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.mortbay.thread.QueuedThreadPool@6ad950be 2015-11-22 19:29:54,081 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped Server@33990a0c 2015-11-22 19:29:54,081 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.ClientRMService entered state STOPPED 2015-11-22 19:29:54,082 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 52028 2015-11-22 19:29:54,082 DEBUG [IPC Server handler 0 on 52028] ipc.Server (Server.java:run(2290)) - IPC Server handler 0 on 52028: exiting 2015-11-22 19:29:54,082 DEBUG [IPC Server handler 2 on 52028] ipc.Server (Server.java:run(2290)) - IPC Server handler 2 on 52028: exiting 2015-11-22 19:29:54,082 DEBUG [IPC Server handler 4 on 52028] ipc.Server (Server.java:run(2290)) - IPC Server handler 4 on 52028: exiting 2015-11-22 19:29:54,082 DEBUG [IPC Server handler 3 on 52028] ipc.Server (Server.java:run(2290)) - IPC Server handler 3 on 52028: exiting 2015-11-22 19:29:54,082 DEBUG [IPC Server handler 7 on 52028] ipc.Server (Server.java:run(2290)) - IPC Server handler 7 on 52028: exiting 2015-11-22 19:29:54,082 DEBUG [IPC Server handler 1 on 52028] ipc.Server (Server.java:run(2290)) - IPC Server handler 1 on 52028: exiting 2015-11-22 19:29:54,082 DEBUG [IPC Server handler 11 on 52028] ipc.Server (Server.java:run(2290)) - IPC Server handler 11 on 52028: exiting 2015-11-22 19:29:54,082 DEBUG [IPC Server handler 10 on 52028] ipc.Server (Server.java:run(2290)) - IPC Server handler 10 on 52028: exiting 2015-11-22 19:29:54,083 DEBUG [IPC Server handler 13 on 52028] ipc.Server (Server.java:run(2290)) - IPC Server handler 13 on 52028: exiting 2015-11-22 19:29:54,082 DEBUG [IPC Server handler 6 on 52028] ipc.Server (Server.java:run(2290)) - IPC Server handler 6 on 52028: exiting 2015-11-22 19:29:54,083 DEBUG [IPC Server handler 8 on 52028] ipc.Server (Server.java:run(2290)) - IPC Server handler 8 on 52028: exiting 2015-11-22 19:29:54,082 DEBUG [IPC Server handler 5 on 52028] ipc.Server (Server.java:run(2290)) - IPC Server handler 5 on 52028: exiting 2015-11-22 19:29:54,083 DEBUG [IPC Server handler 9 on 52028] ipc.Server (Server.java:run(2290)) - IPC Server handler 9 on 52028: exiting 2015-11-22 19:29:54,084 DEBUG [IPC Server handler 22 on 52028] ipc.Server (Server.java:run(2290)) - IPC Server handler 22 on 52028: exiting 2015-11-22 19:29:54,084 DEBUG [main] service.CompositeService (CompositeService.java:serviceStop(129)) - ResourceManager: stopping services, size=4 2015-11-22 19:29:54,083 DEBUG [IPC Server handler 25 on 52028] ipc.Server (Server.java:run(2290)) - IPC Server handler 25 on 52028: exiting 2015-11-22 19:29:54,085 DEBUG [IPC Server handler 40 on 52028] ipc.Server (Server.java:run(2290)) - IPC Server handler 40 on 52028: exiting 2015-11-22 19:29:54,085 DEBUG [IPC Server handler 46 on 52028] ipc.Server (Server.java:run(2290)) - IPC Server handler 46 on 52028: exiting 2015-11-22 19:29:54,085 DEBUG [IPC Server handler 42 on 52028] ipc.Server (Server.java:run(2290)) - IPC Server handler 42 on 52028: exiting 2015-11-22 19:29:54,083 DEBUG [IPC Server handler 16 on 52028] ipc.Server (Server.java:run(2290)) - IPC Server handler 16 on 52028: exiting 2015-11-22 19:29:54,085 DEBUG [IPC Server handler 49 on 52028] ipc.Server (Server.java:run(2290)) - IPC Server handler 49 on 52028: exiting 2015-11-22 19:29:54,083 DEBUG [IPC Server handler 19 on 52028] ipc.Server (Server.java:run(2290)) - IPC Server handler 19 on 52028: exiting 2015-11-22 19:29:54,085 DEBUG [IPC Server handler 45 on 52028] ipc.Server (Server.java:run(2290)) - IPC Server handler 45 on 52028: exiting 2015-11-22 19:29:54,083 DEBUG [IPC Server handler 15 on 52028] ipc.Server (Server.java:run(2290)) - IPC Server handler 15 on 52028: exiting 2015-11-22 19:29:54,083 INFO [IPC Server listener on 52028] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 52028 2015-11-22 19:29:54,082 DEBUG [IPC Server handler 14 on 52028] ipc.Server (Server.java:run(2290)) - IPC Server handler 14 on 52028: exiting 2015-11-22 19:29:54,085 DEBUG [IPC Server handler 44 on 52028] ipc.Server (Server.java:run(2290)) - IPC Server handler 44 on 52028: exiting 2015-11-22 19:29:54,085 DEBUG [IPC Server handler 39 on 52028] ipc.Server (Server.java:run(2290)) - IPC Server handler 39 on 52028: exiting 2015-11-22 19:29:54,085 DEBUG [IPC Server handler 48 on 52028] ipc.Server (Server.java:run(2290)) - IPC Server handler 48 on 52028: exiting 2015-11-22 19:29:54,085 DEBUG [IPC Server handler 47 on 52028] ipc.Server (Server.java:run(2290)) - IPC Server handler 47 on 52028: exiting 2015-11-22 19:29:54,085 DEBUG [IPC Server handler 43 on 52028] ipc.Server (Server.java:run(2290)) - IPC Server handler 43 on 52028: exiting 2015-11-22 19:29:54,085 DEBUG [IPC Server handler 41 on 52028] ipc.Server (Server.java:run(2290)) - IPC Server handler 41 on 52028: exiting 2015-11-22 19:29:54,085 DEBUG [IPC Server handler 38 on 52028] ipc.Server (Server.java:run(2290)) - IPC Server handler 38 on 52028: exiting 2015-11-22 19:29:54,085 DEBUG [IPC Server handler 37 on 52028] ipc.Server (Server.java:run(2290)) - IPC Server handler 37 on 52028: exiting 2015-11-22 19:29:54,084 DEBUG [IPC Server handler 35 on 52028] ipc.Server (Server.java:run(2290)) - IPC Server handler 35 on 52028: exiting 2015-11-22 19:29:54,084 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #3: Service org.apache.hadoop.yarn.server.resourcemanager.metrics.SystemMetricsPublisher in state org.apache.hadoop.yarn.server.resourcemanager.metrics.SystemMetricsPublisher: STARTED 2015-11-22 19:29:54,087 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.metrics.SystemMetricsPublisher entered state STOPPED 2015-11-22 19:29:54,087 DEBUG [main] service.CompositeService (CompositeService.java:serviceStop(129)) - org.apache.hadoop.yarn.server.resourcemanager.metrics.SystemMetricsPublisher: stopping services, size=0 2015-11-22 19:29:54,084 DEBUG [IPC Server handler 36 on 52028] ipc.Server (Server.java:run(2290)) - IPC Server handler 36 on 52028: exiting 2015-11-22 19:29:54,084 DEBUG [IPC Server handler 34 on 52028] ipc.Server (Server.java:run(2290)) - IPC Server handler 34 on 52028: exiting 2015-11-22 19:29:54,084 DEBUG [IPC Server handler 33 on 52028] ipc.Server (Server.java:run(2290)) - IPC Server handler 33 on 52028: exiting 2015-11-22 19:29:54,084 DEBUG [IPC Server handler 24 on 52028] ipc.Server (Server.java:run(2290)) - IPC Server handler 24 on 52028: exiting 2015-11-22 19:29:54,084 DEBUG [IPC Server handler 29 on 52028] ipc.Server (Server.java:run(2290)) - IPC Server handler 29 on 52028: exiting 2015-11-22 19:29:54,084 DEBUG [IPC Server handler 26 on 52028] ipc.Server (Server.java:run(2290)) - IPC Server handler 26 on 52028: exiting 2015-11-22 19:29:54,084 DEBUG [IPC Server handler 32 on 52028] ipc.Server (Server.java:run(2290)) - IPC Server handler 32 on 52028: exiting 2015-11-22 19:29:54,084 DEBUG [IPC Server handler 28 on 52028] ipc.Server (Server.java:run(2290)) - IPC Server handler 28 on 52028: exiting 2015-11-22 19:29:54,084 DEBUG [IPC Server handler 21 on 52028] ipc.Server (Server.java:run(2290)) - IPC Server handler 21 on 52028: exiting 2015-11-22 19:29:54,084 DEBUG [IPC Server handler 20 on 52028] ipc.Server (Server.java:run(2290)) - IPC Server handler 20 on 52028: exiting 2015-11-22 19:29:54,084 DEBUG [IPC Server handler 31 on 52028] ipc.Server (Server.java:run(2290)) - IPC Server handler 31 on 52028: exiting 2015-11-22 19:29:54,084 DEBUG [IPC Server handler 30 on 52028] ipc.Server (Server.java:run(2290)) - IPC Server handler 30 on 52028: exiting 2015-11-22 19:29:54,084 DEBUG [IPC Server handler 27 on 52028] ipc.Server (Server.java:run(2290)) - IPC Server handler 27 on 52028: exiting 2015-11-22 19:29:54,084 DEBUG [IPC Server handler 23 on 52028] ipc.Server (Server.java:run(2290)) - IPC Server handler 23 on 52028: exiting 2015-11-22 19:29:54,084 DEBUG [IPC Server handler 17 on 52028] ipc.Server (Server.java:run(2290)) - IPC Server handler 17 on 52028: exiting 2015-11-22 19:29:54,084 DEBUG [IPC Server Responder] ipc.Server (Server.java:doRunLoop(997)) - Checking for old call responses. 2015-11-22 19:29:54,084 DEBUG [IPC Server handler 18 on 52028] ipc.Server (Server.java:run(2290)) - IPC Server handler 18 on 52028: exiting 2015-11-22 19:29:54,084 DEBUG [IPC Server handler 12 on 52028] ipc.Server (Server.java:run(2290)) - IPC Server handler 12 on 52028: exiting 2015-11-22 19:29:54,088 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:29:54,087 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #2: Service org.apache.hadoop.yarn.server.resourcemanager.ahs.RMApplicationHistoryWriter in state org.apache.hadoop.yarn.server.resourcemanager.ahs.RMApplicationHistoryWriter: STARTED 2015-11-22 19:29:54,089 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.ahs.RMApplicationHistoryWriter entered state STOPPED 2015-11-22 19:29:54,089 DEBUG [main] service.CompositeService (CompositeService.java:serviceStop(129)) - org.apache.hadoop.yarn.server.resourcemanager.ahs.RMApplicationHistoryWriter: stopping services, size=0 2015-11-22 19:29:54,085 DEBUG [IPC Server listener on 52028] ipc.Server (Server.java:close(2894)) - IPC Server listener on 52028: disconnecting client 172.31.20.42:53762. Number of active connections: 0 2015-11-22 19:29:54,089 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #1: Service org.apache.hadoop.yarn.server.resourcemanager.AdminService in state org.apache.hadoop.yarn.server.resourcemanager.AdminService: STARTED 2015-11-22 19:29:54,089 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.AdminService entered state STOPPED 2015-11-22 19:29:54,089 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 53669 2015-11-22 19:29:54,090 DEBUG [IPC Server handler 0 on 53669] ipc.Server (Server.java:run(2290)) - IPC Server handler 0 on 53669: exiting 2015-11-22 19:29:54,090 INFO [IPC Server listener on 53669] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 53669 2015-11-22 19:29:54,090 DEBUG [main] service.CompositeService (CompositeService.java:serviceStop(129)) - org.apache.hadoop.yarn.server.resourcemanager.AdminService: stopping services, size=0 2015-11-22 19:29:54,090 DEBUG [IPC Server Responder] ipc.Server (Server.java:doRunLoop(997)) - Checking for old call responses. 2015-11-22 19:29:54,091 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:29:54,090 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #0: Service Dispatcher in state Dispatcher: STARTED 2015-11-22 19:29:54,091 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: Dispatcher entered state STOPPED 2015-11-22 19:29:54,092 INFO [main] resourcemanager.ResourceManager (ResourceManager.java:transitionToStandby(1051)) - Transitioning to standby state 2015-11-22 19:29:54,092 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: RMActiveServices entered state STOPPED 2015-11-22 19:29:54,092 DEBUG [main] service.CompositeService (CompositeService.java:serviceStop(129)) - RMActiveServices: stopping services, size=13 2015-11-22 19:29:54,092 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #12: Service org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher in state org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher: STARTED 2015-11-22 19:29:54,092 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher entered state STOPPED 2015-11-22 19:29:54,093 WARN [ApplicationMaster Launcher] amlauncher.ApplicationMasterLauncher (ApplicationMasterLauncher.java:run(122)) - org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher$LauncherThread interrupted. Returning. 2015-11-22 19:29:54,093 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #11: Service org.apache.hadoop.yarn.server.resourcemanager.ClientRMService in state org.apache.hadoop.yarn.server.resourcemanager.ClientRMService: STOPPED 2015-11-22 19:29:54,093 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #10: Service org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService in state org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService: STARTED 2015-11-22 19:29:54,093 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService entered state STOPPED 2015-11-22 19:29:54,094 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 46284 2015-11-22 19:29:54,094 DEBUG [IPC Server handler 0 on 46284] ipc.Server (Server.java:run(2290)) - IPC Server handler 0 on 46284: exiting 2015-11-22 19:29:54,094 DEBUG [IPC Server handler 1 on 46284] ipc.Server (Server.java:run(2290)) - IPC Server handler 1 on 46284: exiting 2015-11-22 19:29:54,094 DEBUG [IPC Server handler 2 on 46284] ipc.Server (Server.java:run(2290)) - IPC Server handler 2 on 46284: exiting 2015-11-22 19:29:54,094 DEBUG [IPC Server handler 5 on 46284] ipc.Server (Server.java:run(2290)) - IPC Server handler 5 on 46284: exiting 2015-11-22 19:29:54,094 DEBUG [IPC Server handler 6 on 46284] ipc.Server (Server.java:run(2290)) - IPC Server handler 6 on 46284: exiting 2015-11-22 19:29:54,094 DEBUG [IPC Server handler 8 on 46284] ipc.Server (Server.java:run(2290)) - IPC Server handler 8 on 46284: exiting 2015-11-22 19:29:54,094 DEBUG [IPC Server handler 7 on 46284] ipc.Server (Server.java:run(2290)) - IPC Server handler 7 on 46284: exiting 2015-11-22 19:29:54,095 DEBUG [IPC Server handler 11 on 46284] ipc.Server (Server.java:run(2290)) - IPC Server handler 11 on 46284: exiting 2015-11-22 19:29:54,095 DEBUG [IPC Server handler 12 on 46284] ipc.Server (Server.java:run(2290)) - IPC Server handler 12 on 46284: exiting 2015-11-22 19:29:54,095 DEBUG [IPC Server handler 10 on 46284] ipc.Server (Server.java:run(2290)) - IPC Server handler 10 on 46284: exiting 2015-11-22 19:29:54,095 DEBUG [IPC Server handler 14 on 46284] ipc.Server (Server.java:run(2290)) - IPC Server handler 14 on 46284: exiting 2015-11-22 19:29:54,094 DEBUG [IPC Server handler 9 on 46284] ipc.Server (Server.java:run(2290)) - IPC Server handler 9 on 46284: exiting 2015-11-22 19:29:54,095 DEBUG [IPC Server handler 20 on 46284] ipc.Server (Server.java:run(2290)) - IPC Server handler 20 on 46284: exiting 2015-11-22 19:29:54,096 DEBUG [IPC Server handler 15 on 46284] ipc.Server (Server.java:run(2290)) - IPC Server handler 15 on 46284: exiting 2015-11-22 19:29:54,096 DEBUG [IPC Server handler 18 on 46284] ipc.Server (Server.java:run(2290)) - IPC Server handler 18 on 46284: exiting 2015-11-22 19:29:54,096 DEBUG [IPC Server handler 25 on 46284] ipc.Server (Server.java:run(2290)) - IPC Server handler 25 on 46284: exiting 2015-11-22 19:29:54,096 DEBUG [IPC Server Responder] ipc.Server (Server.java:doRunLoop(997)) - Checking for old call responses. 2015-11-22 19:29:54,096 DEBUG [IPC Server handler 23 on 46284] ipc.Server (Server.java:run(2290)) - IPC Server handler 23 on 46284: exiting 2015-11-22 19:29:54,096 DEBUG [IPC Server handler 28 on 46284] ipc.Server (Server.java:run(2290)) - IPC Server handler 28 on 46284: exiting 2015-11-22 19:29:54,096 DEBUG [IPC Server handler 31 on 46284] ipc.Server (Server.java:run(2290)) - IPC Server handler 31 on 46284: exiting 2015-11-22 19:29:54,096 DEBUG [IPC Server handler 16 on 46284] ipc.Server (Server.java:run(2290)) - IPC Server handler 16 on 46284: exiting 2015-11-22 19:29:54,096 DEBUG [IPC Server handler 38 on 46284] ipc.Server (Server.java:run(2290)) - IPC Server handler 38 on 46284: exiting 2015-11-22 19:29:54,096 DEBUG [IPC Server handler 42 on 46284] ipc.Server (Server.java:run(2290)) - IPC Server handler 42 on 46284: exiting 2015-11-22 19:29:54,096 DEBUG [IPC Server handler 43 on 46284] ipc.Server (Server.java:run(2290)) - IPC Server handler 43 on 46284: exiting 2015-11-22 19:29:54,097 DEBUG [IPC Server handler 33 on 46284] ipc.Server (Server.java:run(2290)) - IPC Server handler 33 on 46284: exiting 2015-11-22 19:29:54,096 DEBUG [IPC Server handler 13 on 46284] ipc.Server (Server.java:run(2290)) - IPC Server handler 13 on 46284: exiting 2015-11-22 19:29:54,097 DEBUG [IPC Server handler 39 on 46284] ipc.Server (Server.java:run(2290)) - IPC Server handler 39 on 46284: exiting 2015-11-22 19:29:54,097 DEBUG [IPC Server handler 46 on 46284] ipc.Server (Server.java:run(2290)) - IPC Server handler 46 on 46284: exiting 2015-11-22 19:29:54,096 DEBUG [IPC Server handler 3 on 46284] ipc.Server (Server.java:run(2290)) - IPC Server handler 3 on 46284: exiting 2015-11-22 19:29:54,097 DEBUG [IPC Server handler 49 on 46284] ipc.Server (Server.java:run(2290)) - IPC Server handler 49 on 46284: exiting 2015-11-22 19:29:54,096 DEBUG [IPC Server handler 4 on 46284] ipc.Server (Server.java:run(2290)) - IPC Server handler 4 on 46284: exiting 2015-11-22 19:29:54,096 DEBUG [IPC Server handler 19 on 46284] ipc.Server (Server.java:run(2290)) - IPC Server handler 19 on 46284: exiting 2015-11-22 19:29:54,097 DEBUG [IPC Server handler 48 on 46284] ipc.Server (Server.java:run(2290)) - IPC Server handler 48 on 46284: exiting 2015-11-22 19:29:54,097 DEBUG [IPC Server handler 47 on 46284] ipc.Server (Server.java:run(2290)) - IPC Server handler 47 on 46284: exiting 2015-11-22 19:29:54,097 DEBUG [IPC Server handler 45 on 46284] ipc.Server (Server.java:run(2290)) - IPC Server handler 45 on 46284: exiting 2015-11-22 19:29:54,097 DEBUG [IPC Server handler 34 on 46284] ipc.Server (Server.java:run(2290)) - IPC Server handler 34 on 46284: exiting 2015-11-22 19:29:54,097 DEBUG [IPC Server handler 44 on 46284] ipc.Server (Server.java:run(2290)) - IPC Server handler 44 on 46284: exiting 2015-11-22 19:29:54,097 DEBUG [IPC Server handler 32 on 46284] ipc.Server (Server.java:run(2290)) - IPC Server handler 32 on 46284: exiting 2015-11-22 19:29:54,096 DEBUG [IPC Server handler 41 on 46284] ipc.Server (Server.java:run(2290)) - IPC Server handler 41 on 46284: exiting 2015-11-22 19:29:54,096 DEBUG [IPC Server handler 40 on 46284] ipc.Server (Server.java:run(2290)) - IPC Server handler 40 on 46284: exiting 2015-11-22 19:29:54,096 DEBUG [IPC Server handler 36 on 46284] ipc.Server (Server.java:run(2290)) - IPC Server handler 36 on 46284: exiting 2015-11-22 19:29:54,096 DEBUG [IPC Server handler 37 on 46284] ipc.Server (Server.java:run(2290)) - IPC Server handler 37 on 46284: exiting 2015-11-22 19:29:54,096 DEBUG [IPC Server handler 35 on 46284] ipc.Server (Server.java:run(2290)) - IPC Server handler 35 on 46284: exiting 2015-11-22 19:29:54,096 DEBUG [IPC Server handler 27 on 46284] ipc.Server (Server.java:run(2290)) - IPC Server handler 27 on 46284: exiting 2015-11-22 19:29:54,096 DEBUG [IPC Server handler 30 on 46284] ipc.Server (Server.java:run(2290)) - IPC Server handler 30 on 46284: exiting 2015-11-22 19:29:54,096 DEBUG [IPC Server handler 29 on 46284] ipc.Server (Server.java:run(2290)) - IPC Server handler 29 on 46284: exiting 2015-11-22 19:29:54,096 DEBUG [IPC Server handler 17 on 46284] ipc.Server (Server.java:run(2290)) - IPC Server handler 17 on 46284: exiting 2015-11-22 19:29:54,096 DEBUG [IPC Server handler 26 on 46284] ipc.Server (Server.java:run(2290)) - IPC Server handler 26 on 46284: exiting 2015-11-22 19:29:54,096 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:29:54,096 DEBUG [IPC Server handler 22 on 46284] ipc.Server (Server.java:run(2290)) - IPC Server handler 22 on 46284: exiting 2015-11-22 19:29:54,096 INFO [IPC Server listener on 46284] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 46284 2015-11-22 19:29:54,096 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #9: Service org.apache.hadoop.yarn.server.resourcemanager.ResourceTrackerService in state org.apache.hadoop.yarn.server.resourcemanager.ResourceTrackerService: STARTED 2015-11-22 19:29:54,096 DEBUG [IPC Server handler 24 on 46284] ipc.Server (Server.java:run(2290)) - IPC Server handler 24 on 46284: exiting 2015-11-22 19:29:54,096 DEBUG [IPC Server handler 21 on 46284] ipc.Server (Server.java:run(2290)) - IPC Server handler 21 on 46284: exiting 2015-11-22 19:29:54,100 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.ResourceTrackerService entered state STOPPED 2015-11-22 19:29:54,100 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 48629 2015-11-22 19:29:54,100 DEBUG [IPC Server handler 0 on 48629] ipc.Server (Server.java:run(2290)) - IPC Server handler 0 on 48629: exiting 2015-11-22 19:29:54,100 DEBUG [IPC Server handler 1 on 48629] ipc.Server (Server.java:run(2290)) - IPC Server handler 1 on 48629: exiting 2015-11-22 19:29:54,100 DEBUG [IPC Server handler 5 on 48629] ipc.Server (Server.java:run(2290)) - IPC Server handler 5 on 48629: exiting 2015-11-22 19:29:54,100 DEBUG [IPC Server handler 4 on 48629] ipc.Server (Server.java:run(2290)) - IPC Server handler 4 on 48629: exiting 2015-11-22 19:29:54,100 DEBUG [IPC Server handler 6 on 48629] ipc.Server (Server.java:run(2290)) - IPC Server handler 6 on 48629: exiting 2015-11-22 19:29:54,100 DEBUG [IPC Server handler 9 on 48629] ipc.Server (Server.java:run(2290)) - IPC Server handler 9 on 48629: exiting 2015-11-22 19:29:54,100 DEBUG [IPC Server handler 3 on 48629] ipc.Server (Server.java:run(2290)) - IPC Server handler 3 on 48629: exiting 2015-11-22 19:29:54,100 DEBUG [IPC Server handler 2 on 48629] ipc.Server (Server.java:run(2290)) - IPC Server handler 2 on 48629: exiting 2015-11-22 19:29:54,100 DEBUG [IPC Server handler 8 on 48629] ipc.Server (Server.java:run(2290)) - IPC Server handler 8 on 48629: exiting 2015-11-22 19:29:54,100 DEBUG [IPC Server handler 7 on 48629] ipc.Server (Server.java:run(2290)) - IPC Server handler 7 on 48629: exiting 2015-11-22 19:29:54,101 DEBUG [IPC Server handler 11 on 48629] ipc.Server (Server.java:run(2290)) - IPC Server handler 11 on 48629: exiting 2015-11-22 19:29:54,101 DEBUG [IPC Server handler 15 on 48629] ipc.Server (Server.java:run(2290)) - IPC Server handler 15 on 48629: exiting 2015-11-22 19:29:54,101 DEBUG [IPC Server handler 18 on 48629] ipc.Server (Server.java:run(2290)) - IPC Server handler 18 on 48629: exiting 2015-11-22 19:29:54,101 DEBUG [IPC Server handler 22 on 48629] ipc.Server (Server.java:run(2290)) - IPC Server handler 22 on 48629: exiting 2015-11-22 19:29:54,101 DEBUG [IPC Server handler 25 on 48629] ipc.Server (Server.java:run(2290)) - IPC Server handler 25 on 48629: exiting 2015-11-22 19:29:54,101 DEBUG [IPC Server handler 13 on 48629] ipc.Server (Server.java:run(2290)) - IPC Server handler 13 on 48629: exiting 2015-11-22 19:29:54,102 DEBUG [IPC Server handler 33 on 48629] ipc.Server (Server.java:run(2290)) - IPC Server handler 33 on 48629: exiting 2015-11-22 19:29:54,101 DEBUG [IPC Server handler 12 on 48629] ipc.Server (Server.java:run(2290)) - IPC Server handler 12 on 48629: exiting 2015-11-22 19:29:54,101 DEBUG [IPC Server handler 10 on 48629] ipc.Server (Server.java:run(2290)) - IPC Server handler 10 on 48629: exiting 2015-11-22 19:29:54,103 DEBUG [IPC Server handler 36 on 48629] ipc.Server (Server.java:run(2290)) - IPC Server handler 36 on 48629: exiting 2015-11-22 19:29:54,103 DEBUG [IPC Server handler 38 on 48629] ipc.Server (Server.java:run(2290)) - IPC Server handler 38 on 48629: exiting 2015-11-22 19:29:54,103 DEBUG [IPC Server handler 40 on 48629] ipc.Server (Server.java:run(2290)) - IPC Server handler 40 on 48629: exiting 2015-11-22 19:29:54,103 DEBUG [IPC Server handler 41 on 48629] ipc.Server (Server.java:run(2290)) - IPC Server handler 41 on 48629: exiting 2015-11-22 19:29:54,103 DEBUG [IPC Server handler 43 on 48629] ipc.Server (Server.java:run(2290)) - IPC Server handler 43 on 48629: exiting 2015-11-22 19:29:54,103 DEBUG [IPC Server handler 45 on 48629] ipc.Server (Server.java:run(2290)) - IPC Server handler 45 on 48629: exiting 2015-11-22 19:29:54,103 DEBUG [IPC Server handler 44 on 48629] ipc.Server (Server.java:run(2290)) - IPC Server handler 44 on 48629: exiting 2015-11-22 19:29:54,102 DEBUG [IPC Server handler 28 on 48629] ipc.Server (Server.java:run(2290)) - IPC Server handler 28 on 48629: exiting 2015-11-22 19:29:54,103 DEBUG [IPC Server handler 46 on 48629] ipc.Server (Server.java:run(2290)) - IPC Server handler 46 on 48629: exiting 2015-11-22 19:29:54,102 DEBUG [IPC Server handler 30 on 48629] ipc.Server (Server.java:run(2290)) - IPC Server handler 30 on 48629: exiting 2015-11-22 19:29:54,102 DEBUG [IPC Server handler 27 on 48629] ipc.Server (Server.java:run(2290)) - IPC Server handler 27 on 48629: exiting 2015-11-22 19:29:54,102 DEBUG [IPC Server Responder] ipc.Server (Server.java:doRunLoop(997)) - Checking for old call responses. 2015-11-22 19:29:54,104 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:29:54,102 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #8: Service NMLivelinessMonitor in state NMLivelinessMonitor: STARTED 2015-11-22 19:29:54,102 INFO [IPC Server listener on 48629] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 48629 2015-11-22 19:29:54,101 DEBUG [IPC Server handler 32 on 48629] ipc.Server (Server.java:run(2290)) - IPC Server handler 32 on 48629: exiting 2015-11-22 19:29:54,101 DEBUG [IPC Server handler 31 on 48629] ipc.Server (Server.java:run(2290)) - IPC Server handler 31 on 48629: exiting 2015-11-22 19:29:54,101 DEBUG [IPC Server handler 26 on 48629] ipc.Server (Server.java:run(2290)) - IPC Server handler 26 on 48629: exiting 2015-11-22 19:29:54,101 DEBUG [IPC Server handler 29 on 48629] ipc.Server (Server.java:run(2290)) - IPC Server handler 29 on 48629: exiting 2015-11-22 19:29:54,101 DEBUG [IPC Server handler 24 on 48629] ipc.Server (Server.java:run(2290)) - IPC Server handler 24 on 48629: exiting 2015-11-22 19:29:54,101 DEBUG [IPC Server handler 23 on 48629] ipc.Server (Server.java:run(2290)) - IPC Server handler 23 on 48629: exiting 2015-11-22 19:29:54,101 DEBUG [IPC Server handler 17 on 48629] ipc.Server (Server.java:run(2290)) - IPC Server handler 17 on 48629: exiting 2015-11-22 19:29:54,101 DEBUG [IPC Server handler 20 on 48629] ipc.Server (Server.java:run(2290)) - IPC Server handler 20 on 48629: exiting 2015-11-22 19:29:54,101 DEBUG [IPC Server handler 19 on 48629] ipc.Server (Server.java:run(2290)) - IPC Server handler 19 on 48629: exiting 2015-11-22 19:29:54,101 DEBUG [IPC Server handler 16 on 48629] ipc.Server (Server.java:run(2290)) - IPC Server handler 16 on 48629: exiting 2015-11-22 19:29:54,101 DEBUG [IPC Server handler 21 on 48629] ipc.Server (Server.java:run(2290)) - IPC Server handler 21 on 48629: exiting 2015-11-22 19:29:54,101 DEBUG [IPC Server handler 14 on 48629] ipc.Server (Server.java:run(2290)) - IPC Server handler 14 on 48629: exiting 2015-11-22 19:29:54,104 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: NMLivelinessMonitor entered state STOPPED 2015-11-22 19:29:54,104 DEBUG [IPC Server handler 49 on 48629] ipc.Server (Server.java:run(2290)) - IPC Server handler 49 on 48629: exiting 2015-11-22 19:29:54,107 INFO [Ping Checker] util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(135)) - NMLivelinessMonitor thread interrupted 2015-11-22 19:29:54,103 DEBUG [IPC Server handler 48 on 48629] ipc.Server (Server.java:run(2290)) - IPC Server handler 48 on 48629: exiting 2015-11-22 19:29:54,103 DEBUG [IPC Server handler 47 on 48629] ipc.Server (Server.java:run(2290)) - IPC Server handler 47 on 48629: exiting 2015-11-22 19:29:54,103 DEBUG [IPC Server handler 42 on 48629] ipc.Server (Server.java:run(2290)) - IPC Server handler 42 on 48629: exiting 2015-11-22 19:29:54,103 DEBUG [IPC Server handler 34 on 48629] ipc.Server (Server.java:run(2290)) - IPC Server handler 34 on 48629: exiting 2015-11-22 19:29:54,103 DEBUG [IPC Server handler 39 on 48629] ipc.Server (Server.java:run(2290)) - IPC Server handler 39 on 48629: exiting 2015-11-22 19:29:54,103 DEBUG [IPC Server handler 35 on 48629] ipc.Server (Server.java:run(2290)) - IPC Server handler 35 on 48629: exiting 2015-11-22 19:29:54,103 DEBUG [IPC Server handler 37 on 48629] ipc.Server (Server.java:run(2290)) - IPC Server handler 37 on 48629: exiting 2015-11-22 19:29:54,107 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #7: Service org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher in state org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher: STARTED 2015-11-22 19:29:54,108 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher entered state STOPPED 2015-11-22 19:29:54,108 ERROR [ResourceManager Event Processor] resourcemanager.ResourceManager (ResourceManager.java:run(691)) - Returning, interrupted : java.lang.InterruptedException 2015-11-22 19:29:54,108 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #6: Service org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler in state org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler: STARTED 2015-11-22 19:29:54,108 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler entered state STOPPED 2015-11-22 19:29:54,108 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #5: Service org.apache.hadoop.yarn.server.resourcemanager.NodesListManager in state org.apache.hadoop.yarn.server.resourcemanager.NodesListManager: STARTED 2015-11-22 19:29:54,108 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.NodesListManager entered state STOPPED 2015-11-22 19:29:54,109 DEBUG [main] service.CompositeService (CompositeService.java:serviceStop(129)) - org.apache.hadoop.yarn.server.resourcemanager.NodesListManager: stopping services, size=0 2015-11-22 19:29:54,109 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #4: Service org.apache.hadoop.yarn.nodelabels.CommonNodeLabelsManager in state org.apache.hadoop.yarn.nodelabels.CommonNodeLabelsManager: STARTED 2015-11-22 19:29:54,109 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.nodelabels.CommonNodeLabelsManager entered state STOPPED 2015-11-22 19:29:54,109 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: Dispatcher entered state STOPPED 2015-11-22 19:29:54,109 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:serviceStop(141)) - AsyncDispatcher is draining to stop, igonring any new events. 2015-11-22 19:29:54,109 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #3: Service AMLivelinessMonitor in state AMLivelinessMonitor: STARTED 2015-11-22 19:29:54,109 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: AMLivelinessMonitor entered state STOPPED 2015-11-22 19:29:54,109 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #2: Service AMLivelinessMonitor in state AMLivelinessMonitor: STARTED 2015-11-22 19:29:54,109 INFO [Ping Checker] util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(135)) - AMLivelinessMonitor thread interrupted 2015-11-22 19:29:54,110 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: AMLivelinessMonitor entered state STOPPED 2015-11-22 19:29:54,110 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #1: Service org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.ContainerAllocationExpirer in state org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.ContainerAllocationExpirer: STARTED 2015-11-22 19:29:54,110 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.ContainerAllocationExpirer entered state STOPPED 2015-11-22 19:29:54,110 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #0: Service org.apache.hadoop.yarn.server.resourcemanager.RMSecretManagerService in state org.apache.hadoop.yarn.server.resourcemanager.RMSecretManagerService: STARTED 2015-11-22 19:29:54,110 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.RMSecretManagerService entered state STOPPED 2015-11-22 19:29:54,110 DEBUG [main] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:stopThreads(619)) - Stopping expired delegation token remover thread 2015-11-22 19:29:54,110 ERROR [Thread[Thread-3436,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(670)) - ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted 2015-11-22 19:29:54,110 INFO [Ping Checker] util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(135)) - AMLivelinessMonitor thread interrupted 2015-11-22 19:29:54,110 INFO [Ping Checker] util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(135)) - org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.ContainerAllocationExpirer thread interrupted 2015-11-22 19:29:54,111 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:shutdown(588)) - refCount=1 2015-11-22 19:29:54,111 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stop(211)) - Stopping ApplicationHistoryServer metrics system... 2015-11-22 19:29:54,111 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stopSources(458)) - Stopping metrics source JvmMetrics: class=class org.apache.hadoop.metrics2.source.JvmMetrics 2015-11-22 19:29:54,112 DEBUG [main] util.MBeans (MBeans.java:unregister(103)) - Unregistering Hadoop:service=ResourceManager,name=JvmMetrics 2015-11-22 19:29:54,112 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stopSources(458)) - Stopping metrics source NodeManagerMetrics: class=class org.apache.hadoop.metrics2.lib.MetricsSourceBuilder$1 2015-11-22 19:29:54,112 DEBUG [main] util.MBeans (MBeans.java:unregister(103)) - Unregistering Hadoop:service=ResourceManager,name=NodeManagerMetrics 2015-11-22 19:29:54,112 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stopSources(458)) - Stopping metrics source ClusterMetrics: class=class org.apache.hadoop.metrics2.lib.MetricsSourceBuilder$1 2015-11-22 19:29:54,112 DEBUG [main] util.MBeans (MBeans.java:unregister(103)) - Unregistering Hadoop:service=ResourceManager,name=ClusterMetrics 2015-11-22 19:29:54,112 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stopSources(458)) - Stopping metrics source QueueMetrics,q0=root: class=class org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CSQueueMetrics 2015-11-22 19:29:54,112 DEBUG [main] util.MBeans (MBeans.java:unregister(103)) - Unregistering Hadoop:service=ResourceManager,name=QueueMetrics,q0=root 2015-11-22 19:29:54,112 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stopSources(458)) - Stopping metrics source QueueMetrics,q0=root,q1=default: class=class org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CSQueueMetrics 2015-11-22 19:29:54,112 DEBUG [main] util.MBeans (MBeans.java:unregister(103)) - Unregistering Hadoop:service=ResourceManager,name=QueueMetrics,q0=root,q1=default 2015-11-22 19:29:54,113 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stopSources(458)) - Stopping metrics source TimelineDataManagerMetrics: class=class org.apache.hadoop.metrics2.lib.MetricsSourceBuilder$1 2015-11-22 19:29:54,113 DEBUG [main] util.MBeans (MBeans.java:unregister(103)) - Unregistering Hadoop:service=ApplicationHistoryServer,name=TimelineDataManagerMetrics 2015-11-22 19:29:54,113 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stopSources(458)) - Stopping metrics source RpcActivityForPort48629: class=class org.apache.hadoop.metrics2.lib.MetricsSourceBuilder$1 2015-11-22 19:29:54,113 DEBUG [main] util.MBeans (MBeans.java:unregister(103)) - Unregistering Hadoop:service=ApplicationHistoryServer,name=RpcActivityForPort48629 2015-11-22 19:29:54,113 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stopSources(458)) - Stopping metrics source RpcDetailedActivityForPort48629: class=class org.apache.hadoop.metrics2.lib.MetricsSourceBuilder$1 2015-11-22 19:29:54,113 DEBUG [main] util.MBeans (MBeans.java:unregister(103)) - Unregistering Hadoop:service=ApplicationHistoryServer,name=RpcDetailedActivityForPort48629 2015-11-22 19:29:54,113 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stopSources(458)) - Stopping metrics source RpcActivityForPort46284: class=class org.apache.hadoop.metrics2.lib.MetricsSourceBuilder$1 2015-11-22 19:29:54,114 DEBUG [main] util.MBeans (MBeans.java:unregister(103)) - Unregistering Hadoop:service=ApplicationHistoryServer,name=RpcActivityForPort46284 2015-11-22 19:29:54,114 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stopSources(458)) - Stopping metrics source RpcDetailedActivityForPort46284: class=class org.apache.hadoop.metrics2.lib.MetricsSourceBuilder$1 2015-11-22 19:29:54,114 DEBUG [main] util.MBeans (MBeans.java:unregister(103)) - Unregistering Hadoop:service=ApplicationHistoryServer,name=RpcDetailedActivityForPort46284 2015-11-22 19:29:54,114 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stopSources(458)) - Stopping metrics source RpcActivityForPort52028: class=class org.apache.hadoop.metrics2.lib.MetricsSourceBuilder$1 2015-11-22 19:29:54,114 DEBUG [main] util.MBeans (MBeans.java:unregister(103)) - Unregistering Hadoop:service=ApplicationHistoryServer,name=RpcActivityForPort52028 2015-11-22 19:29:54,114 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stopSources(458)) - Stopping metrics source RpcDetailedActivityForPort52028: class=class org.apache.hadoop.metrics2.lib.MetricsSourceBuilder$1 2015-11-22 19:29:54,115 DEBUG [main] util.MBeans (MBeans.java:unregister(103)) - Unregistering Hadoop:service=ApplicationHistoryServer,name=RpcDetailedActivityForPort52028 2015-11-22 19:29:54,115 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stopSources(458)) - Stopping metrics source RpcActivityForPort53669: class=class org.apache.hadoop.metrics2.lib.MetricsSourceBuilder$1 2015-11-22 19:29:54,115 DEBUG [main] util.MBeans (MBeans.java:unregister(103)) - Unregistering Hadoop:service=ApplicationHistoryServer,name=RpcActivityForPort53669 2015-11-22 19:29:54,115 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stopSources(458)) - Stopping metrics source RpcDetailedActivityForPort53669: class=class org.apache.hadoop.metrics2.lib.MetricsSourceBuilder$1 2015-11-22 19:29:54,115 DEBUG [main] util.MBeans (MBeans.java:unregister(103)) - Unregistering Hadoop:service=ApplicationHistoryServer,name=RpcDetailedActivityForPort53669 2015-11-22 19:29:54,115 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stopSources(458)) - Stopping metrics source RpcActivityForPort50663: class=class org.apache.hadoop.metrics2.lib.MetricsSourceBuilder$1 2015-11-22 19:29:54,116 DEBUG [main] util.MBeans (MBeans.java:unregister(103)) - Unregistering Hadoop:service=ApplicationHistoryServer,name=RpcActivityForPort50663 2015-11-22 19:29:54,116 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stopSources(458)) - Stopping metrics source RpcDetailedActivityForPort50663: class=class org.apache.hadoop.metrics2.lib.MetricsSourceBuilder$1 2015-11-22 19:29:54,116 DEBUG [main] util.MBeans (MBeans.java:unregister(103)) - Unregistering Hadoop:service=ApplicationHistoryServer,name=RpcDetailedActivityForPort50663 2015-11-22 19:29:54,116 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stopSources(458)) - Stopping metrics source RpcActivityForPort47651: class=class org.apache.hadoop.metrics2.lib.MetricsSourceBuilder$1 2015-11-22 19:29:54,116 DEBUG [main] util.MBeans (MBeans.java:unregister(103)) - Unregistering Hadoop:service=ApplicationHistoryServer,name=RpcActivityForPort47651 2015-11-22 19:29:54,116 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stopSources(458)) - Stopping metrics source RpcDetailedActivityForPort47651: class=class org.apache.hadoop.metrics2.lib.MetricsSourceBuilder$1 2015-11-22 19:29:54,117 DEBUG [main] util.MBeans (MBeans.java:unregister(103)) - Unregistering Hadoop:service=ApplicationHistoryServer,name=RpcDetailedActivityForPort47651 2015-11-22 19:29:54,117 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stopSources(458)) - Stopping metrics source RpcActivityForPort32936: class=class org.apache.hadoop.metrics2.lib.MetricsSourceBuilder$1 2015-11-22 19:29:54,117 DEBUG [main] util.MBeans (MBeans.java:unregister(103)) - Unregistering Hadoop:service=ApplicationHistoryServer,name=RpcActivityForPort32936 2015-11-22 19:29:54,117 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stopSources(458)) - Stopping metrics source RpcDetailedActivityForPort32936: class=class org.apache.hadoop.metrics2.lib.MetricsSourceBuilder$1 2015-11-22 19:29:54,117 DEBUG [main] util.MBeans (MBeans.java:unregister(103)) - Unregistering Hadoop:service=ApplicationHistoryServer,name=RpcDetailedActivityForPort32936 2015-11-22 19:29:54,117 DEBUG [main] util.MBeans (MBeans.java:unregister(103)) - Unregistering Hadoop:service=ResourceManager,name=MetricsSystem,sub=Stats 2015-11-22 19:29:54,118 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stop(217)) - ApplicationHistoryServer metrics system stopped. 2015-11-22 19:29:54,118 DEBUG [main] util.MBeans (MBeans.java:unregister(103)) - Unregistering Hadoop:service=ResourceManager,name=MetricsSystem,sub=Control 2015-11-22 19:29:54,118 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:shutdown(608)) - ApplicationHistoryServer metrics system shutdown complete. 2015-11-22 19:29:54,118 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore entered state STOPPED 2015-11-22 19:29:54,118 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: Dispatcher entered state STOPPED 2015-11-22 19:29:54,118 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:serviceStop(141)) - AsyncDispatcher is draining to stop, igonring any new events. 2015-11-22 19:29:54,118 INFO [main] resourcemanager.ResourceManager (ResourceManager.java:transitionToStandby(1058)) - Transitioned to standby state 2015-11-22 19:29:54,119 DEBUG [main] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.fs.FileContext.getAbstractFileSystem(FileContext.java:339) 2015-11-22 19:29:54,137 INFO [main] distributedshell.TestDistributedShell (TestDistributedShell.java:setupInternal(77)) - Starting up YARN cluster 2015-11-22 19:29:54,146 INFO [main] net.ServerSocketUtil (ServerSocketUtil.java:getPort(52)) - Using port 10021 2015-11-22 19:29:54,147 DEBUG [main] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.fs.FileContext.getAbstractFileSystem(FileContext.java:339) 2015-11-22 19:29:54,157 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: TestDistributedShell entered state INITED 2015-11-22 19:29:54,158 DEBUG [main] service.CompositeService (CompositeService.java:addService(73)) - Adding service org.apache.hadoop.yarn.server.MiniYARNCluster$ResourceManagerWrapper_0 2015-11-22 19:29:54,158 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:register(232)) - JvmMetrics, JVM related metrics etc. 2015-11-22 19:29:54,158 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterInt org.apache.hadoop.yarn.server.nodemanager.metrics.NodeManagerMetrics.containersLaunched with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[]) 2015-11-22 19:29:54,158 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterInt org.apache.hadoop.yarn.server.nodemanager.metrics.NodeManagerMetrics.containersCompleted with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[]) 2015-11-22 19:29:54,158 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterInt org.apache.hadoop.yarn.server.nodemanager.metrics.NodeManagerMetrics.containersFailed with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[]) 2015-11-22 19:29:54,158 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterInt org.apache.hadoop.yarn.server.nodemanager.metrics.NodeManagerMetrics.containersKilled with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[]) 2015-11-22 19:29:54,159 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.nodemanager.metrics.NodeManagerMetrics.containersIniting with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[# of initializing containers]) 2015-11-22 19:29:54,159 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.nodemanager.metrics.NodeManagerMetrics.containersRunning with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[]) 2015-11-22 19:29:54,159 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.nodemanager.metrics.NodeManagerMetrics.allocatedGB with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Current allocated memory in GB]) 2015-11-22 19:29:54,159 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.nodemanager.metrics.NodeManagerMetrics.allocatedContainers with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Current # of allocated containers]) 2015-11-22 19:29:54,159 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.nodemanager.metrics.NodeManagerMetrics.availableGB with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[]) 2015-11-22 19:29:54,159 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.nodemanager.metrics.NodeManagerMetrics.allocatedVCores with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Current allocated Virtual Cores]) 2015-11-22 19:29:54,159 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.nodemanager.metrics.NodeManagerMetrics.availableVCores with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[]) 2015-11-22 19:29:54,160 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.yarn.server.nodemanager.metrics.NodeManagerMetrics.containerLaunchDuration with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Container launch duration]) 2015-11-22 19:29:54,160 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.nodemanager.metrics.NodeManagerMetrics.badLocalDirs with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[# of bad local dirs]) 2015-11-22 19:29:54,160 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.nodemanager.metrics.NodeManagerMetrics.badLogDirs with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[# of bad log dirs]) 2015-11-22 19:29:54,160 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.nodemanager.metrics.NodeManagerMetrics.goodLocalDirsDiskUtilizationPerc with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Disk utilization % on good local dirs]) 2015-11-22 19:29:54,160 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.nodemanager.metrics.NodeManagerMetrics.goodLogDirsDiskUtilizationPerc with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Disk utilization % on good log dirs]) 2015-11-22 19:29:54,160 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:register(232)) - NodeManagerMetrics, Metrics for node manager 2015-11-22 19:29:54,160 DEBUG [main] service.CompositeService (CompositeService.java:addService(73)) - Adding service org.apache.hadoop.yarn.server.MiniYARNCluster$NodeManagerWrapper_0 2015-11-22 19:29:54,160 DEBUG [main] service.CompositeService (CompositeService.java:addService(73)) - Adding service org.apache.hadoop.yarn.server.MiniYARNCluster$ApplicationHistoryServerWrapper 2015-11-22 19:29:54,161 DEBUG [main] service.CompositeService (CompositeService.java:serviceInit(104)) - TestDistributedShell: initing services, size=3 2015-11-22 19:29:54,161 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.MiniYARNCluster$ResourceManagerWrapper_0 entered state INITED 2015-11-22 19:29:54,161 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: ResourceManager entered state INITED 2015-11-22 19:29:54,161 INFO [main] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2363)) - found resource core-site.xml at file:/home/ubuntu/hadoop-dev/hadoop-common-project/hadoop-common/target/test-classes/core-site.xml 2015-11-22 19:29:54,171 WARN [main] conf.Configuration (Configuration.java:loadProperty(2687)) - core-site.xml:an attempt to override final parameter: hadoop.tmp.dir; Ignoring. 2015-11-22 19:29:54,171 DEBUG [main] util.PerformanceAdvisory (JniBasedUnixGroupsMappingWithFallback.java:(41)) - Falling back to shell based 2015-11-22 19:29:54,171 DEBUG [main] security.JniBasedUnixGroupsMappingWithFallback (JniBasedUnixGroupsMappingWithFallback.java:(45)) - Group mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping 2015-11-22 19:29:54,172 DEBUG [main] security.Groups (Groups.java:(112)) - Group mapping impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback; cacheTimeout=300000; warningDeltaMs=5000 2015-11-22 19:29:54,172 INFO [main] security.Groups (Groups.java:refresh(256)) - clearing userToGroupsMap cache 2015-11-22 19:29:54,172 INFO [main] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2363)) - found resource yarn-site.xml at file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/test-classes/yarn-site.xml 2015-11-22 19:29:54,187 WARN [main] conf.Configuration (Configuration.java:loadProperty(2687)) - yarn-site.xml:an attempt to override final parameter: hadoop.tmp.dir; Ignoring. 2015-11-22 19:29:54,188 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.RMFatalEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMFatalEventDispatcher 2015-11-22 19:29:54,188 DEBUG [main] service.CompositeService (CompositeService.java:addService(73)) - Adding service Dispatcher 2015-11-22 19:29:54,188 DEBUG [main] service.CompositeService (CompositeService.java:addService(73)) - Adding service org.apache.hadoop.yarn.server.resourcemanager.AdminService 2015-11-22 19:29:54,188 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: RMActiveServices entered state INITED 2015-11-22 19:29:54,189 INFO [main] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:(75)) - NMTokenKeyRollingInterval: 86400000ms and NMTokenKeyActivationDelay: 900000ms 2015-11-22 19:29:54,189 INFO [main] security.RMContainerTokenSecretManager (RMContainerTokenSecretManager.java:(78)) - ContainerTokenKeyRollingInterval: 86400000ms and ContainerTokenKeyActivationDelay: 900000ms 2015-11-22 19:29:54,189 INFO [main] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:(94)) - AMRMTokenKeyRollingInterval: 86400000ms and AMRMTokenKeyActivationDelay: 900000 ms 2015-11-22 19:29:54,189 DEBUG [main] service.CompositeService (CompositeService.java:addService(73)) - Adding service org.apache.hadoop.yarn.server.resourcemanager.RMSecretManagerService 2015-11-22 19:29:54,189 DEBUG [main] service.CompositeService (CompositeService.java:addService(73)) - Adding service org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.ContainerAllocationExpirer 2015-11-22 19:29:54,190 DEBUG [main] service.CompositeService (CompositeService.java:addService(73)) - Adding service AMLivelinessMonitor 2015-11-22 19:29:54,190 DEBUG [main] service.CompositeService (CompositeService.java:addService(73)) - Adding service AMLivelinessMonitor 2015-11-22 19:29:54,190 DEBUG [main] service.CompositeService (CompositeService.java:addService(73)) - Adding service org.apache.hadoop.yarn.nodelabels.CommonNodeLabelsManager 2015-11-22 19:29:54,190 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore entered state INITED 2015-11-22 19:29:54,190 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: Dispatcher entered state INITED 2015-11-22 19:29:54,190 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStoreEventType for class org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$ForwardingEventHandler 2015-11-22 19:29:54,190 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.NodesListManagerEventType for class org.apache.hadoop.yarn.server.resourcemanager.NodesListManager 2015-11-22 19:29:54,190 DEBUG [main] service.CompositeService (CompositeService.java:addService(73)) - Adding service org.apache.hadoop.yarn.server.resourcemanager.NodesListManager 2015-11-22 19:29:54,190 INFO [main] resourcemanager.ResourceManager (ResourceManager.java:createScheduler(295)) - Using Scheduler: org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler 2015-11-22 19:29:54,191 DEBUG [main] service.CompositeService (CompositeService.java:addService(73)) - Adding service org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler 2015-11-22 19:29:54,191 DEBUG [main] service.CompositeService (CompositeService.java:addService(73)) - Adding service org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher 2015-11-22 19:29:54,191 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.SchedulerEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher 2015-11-22 19:29:54,191 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationEventDispatcher 2015-11-22 19:29:54,191 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationAttemptEventDispatcher 2015-11-22 19:29:54,191 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$NodeEventDispatcher 2015-11-22 19:29:54,191 DEBUG [main] service.CompositeService (CompositeService.java:addService(73)) - Adding service NMLivelinessMonitor 2015-11-22 19:29:54,191 DEBUG [main] service.CompositeService (CompositeService.java:addService(73)) - Adding service org.apache.hadoop.yarn.server.resourcemanager.ResourceTrackerService 2015-11-22 19:29:54,192 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:initMode(622)) - from system property: null 2015-11-22 19:29:54,192 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:initMode(623)) - from environment variable: null 2015-11-22 19:29:54,192 DEBUG [main] configuration.ConfigurationUtils (ConfigurationUtils.java:locate(447)) - ConfigurationUtils.locate(): base is null, name is hadoop-metrics2-resourcemanager.properties 2015-11-22 19:29:54,192 DEBUG [main] configuration.ConfigurationUtils (ConfigurationUtils.java:locate(447)) - ConfigurationUtils.locate(): base is null, name is hadoop-metrics2.properties 2015-11-22 19:29:54,193 DEBUG [main] configuration.ConfigurationUtils (ConfigurationUtils.java:locateFromClasspath(580)) - Loading configuration from the context classpath (hadoop-metrics2.properties) 2015-11-22 19:29:54,193 INFO [main] impl.MetricsConfig (MetricsConfig.java:loadFirst(115)) - loaded properties from hadoop-metrics2.properties 2015-11-22 19:29:54,193 DEBUG [main] impl.MetricsConfig (MetricsConfig.java:loadFirst(116)) - *.sink.file.class = org.apache.hadoop.metrics2.sink.FileSink *.period = 10 2015-11-22 19:29:54,194 DEBUG [main] impl.MetricsConfig (MetricsConfig.java:loadFirst(118)) - 2015-11-22 19:29:54,195 DEBUG [main] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'PropertiesConfiguration' for key: period 2015-11-22 19:29:54,195 DEBUG [main] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'PropertiesConfiguration' for key: periodMillis 2015-11-22 19:29:54,196 DEBUG [main] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'PropertiesConfiguration' for key: source.source.start_mbeans 2015-11-22 19:29:54,196 DEBUG [main] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'MetricsConfig' for key: source.start_mbeans 2015-11-22 19:29:54,196 DEBUG [main] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'PropertiesConfiguration' for key: *.source.start_mbeans 2015-11-22 19:29:54,196 DEBUG [main] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateAttrCache(254)) - Updating attr cache... 2015-11-22 19:29:54,196 DEBUG [main] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateAttrCache(268)) - Done. # tags & metrics=10 2015-11-22 19:29:54,197 DEBUG [main] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateInfoCache(248)) - Updating info cache... 2015-11-22 19:29:54,197 DEBUG [main] impl.MetricsSystemImpl (MBeanInfoBuilder.java:get(109)) - [javax.management.MBeanAttributeInfo[description=Metrics context, name=tag.Context, type=java.lang.String, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of active metrics sources, name=NumActiveSources, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of all registered metrics sources, name=NumAllSources, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of active metrics sinks, name=NumActiveSinks, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of all registered metrics sinks, name=NumAllSinks, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of ops for snapshot stats, name=SnapshotNumOps, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Average time for snapshot stats, name=SnapshotAvgTime, type=java.lang.Double, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of ops for publishing stats, name=PublishNumOps, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Average time for publishing stats, name=PublishAvgTime, type=java.lang.Double, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Dropped updates by all sinks, name=DroppedPubAll, type=java.lang.Long, read-only, descriptor={}]] 2015-11-22 19:29:54,197 DEBUG [main] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateInfoCache(250)) - Done 2015-11-22 19:29:54,197 DEBUG [main] util.MBeans (MBeans.java:register(67)) - Registered Hadoop:service=ResourceManager,name=MetricsSystem,sub=Stats 2015-11-22 19:29:54,198 DEBUG [main] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:startMBeans(227)) - MBean for source MetricsSystem,sub=Stats registered. 2015-11-22 19:29:54,204 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:startTimer(374)) - Scheduled snapshot period at 10 second(s). 2015-11-22 19:29:54,204 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:start(192)) - ResourceManager metrics system started 2015-11-22 19:29:54,204 DEBUG [main] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'PropertiesConfiguration' for key: source.source.start_mbeans 2015-11-22 19:29:54,204 DEBUG [main] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'MetricsConfig' for key: source.start_mbeans 2015-11-22 19:29:54,204 DEBUG [main] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'PropertiesConfiguration' for key: *.source.start_mbeans 2015-11-22 19:29:54,205 DEBUG [main] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateAttrCache(254)) - Updating attr cache... 2015-11-22 19:29:54,205 DEBUG [main] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateAttrCache(268)) - Done. # tags & metrics=27 2015-11-22 19:29:54,205 DEBUG [main] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateInfoCache(248)) - Updating info cache... 2015-11-22 19:29:54,205 DEBUG [main] impl.MetricsSystemImpl (MBeanInfoBuilder.java:get(109)) - [javax.management.MBeanAttributeInfo[description=Metrics context, name=tag.Context, type=java.lang.String, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Process name, name=tag.ProcessName, type=java.lang.String, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Session ID, name=tag.SessionId, type=java.lang.String, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Local hostname, name=tag.Hostname, type=java.lang.String, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Non-heap memory used in MB, name=MemNonHeapUsedM, type=java.lang.Float, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Non-heap memory committed in MB, name=MemNonHeapCommittedM, type=java.lang.Float, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Non-heap memory max in MB, name=MemNonHeapMaxM, type=java.lang.Float, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Heap memory used in MB, name=MemHeapUsedM, type=java.lang.Float, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Heap memory committed in MB, name=MemHeapCommittedM, type=java.lang.Float, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Heap memory max in MB, name=MemHeapMaxM, type=java.lang.Float, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Max memory size in MB, name=MemMaxM, type=java.lang.Float, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=GC Count for PS Scavenge, name=GcCountPS Scavenge, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=GC Time for PS Scavenge, name=GcTimeMillisPS Scavenge, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=GC Count for PS MarkSweep, name=GcCountPS MarkSweep, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=GC Time for PS MarkSweep, name=GcTimeMillisPS MarkSweep, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Total GC count, name=GcCount, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Total GC time in milliseconds, name=GcTimeMillis, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of new threads, name=ThreadsNew, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of runnable threads, name=ThreadsRunnable, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of blocked threads, name=ThreadsBlocked, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of waiting threads, name=ThreadsWaiting, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of timed waiting threads, name=ThreadsTimedWaiting, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of terminated threads, name=ThreadsTerminated, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Total number of fatal log events, name=LogFatal, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Total number of error log events, name=LogError, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Total number of warning log events, name=LogWarn, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Total number of info log events, name=LogInfo, type=java.lang.Long, read-only, descriptor={}]] 2015-11-22 19:29:54,206 DEBUG [main] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateInfoCache(250)) - Done 2015-11-22 19:29:54,206 DEBUG [main] util.MBeans (MBeans.java:register(67)) - Registered Hadoop:service=ResourceManager,name=JvmMetrics 2015-11-22 19:29:54,206 DEBUG [main] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:startMBeans(227)) - MBean for source JvmMetrics registered. 2015-11-22 19:29:54,206 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:registerSource(269)) - Registered source JvmMetrics 2015-11-22 19:29:54,206 DEBUG [main] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'PropertiesConfiguration' for key: source.source.start_mbeans 2015-11-22 19:29:54,207 DEBUG [main] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'MetricsConfig' for key: source.start_mbeans 2015-11-22 19:29:54,207 DEBUG [main] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'PropertiesConfiguration' for key: *.source.start_mbeans 2015-11-22 19:29:54,207 DEBUG [main] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateAttrCache(254)) - Updating attr cache... 2015-11-22 19:29:54,207 DEBUG [main] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateAttrCache(268)) - Done. # tags & metrics=19 2015-11-22 19:29:54,207 DEBUG [main] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateInfoCache(248)) - Updating info cache... 2015-11-22 19:29:54,208 DEBUG [main] impl.MetricsSystemImpl (MBeanInfoBuilder.java:get(109)) - [javax.management.MBeanAttributeInfo[description=Metrics context, name=tag.Context, type=java.lang.String, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Local hostname, name=tag.Hostname, type=java.lang.String, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=ContainersLaunched, name=ContainersLaunched, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=ContainersCompleted, name=ContainersCompleted, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=ContainersFailed, name=ContainersFailed, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=ContainersKilled, name=ContainersKilled, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=# of initializing containers, name=ContainersIniting, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=ContainersRunning, name=ContainersRunning, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Current allocated memory in GB, name=AllocatedGB, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Current # of allocated containers, name=AllocatedContainers, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=AvailableGB, name=AvailableGB, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Current allocated Virtual Cores, name=AllocatedVCores, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=AvailableVCores, name=AvailableVCores, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of ops for container launch duration, name=ContainerLaunchDurationNumOps, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Average time for container launch duration, name=ContainerLaunchDurationAvgTime, type=java.lang.Double, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=# of bad local dirs, name=BadLocalDirs, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=# of bad log dirs, name=BadLogDirs, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Disk utilization % on good local dirs, name=GoodLocalDirsDiskUtilizationPerc, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Disk utilization % on good log dirs, name=GoodLogDirsDiskUtilizationPerc, type=java.lang.Integer, read-only, descriptor={}]] 2015-11-22 19:29:54,208 DEBUG [main] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateInfoCache(250)) - Done 2015-11-22 19:29:54,208 DEBUG [main] util.MBeans (MBeans.java:register(67)) - Registered Hadoop:service=ResourceManager,name=NodeManagerMetrics 2015-11-22 19:29:54,208 DEBUG [main] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:startMBeans(227)) - MBean for source NodeManagerMetrics registered. 2015-11-22 19:29:54,208 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:registerSource(269)) - Registered source NodeManagerMetrics 2015-11-22 19:29:54,209 DEBUG [main] util.MBeans (MBeans.java:register(67)) - Registered Hadoop:service=ResourceManager,name=MetricsSystem,sub=Control 2015-11-22 19:29:54,209 DEBUG [main] service.CompositeService (CompositeService.java:addService(73)) - Adding service org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService 2015-11-22 19:29:54,209 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.RMAppManagerEventType for class org.apache.hadoop.yarn.server.resourcemanager.RMAppManager 2015-11-22 19:29:54,209 DEBUG [main] service.CompositeService (CompositeService.java:addService(73)) - Adding service org.apache.hadoop.yarn.server.resourcemanager.ClientRMService 2015-11-22 19:29:54,210 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncherEventType for class org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher 2015-11-22 19:29:54,210 DEBUG [main] service.CompositeService (CompositeService.java:addService(73)) - Adding service org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher 2015-11-22 19:29:54,210 WARN [main] util.MBeans (MBeans.java:register(73)) - Failed to register MBean "Hadoop:service=ResourceManager,name=RMNMInfo": Instance already exists. 2015-11-22 19:29:54,210 INFO [main] resourcemanager.RMNMInfo (RMNMInfo.java:(63)) - Registered RMNMInfo MBean 2015-11-22 19:29:54,210 DEBUG [main] service.CompositeService (CompositeService.java:serviceInit(104)) - RMActiveServices: initing services, size=13 2015-11-22 19:29:54,210 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.RMSecretManagerService entered state INITED 2015-11-22 19:29:54,210 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.ContainerAllocationExpirer entered state INITED 2015-11-22 19:29:54,211 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: AMLivelinessMonitor entered state INITED 2015-11-22 19:29:54,211 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: AMLivelinessMonitor entered state INITED 2015-11-22 19:29:54,211 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.nodelabels.CommonNodeLabelsManager entered state INITED 2015-11-22 19:29:54,211 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.NodesListManager entered state INITED 2015-11-22 19:29:54,211 INFO [main] util.HostsFileReader (HostsFileReader.java:refresh(132)) - Refreshing hosts (include/exclude) list 2015-11-22 19:29:54,211 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.resourcemanager.ClusterMetrics.numActiveNMs with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[# of active NMs]) 2015-11-22 19:29:54,212 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.resourcemanager.ClusterMetrics.numDecommissionedNMs with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[# of decommissioned NMs]) 2015-11-22 19:29:54,212 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.resourcemanager.ClusterMetrics.numLostNMs with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[# of lost NMs]) 2015-11-22 19:29:54,212 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.resourcemanager.ClusterMetrics.numUnhealthyNMs with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[# of unhealthy NMs]) 2015-11-22 19:29:54,212 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.resourcemanager.ClusterMetrics.numRebootedNMs with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[# of Rebooted NMs]) 2015-11-22 19:29:54,212 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.resourcemanager.ClusterMetrics.numShutdownNMs with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[# of Shutdown NMs]) 2015-11-22 19:29:54,213 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.yarn.server.resourcemanager.ClusterMetrics.aMLaunchDelay with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[AM container launch delay]) 2015-11-22 19:29:54,213 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.yarn.server.resourcemanager.ClusterMetrics.aMRegisterDelay with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[AM register delay]) 2015-11-22 19:29:54,213 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:register(232)) - ClusterMetrics, Metrics for the Yarn Cluster 2015-11-22 19:29:54,213 DEBUG [main] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'PropertiesConfiguration' for key: source.source.start_mbeans 2015-11-22 19:29:54,213 DEBUG [main] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'MetricsConfig' for key: source.start_mbeans 2015-11-22 19:29:54,213 DEBUG [main] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'PropertiesConfiguration' for key: *.source.start_mbeans 2015-11-22 19:29:54,213 DEBUG [main] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateAttrCache(254)) - Updating attr cache... 2015-11-22 19:29:54,213 DEBUG [main] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateAttrCache(268)) - Done. # tags & metrics=13 2015-11-22 19:29:54,214 DEBUG [main] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateInfoCache(248)) - Updating info cache... 2015-11-22 19:29:54,214 DEBUG [main] impl.MetricsSystemImpl (MBeanInfoBuilder.java:get(109)) - [javax.management.MBeanAttributeInfo[description=Metrics for the Yarn Cluster, name=tag.ClusterMetrics, type=java.lang.String, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Metrics context, name=tag.Context, type=java.lang.String, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Local hostname, name=tag.Hostname, type=java.lang.String, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=# of active NMs, name=NumActiveNMs, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=# of decommissioned NMs, name=NumDecommissionedNMs, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=# of lost NMs, name=NumLostNMs, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=# of unhealthy NMs, name=NumUnhealthyNMs, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=# of Rebooted NMs, name=NumRebootedNMs, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=# of Shutdown NMs, name=NumShutdownNMs, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of ops for aM container launch delay, name=AMLaunchDelayNumOps, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Average time for aM container launch delay, name=AMLaunchDelayAvgTime, type=java.lang.Double, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of ops for aM register delay, name=AMRegisterDelayNumOps, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Average time for aM register delay, name=AMRegisterDelayAvgTime, type=java.lang.Double, read-only, descriptor={}]] 2015-11-22 19:29:54,214 DEBUG [main] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateInfoCache(250)) - Done 2015-11-22 19:29:54,214 DEBUG [main] util.MBeans (MBeans.java:register(67)) - Registered Hadoop:service=ResourceManager,name=ClusterMetrics 2015-11-22 19:29:54,214 DEBUG [main] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:startMBeans(227)) - MBean for source ClusterMetrics registered. 2015-11-22 19:29:54,214 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:registerSource(269)) - Registered source ClusterMetrics 2015-11-22 19:29:54,214 DEBUG [main] resourcemanager.NodesListManager (NodesListManager.java:printConfiguredHosts(118)) - hostsReader: in= out= 2015-11-22 19:29:54,214 DEBUG [main] service.CompositeService (CompositeService.java:serviceInit(104)) - org.apache.hadoop.yarn.server.resourcemanager.NodesListManager: initing services, size=0 2015-11-22 19:29:54,215 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler entered state INITED 2015-11-22 19:29:54,215 INFO [main] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2363)) - found resource capacity-scheduler.xml at jar:file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/target/hadoop-yarn-server-tests-2.8.0-SNAPSHOT-tests.jar!/capacity-scheduler.xml 2015-11-22 19:29:54,226 DEBUG [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getQueues(584)) - CSConf - getQueues called for: queuePrefix=yarn.scheduler.capacity.root. 2015-11-22 19:29:54,226 DEBUG [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getQueues(594)) - CSConf - getQueues: queuePrefix=yarn.scheduler.capacity.root., queues=default 2015-11-22 19:29:54,226 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CSQueueMetrics.AMResourceLimitMB with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[AM memory limit in MB]) 2015-11-22 19:29:54,226 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CSQueueMetrics.AMResourceLimitVCores with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[AM CPU limit in virtual cores]) 2015-11-22 19:29:54,227 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CSQueueMetrics.usedAMResourceMB with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Used AM memory limit in MB]) 2015-11-22 19:29:54,227 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CSQueueMetrics.usedAMResourceVCores with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Used AM CPU limit in virtual cores]) 2015-11-22 19:29:54,227 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterInt org.apache.hadoop.yarn.server.resourcemanager.scheduler.QueueMetrics.appsSubmitted with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[# of apps submitted]) 2015-11-22 19:29:54,227 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.resourcemanager.scheduler.QueueMetrics.appsRunning with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[# of running apps]) 2015-11-22 19:29:54,227 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.resourcemanager.scheduler.QueueMetrics.appsPending with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[# of pending apps]) 2015-11-22 19:29:54,227 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterInt org.apache.hadoop.yarn.server.resourcemanager.scheduler.QueueMetrics.appsCompleted with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[# of apps completed]) 2015-11-22 19:29:54,227 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterInt org.apache.hadoop.yarn.server.resourcemanager.scheduler.QueueMetrics.appsKilled with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[# of apps killed]) 2015-11-22 19:29:54,227 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterInt org.apache.hadoop.yarn.server.resourcemanager.scheduler.QueueMetrics.appsFailed with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[# of apps failed]) 2015-11-22 19:29:54,228 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.resourcemanager.scheduler.QueueMetrics.allocatedMB with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Allocated memory in MB]) 2015-11-22 19:29:54,228 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.resourcemanager.scheduler.QueueMetrics.allocatedVCores with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Allocated CPU in virtual cores]) 2015-11-22 19:29:54,228 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.resourcemanager.scheduler.QueueMetrics.allocatedContainers with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[# of allocated containers]) 2015-11-22 19:29:54,228 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.yarn.server.resourcemanager.scheduler.QueueMetrics.aggregateContainersAllocated with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Aggregate # of allocated containers]) 2015-11-22 19:29:54,228 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.yarn.server.resourcemanager.scheduler.QueueMetrics.aggregateNodeLocalContainersAllocated with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Aggregate # of allocated node-local containers]) 2015-11-22 19:29:54,228 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.yarn.server.resourcemanager.scheduler.QueueMetrics.aggregateRackLocalContainersAllocated with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Aggregate # of allocated rack-local containers]) 2015-11-22 19:29:54,228 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.yarn.server.resourcemanager.scheduler.QueueMetrics.aggregateOffSwitchContainersAllocated with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Aggregate # of allocated off-switch containers]) 2015-11-22 19:29:54,228 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.yarn.server.resourcemanager.scheduler.QueueMetrics.aggregateContainersReleased with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Aggregate # of released containers]) 2015-11-22 19:29:54,229 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.resourcemanager.scheduler.QueueMetrics.availableMB with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Available memory in MB]) 2015-11-22 19:29:54,229 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.resourcemanager.scheduler.QueueMetrics.availableVCores with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Available CPU in virtual cores]) 2015-11-22 19:29:54,229 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.resourcemanager.scheduler.QueueMetrics.pendingMB with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Pending memory allocation in MB]) 2015-11-22 19:29:54,229 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.resourcemanager.scheduler.QueueMetrics.pendingVCores with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Pending CPU allocation in virtual cores]) 2015-11-22 19:29:54,229 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.resourcemanager.scheduler.QueueMetrics.pendingContainers with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[# of pending containers]) 2015-11-22 19:29:54,230 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.resourcemanager.scheduler.QueueMetrics.reservedMB with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[# of reserved memory in MB]) 2015-11-22 19:29:54,230 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.resourcemanager.scheduler.QueueMetrics.reservedVCores with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Reserved CPU in virtual cores]) 2015-11-22 19:29:54,230 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.resourcemanager.scheduler.QueueMetrics.reservedContainers with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[# of reserved containers]) 2015-11-22 19:29:54,230 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.resourcemanager.scheduler.QueueMetrics.activeUsers with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[# of active users]) 2015-11-22 19:29:54,230 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.resourcemanager.scheduler.QueueMetrics.activeApplications with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[# of active applications]) 2015-11-22 19:29:54,230 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.yarn.server.resourcemanager.scheduler.QueueMetrics.appAttemptFirstContainerAllocationDelay with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[App Attempt First Container Allocation Delay]) 2015-11-22 19:29:54,230 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:register(232)) - QueueMetrics,q0=root, Metrics for queue: root 2015-11-22 19:29:54,230 DEBUG [main] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'PropertiesConfiguration' for key: source.source.start_mbeans 2015-11-22 19:29:54,231 DEBUG [main] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'MetricsConfig' for key: source.start_mbeans 2015-11-22 19:29:54,231 DEBUG [main] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'PropertiesConfiguration' for key: *.source.start_mbeans 2015-11-22 19:29:54,231 DEBUG [main] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateAttrCache(254)) - Updating attr cache... 2015-11-22 19:29:54,231 DEBUG [main] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateAttrCache(268)) - Done. # tags & metrics=37 2015-11-22 19:29:54,231 DEBUG [main] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateInfoCache(248)) - Updating info cache... 2015-11-22 19:29:54,231 DEBUG [main] impl.MetricsSystemImpl (MBeanInfoBuilder.java:get(109)) - [javax.management.MBeanAttributeInfo[description=Metrics by queue, name=tag.Queue, type=java.lang.String, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Metrics context, name=tag.Context, type=java.lang.String, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Local hostname, name=tag.Hostname, type=java.lang.String, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=, name=running_0, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=, name=running_60, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=, name=running_300, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=, name=running_1440, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=AM memory limit in MB, name=AMResourceLimitMB, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=AM CPU limit in virtual cores, name=AMResourceLimitVCores, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Used AM memory limit in MB, name=UsedAMResourceMB, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Used AM CPU limit in virtual cores, name=UsedAMResourceVCores, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=# of apps submitted, name=AppsSubmitted, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=# of running apps, name=AppsRunning, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=# of pending apps, name=AppsPending, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=# of apps completed, name=AppsCompleted, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=# of apps killed, name=AppsKilled, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=# of apps failed, name=AppsFailed, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Allocated memory in MB, name=AllocatedMB, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Allocated CPU in virtual cores, name=AllocatedVCores, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=# of allocated containers, name=AllocatedContainers, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Aggregate # of allocated containers, name=AggregateContainersAllocated, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Aggregate # of allocated node-local containers, name=AggregateNodeLocalContainersAllocated, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Aggregate # of allocated rack-local containers, name=AggregateRackLocalContainersAllocated, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Aggregate # of allocated off-switch containers, name=AggregateOffSwitchContainersAllocated, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Aggregate # of released containers, name=AggregateContainersReleased, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Available memory in MB, name=AvailableMB, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Available CPU in virtual cores, name=AvailableVCores, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Pending memory allocation in MB, name=PendingMB, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Pending CPU allocation in virtual cores, name=PendingVCores, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=# of pending containers, name=PendingContainers, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=# of reserved memory in MB, name=ReservedMB, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Reserved CPU in virtual cores, name=ReservedVCores, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=# of reserved containers, name=ReservedContainers, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=# of active users, name=ActiveUsers, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=# of active applications, name=ActiveApplications, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of ops for app Attempt First Container Allocation Delay, name=AppAttemptFirstContainerAllocationDelayNumOps, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Average time for app Attempt First Container Allocation Delay, name=AppAttemptFirstContainerAllocationDelayAvgTime, type=java.lang.Double, read-only, descriptor={}]] 2015-11-22 19:29:54,231 DEBUG [main] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateInfoCache(250)) - Done 2015-11-22 19:29:54,232 DEBUG [main] util.MBeans (MBeans.java:register(67)) - Registered Hadoop:service=ResourceManager,name=QueueMetrics,q0=root 2015-11-22 19:29:54,232 DEBUG [main] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:startMBeans(227)) - MBean for source QueueMetrics,q0=root registered. 2015-11-22 19:29:54,232 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:registerSource(269)) - Registered source QueueMetrics,q0=root 2015-11-22 19:29:54,232 DEBUG [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getNonLabeledQueueCapacity(333)) - CSConf - getCapacity: queuePrefix=yarn.scheduler.capacity.root., capacity=100.0 2015-11-22 19:29:54,233 DEBUG [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getNonLabeledQueueCapacity(333)) - CSConf - getCapacity: queuePrefix=yarn.scheduler.capacity.root., capacity=100.0 2015-11-22 19:29:54,233 DEBUG [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(640)) - max alloc mb per queue for root is -1 2015-11-22 19:29:54,233 DEBUG [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(642)) - max alloc vcores per queue for root is -1 2015-11-22 19:29:54,233 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(647)) - max alloc mb per queue for root is undefined 2015-11-22 19:29:54,233 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(651)) - max alloc vcore per queue for root is undefined 2015-11-22 19:29:54,233 INFO [main] capacity.ParentQueue (ParentQueue.java:setupQueueConfigs(125)) - root, capacity=1.0, asboluteCapacity=1.0, maxCapacity=1.0, asboluteMaxCapacity=1.0, state=RUNNING, acls=ADMINISTER_QUEUE:*SUBMIT_APP:*, labels=*, , reservationsContinueLooking=true 2015-11-22 19:29:54,233 INFO [main] capacity.ParentQueue (ParentQueue.java:(104)) - Initialized parent-queue root name=root, fullname=root 2015-11-22 19:29:54,233 DEBUG [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getQueues(584)) - CSConf - getQueues called for: queuePrefix=yarn.scheduler.capacity.root.default. 2015-11-22 19:29:54,233 DEBUG [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getQueues(594)) - CSConf - getQueues: queuePrefix=yarn.scheduler.capacity.root.default., queues= 2015-11-22 19:29:54,234 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CSQueueMetrics.AMResourceLimitMB with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[AM memory limit in MB]) 2015-11-22 19:29:54,234 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CSQueueMetrics.AMResourceLimitVCores with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[AM CPU limit in virtual cores]) 2015-11-22 19:29:54,234 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CSQueueMetrics.usedAMResourceMB with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Used AM memory limit in MB]) 2015-11-22 19:29:54,234 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CSQueueMetrics.usedAMResourceVCores with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Used AM CPU limit in virtual cores]) 2015-11-22 19:29:54,234 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterInt org.apache.hadoop.yarn.server.resourcemanager.scheduler.QueueMetrics.appsSubmitted with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[# of apps submitted]) 2015-11-22 19:29:54,234 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.resourcemanager.scheduler.QueueMetrics.appsRunning with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[# of running apps]) 2015-11-22 19:29:54,234 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.resourcemanager.scheduler.QueueMetrics.appsPending with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[# of pending apps]) 2015-11-22 19:29:54,234 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterInt org.apache.hadoop.yarn.server.resourcemanager.scheduler.QueueMetrics.appsCompleted with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[# of apps completed]) 2015-11-22 19:29:54,235 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterInt org.apache.hadoop.yarn.server.resourcemanager.scheduler.QueueMetrics.appsKilled with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[# of apps killed]) 2015-11-22 19:29:54,235 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterInt org.apache.hadoop.yarn.server.resourcemanager.scheduler.QueueMetrics.appsFailed with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[# of apps failed]) 2015-11-22 19:29:54,235 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.resourcemanager.scheduler.QueueMetrics.allocatedMB with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Allocated memory in MB]) 2015-11-22 19:29:54,235 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.resourcemanager.scheduler.QueueMetrics.allocatedVCores with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Allocated CPU in virtual cores]) 2015-11-22 19:29:54,235 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.resourcemanager.scheduler.QueueMetrics.allocatedContainers with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[# of allocated containers]) 2015-11-22 19:29:54,235 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.yarn.server.resourcemanager.scheduler.QueueMetrics.aggregateContainersAllocated with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Aggregate # of allocated containers]) 2015-11-22 19:29:54,235 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.yarn.server.resourcemanager.scheduler.QueueMetrics.aggregateNodeLocalContainersAllocated with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Aggregate # of allocated node-local containers]) 2015-11-22 19:29:54,235 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.yarn.server.resourcemanager.scheduler.QueueMetrics.aggregateRackLocalContainersAllocated with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Aggregate # of allocated rack-local containers]) 2015-11-22 19:29:54,236 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.yarn.server.resourcemanager.scheduler.QueueMetrics.aggregateOffSwitchContainersAllocated with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Aggregate # of allocated off-switch containers]) 2015-11-22 19:29:54,236 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.yarn.server.resourcemanager.scheduler.QueueMetrics.aggregateContainersReleased with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Aggregate # of released containers]) 2015-11-22 19:29:54,236 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.resourcemanager.scheduler.QueueMetrics.availableMB with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Available memory in MB]) 2015-11-22 19:29:54,236 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.resourcemanager.scheduler.QueueMetrics.availableVCores with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Available CPU in virtual cores]) 2015-11-22 19:29:54,236 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.resourcemanager.scheduler.QueueMetrics.pendingMB with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Pending memory allocation in MB]) 2015-11-22 19:29:54,236 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.resourcemanager.scheduler.QueueMetrics.pendingVCores with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Pending CPU allocation in virtual cores]) 2015-11-22 19:29:54,236 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.resourcemanager.scheduler.QueueMetrics.pendingContainers with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[# of pending containers]) 2015-11-22 19:29:54,236 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.resourcemanager.scheduler.QueueMetrics.reservedMB with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[# of reserved memory in MB]) 2015-11-22 19:29:54,237 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.resourcemanager.scheduler.QueueMetrics.reservedVCores with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Reserved CPU in virtual cores]) 2015-11-22 19:29:54,237 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.resourcemanager.scheduler.QueueMetrics.reservedContainers with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[# of reserved containers]) 2015-11-22 19:29:54,237 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.resourcemanager.scheduler.QueueMetrics.activeUsers with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[# of active users]) 2015-11-22 19:29:54,237 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.yarn.server.resourcemanager.scheduler.QueueMetrics.activeApplications with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[# of active applications]) 2015-11-22 19:29:54,237 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.yarn.server.resourcemanager.scheduler.QueueMetrics.appAttemptFirstContainerAllocationDelay with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[App Attempt First Container Allocation Delay]) 2015-11-22 19:29:54,237 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:register(232)) - QueueMetrics,q0=root,q1=default, Metrics for queue: root.default 2015-11-22 19:29:54,237 DEBUG [main] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'PropertiesConfiguration' for key: source.source.start_mbeans 2015-11-22 19:29:54,237 DEBUG [main] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'MetricsConfig' for key: source.start_mbeans 2015-11-22 19:29:54,238 DEBUG [main] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'PropertiesConfiguration' for key: *.source.start_mbeans 2015-11-22 19:29:54,238 DEBUG [main] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateAttrCache(254)) - Updating attr cache... 2015-11-22 19:29:54,238 DEBUG [main] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateAttrCache(268)) - Done. # tags & metrics=37 2015-11-22 19:29:54,238 DEBUG [main] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateInfoCache(248)) - Updating info cache... 2015-11-22 19:29:54,238 DEBUG [main] impl.MetricsSystemImpl (MBeanInfoBuilder.java:get(109)) - [javax.management.MBeanAttributeInfo[description=Metrics by queue, name=tag.Queue, type=java.lang.String, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Metrics context, name=tag.Context, type=java.lang.String, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Local hostname, name=tag.Hostname, type=java.lang.String, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=, name=running_0, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=, name=running_60, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=, name=running_300, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=, name=running_1440, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=AM memory limit in MB, name=AMResourceLimitMB, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=AM CPU limit in virtual cores, name=AMResourceLimitVCores, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Used AM memory limit in MB, name=UsedAMResourceMB, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Used AM CPU limit in virtual cores, name=UsedAMResourceVCores, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=# of apps submitted, name=AppsSubmitted, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=# of running apps, name=AppsRunning, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=# of pending apps, name=AppsPending, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=# of apps completed, name=AppsCompleted, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=# of apps killed, name=AppsKilled, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=# of apps failed, name=AppsFailed, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Allocated memory in MB, name=AllocatedMB, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Allocated CPU in virtual cores, name=AllocatedVCores, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=# of allocated containers, name=AllocatedContainers, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Aggregate # of allocated containers, name=AggregateContainersAllocated, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Aggregate # of allocated node-local containers, name=AggregateNodeLocalContainersAllocated, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Aggregate # of allocated rack-local containers, name=AggregateRackLocalContainersAllocated, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Aggregate # of allocated off-switch containers, name=AggregateOffSwitchContainersAllocated, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Aggregate # of released containers, name=AggregateContainersReleased, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Available memory in MB, name=AvailableMB, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Available CPU in virtual cores, name=AvailableVCores, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Pending memory allocation in MB, name=PendingMB, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Pending CPU allocation in virtual cores, name=PendingVCores, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=# of pending containers, name=PendingContainers, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=# of reserved memory in MB, name=ReservedMB, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Reserved CPU in virtual cores, name=ReservedVCores, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=# of reserved containers, name=ReservedContainers, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=# of active users, name=ActiveUsers, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=# of active applications, name=ActiveApplications, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of ops for app Attempt First Container Allocation Delay, name=AppAttemptFirstContainerAllocationDelayNumOps, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Average time for app Attempt First Container Allocation Delay, name=AppAttemptFirstContainerAllocationDelayAvgTime, type=java.lang.Double, read-only, descriptor={}]] 2015-11-22 19:29:54,239 DEBUG [main] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateInfoCache(250)) - Done 2015-11-22 19:29:54,239 DEBUG [main] util.MBeans (MBeans.java:register(67)) - Registered Hadoop:service=ResourceManager,name=QueueMetrics,q0=root,q1=default 2015-11-22 19:29:54,239 DEBUG [main] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:startMBeans(227)) - MBean for source QueueMetrics,q0=root,q1=default registered. 2015-11-22 19:29:54,239 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:registerSource(269)) - Registered source QueueMetrics,q0=root,q1=default 2015-11-22 19:29:54,240 DEBUG [main] capacity.LeafQueue (LeafQueue.java:(134)) - LeafQueue: name=default, fullname=root.default 2015-11-22 19:29:54,240 DEBUG [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getNonLabeledQueueCapacity(333)) - CSConf - getCapacity: queuePrefix=yarn.scheduler.capacity.root.default., capacity=100.0 2015-11-22 19:29:54,240 DEBUG [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(640)) - max alloc mb per queue for root.default is -1 2015-11-22 19:29:54,241 DEBUG [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(642)) - max alloc vcores per queue for root.default is -1 2015-11-22 19:29:54,241 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(647)) - max alloc mb per queue for root.default is undefined 2015-11-22 19:29:54,241 INFO [main] capacity.CapacitySchedulerConfiguration (CapacitySchedulerConfiguration.java:getMaximumAllocationPerQueue(651)) - max alloc vcore per queue for root.default is undefined 2015-11-22 19:29:54,242 INFO [main] capacity.LeafQueue (LeafQueue.java:setupQueueConfigs(217)) - Initializing default capacity = 1.0 [= (float) configuredCapacity / 100 ] asboluteCapacity = 1.0 [= parentAbsoluteCapacity * capacity ] maxCapacity = 1.0 [= configuredMaxCapacity ] absoluteMaxCapacity = 1.0 [= 1.0 maximumCapacity undefined, (parentAbsoluteMaxCapacity * maximumCapacity) / 100 otherwise ] userLimit = 100 [= configuredUserLimit ] userLimitFactor = 1.0 [= configuredUserLimitFactor ] maxApplications = 10000 [= configuredMaximumSystemApplicationsPerQueue or (int)(configuredMaximumSystemApplications * absoluteCapacity)] maxApplicationsPerUser = 10000 [= (int)(maxApplications * (userLimit / 100.0f) * userLimitFactor) ] usedCapacity = 0.0 [= usedResourcesMemory / (clusterResourceMemory * absoluteCapacity)] absoluteUsedCapacity = 0.0 [= usedResourcesMemory / clusterResourceMemory] maxAMResourcePerQueuePercent = 0.1 [= configuredMaximumAMResourcePercent ] minimumAllocationFactor = 0.984375 [= (float)(maximumAllocationMemory - minimumAllocationMemory) / maximumAllocationMemory ] maximumAllocation = [= configuredMaxAllocation ] numContainers = 0 [= currentNumContainers ] state = RUNNING [= configuredState ] acls = ADMINISTER_QUEUE:*SUBMIT_APP:* [= configuredAcls ] nodeLocalityDelay = -1 labels=*, nodeLocalityDelay = -1 reservationsContinueLooking = true preemptionDisabled = true defaultAppPriorityPerQueue = 0 2015-11-22 19:29:54,242 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:parseQueue(657)) - Initialized queue: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=0, numContainers=0 2015-11-22 19:29:54,242 DEBUG [main] capacity.ParentQueue (ParentQueue.java:setChildQueues(170)) - setChildQueues: root.defaultusedCapacity=(0.0), label=(*) 2015-11-22 19:29:54,242 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:parseQueue(657)) - Initialized queue: root: numChildQueue= 1, capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.0, numApps=0, numContainers=0 2015-11-22 19:29:54,242 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:initializeQueues(501)) - Initialized root queue root: numChildQueue= 1, capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.0, numApps=0, numContainers=0 2015-11-22 19:29:54,242 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:getUserGroupMappingPlacementRule(449)) - Initialized queue mappings, override: false 2015-11-22 19:29:54,243 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:initScheduler(314)) - Initialized CapacityScheduler with calculator=class org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator, minimumAllocation=<>, maximumAllocation=<>, asynchronousScheduling=false, asyncScheduleInterval=5ms 2015-11-22 19:29:54,243 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher entered state INITED 2015-11-22 19:29:54,243 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: NMLivelinessMonitor entered state INITED 2015-11-22 19:29:54,243 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.ResourceTrackerService entered state INITED 2015-11-22 19:29:54,243 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService entered state INITED 2015-11-22 19:29:54,244 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.ClientRMService entered state INITED 2015-11-22 19:29:54,244 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher entered state INITED 2015-11-22 19:29:54,244 DEBUG [main] service.CompositeService (CompositeService.java:addService(73)) - Adding service org.apache.hadoop.yarn.server.resourcemanager.ahs.RMApplicationHistoryWriter 2015-11-22 19:29:54,244 DEBUG [main] service.CompositeService (CompositeService.java:addService(73)) - Adding service org.apache.hadoop.yarn.server.resourcemanager.metrics.SystemMetricsPublisher 2015-11-22 19:29:54,244 DEBUG [main] service.CompositeService (CompositeService.java:serviceInit(104)) - ResourceManager: initing services, size=4 2015-11-22 19:29:54,245 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: Dispatcher entered state INITED 2015-11-22 19:29:54,245 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.AdminService entered state INITED 2015-11-22 19:29:54,245 DEBUG [main] service.CompositeService (CompositeService.java:serviceInit(104)) - org.apache.hadoop.yarn.server.resourcemanager.AdminService: initing services, size=0 2015-11-22 19:29:54,245 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.ahs.RMApplicationHistoryWriter entered state INITED 2015-11-22 19:29:54,245 DEBUG [main] service.CompositeService (CompositeService.java:serviceInit(104)) - org.apache.hadoop.yarn.server.resourcemanager.ahs.RMApplicationHistoryWriter: initing services, size=0 2015-11-22 19:29:54,245 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.metrics.SystemMetricsPublisher entered state INITED 2015-11-22 19:29:54,245 INFO [main] metrics.SystemMetricsPublisher (SystemMetricsPublisher.java:serviceInit(95)) - YARN system metrics publishing service is not enabled 2015-11-22 19:29:54,245 DEBUG [main] service.CompositeService (CompositeService.java:serviceInit(104)) - org.apache.hadoop.yarn.server.resourcemanager.metrics.SystemMetricsPublisher: initing services, size=0 2015-11-22 19:29:54,245 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEventType for class org.apache.hadoop.yarn.server.MiniYARNCluster$1 2015-11-22 19:29:54,245 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.MiniYARNCluster$NodeManagerWrapper_0 entered state INITED 2015-11-22 19:29:54,246 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:prepareDirs(558)) - Created localDir in /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0 2015-11-22 19:29:54,246 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:prepareDirs(558)) - Created logDir in /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-logDir-nm-0_0 2015-11-22 19:29:54,246 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceInit(540)) - Starting NM: 0 2015-11-22 19:29:54,246 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: NodeManager entered state INITED 2015-11-22 19:29:54,246 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.nodemanager.recovery.NMNullStateStoreService entered state INITED 2015-11-22 19:29:54,246 DEBUG [main] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.nodemanager.recovery.NMNullStateStoreService is started 2015-11-22 19:29:54,246 DEBUG [main] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.fs.FileContext.getAbstractFileSystem(FileContext.java:339) 2015-11-22 19:29:54,256 DEBUG [main] service.CompositeService (CompositeService.java:addService(73)) - Adding service org.apache.hadoop.yarn.server.nodemanager.DeletionService 2015-11-22 19:29:54,256 INFO [main] nodemanager.NodeManager (NodeManager.java:getNodeHealthScriptRunner(255)) - Node Manager health check script is not available or doesn't have execute permission, so not starting the node health script runner. 2015-11-22 19:29:54,256 DEBUG [main] service.CompositeService (CompositeService.java:addService(73)) - Adding service org.apache.hadoop.yarn.server.nodemanager.NodeHealthCheckerService 2015-11-22 19:29:54,257 DEBUG [main] service.CompositeService (CompositeService.java:addService(73)) - Adding service org.apache.hadoop.yarn.server.nodemanager.NodeResourceMonitorImpl 2015-11-22 19:29:54,257 DEBUG [main] service.CompositeService (CompositeService.java:addService(73)) - Adding service org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService 2015-11-22 19:29:54,257 DEBUG [main] service.CompositeService (CompositeService.java:addService(73)) - Adding service containers-launcher 2015-11-22 19:29:54,257 DEBUG [main] service.CompositeService (CompositeService.java:addService(73)) - Adding service org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices 2015-11-22 19:29:54,257 DEBUG [main] service.CompositeService (CompositeService.java:addService(73)) - Adding service containers-monitor 2015-11-22 19:29:54,257 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ContainerEventDispatcher 2015-11-22 19:29:54,257 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ApplicationEventDispatcher 2015-11-22 19:29:54,257 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizationEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService 2015-11-22 19:29:54,257 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServicesEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices 2015-11-22 19:29:54,257 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl 2015-11-22 19:29:54,258 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncherEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncher 2015-11-22 19:29:54,258 DEBUG [main] service.CompositeService (CompositeService.java:addService(73)) - Adding service Dispatcher 2015-11-22 19:29:54,258 DEBUG [main] service.CompositeService (CompositeService.java:addService(73)) - Adding service org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl 2015-11-22 19:29:54,258 DEBUG [main] service.CompositeService (CompositeService.java:addService(73)) - Adding service org.apache.hadoop.yarn.server.nodemanager.webapp.WebServer 2015-11-22 19:29:54,258 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.ContainerManagerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl 2015-11-22 19:29:54,258 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.NodeManagerEventType for class org.apache.hadoop.yarn.server.MiniYARNCluster$ShortCircuitedNodeManager 2015-11-22 19:29:54,258 DEBUG [main] service.CompositeService (CompositeService.java:addService(73)) - Adding service Dispatcher 2015-11-22 19:29:54,258 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:init(159)) - NodeManager metrics system started (again) 2015-11-22 19:29:54,258 DEBUG [main] service.CompositeService (CompositeService.java:addService(73)) - Adding service org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl 2015-11-22 19:29:54,258 DEBUG [main] service.CompositeService (CompositeService.java:serviceInit(104)) - NodeManager: initing services, size=7 2015-11-22 19:29:54,258 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.nodemanager.DeletionService entered state INITED 2015-11-22 19:29:54,258 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.nodemanager.NodeHealthCheckerService entered state INITED 2015-11-22 19:29:54,259 DEBUG [main] service.CompositeService (CompositeService.java:addService(73)) - Adding service org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService 2015-11-22 19:29:54,259 DEBUG [main] service.CompositeService (CompositeService.java:serviceInit(104)) - org.apache.hadoop.yarn.server.nodemanager.NodeHealthCheckerService: initing services, size=1 2015-11-22 19:29:54,259 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService entered state INITED 2015-11-22 19:29:54,259 DEBUG [main] service.AbstractService (AbstractService.java:serviceInit(320)) - Config has been overridden during init 2015-11-22 19:29:54,259 DEBUG [main] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.fs.FileContext.getAbstractFileSystem(FileContext.java:339) 2015-11-22 19:29:54,260 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.nodemanager.NodeResourceMonitorImpl entered state INITED 2015-11-22 19:29:54,260 INFO [main] nodemanager.NodeResourceMonitorImpl (NodeResourceMonitorImpl.java:serviceInit(76)) - Using ResourceCalculatorPlugin : org.apache.hadoop.yarn.util.ResourceCalculatorPlugin@56cdfb3b 2015-11-22 19:29:54,260 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl entered state INITED 2015-11-22 19:29:54,260 DEBUG [main] service.CompositeService (CompositeService.java:addService(73)) - Adding service org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.NonAggregatingLogHandler 2015-11-22 19:29:54,260 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.event.LogHandlerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.NonAggregatingLogHandler 2015-11-22 19:29:54,260 DEBUG [main] service.CompositeService (CompositeService.java:addService(73)) - Adding service org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.sharedcache.SharedCacheUploadService 2015-11-22 19:29:54,261 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.sharedcache.SharedCacheUploadEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.sharedcache.SharedCacheUploadService 2015-11-22 19:29:54,261 INFO [main] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:serviceInit(261)) - AMRMProxyService is disabled 2015-11-22 19:29:54,261 DEBUG [main] service.CompositeService (CompositeService.java:serviceInit(104)) - org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl: initing services, size=7 2015-11-22 19:29:54,261 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService entered state INITED 2015-11-22 19:29:54,261 INFO [main] localizer.ResourceLocalizationService (ResourceLocalizationService.java:validateConf(224)) - per directory file limit = 8192 2015-11-22 19:29:54,261 DEBUG [main] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.fs.FileContext.getAbstractFileSystem(FileContext.java:339) 2015-11-22 19:29:54,271 DEBUG [main] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.fs.FileContext.getAbstractFileSystem(FileContext.java:339) 2015-11-22 19:29:54,272 DEBUG [main] service.CompositeService (CompositeService.java:addService(73)) - Adding service org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerTracker 2015-11-22 19:29:54,272 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerTracker 2015-11-22 19:29:54,272 DEBUG [main] service.CompositeService (CompositeService.java:serviceInit(104)) - org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: initing services, size=1 2015-11-22 19:29:54,272 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerTracker entered state INITED 2015-11-22 19:29:54,272 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: containers-launcher entered state INITED 2015-11-22 19:29:54,272 DEBUG [main] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.fs.FileContext.getAbstractFileSystem(FileContext.java:339) 2015-11-22 19:29:54,272 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices entered state INITED 2015-11-22 19:29:54,272 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: containers-monitor entered state INITED 2015-11-22 19:29:54,272 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(116)) - Using ResourceCalculatorPlugin : org.apache.hadoop.yarn.util.ResourceCalculatorPlugin@2b91004a 2015-11-22 19:29:54,273 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(121)) - Using ResourceCalculatorProcessTree : null 2015-11-22 19:29:54,273 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(158)) - Physical memory check enabled: false 2015-11-22 19:29:54,273 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(159)) - Virtual memory check enabled: false 2015-11-22 19:29:54,273 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:isEnabled(206)) - Neither virutal-memory nor physical-memory monitoring is needed. Not running the monitor-thread 2015-11-22 19:29:54,273 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(162)) - ContainersMonitor enabled: false 2015-11-22 19:29:54,273 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: Dispatcher entered state INITED 2015-11-22 19:29:54,273 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.NonAggregatingLogHandler entered state INITED 2015-11-22 19:29:54,273 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.sharedcache.SharedCacheUploadService entered state INITED 2015-11-22 19:29:54,273 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.nodemanager.webapp.WebServer entered state INITED 2015-11-22 19:29:54,273 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: Dispatcher entered state INITED 2015-11-22 19:29:54,273 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl entered state INITED 2015-11-22 19:29:54,274 INFO [main] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:serviceInit(175)) - Nodemanager resources: memory set to 4096MB. 2015-11-22 19:29:54,274 INFO [main] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:serviceInit(176)) - Nodemanager resources: vcores set to 8. 2015-11-22 19:29:54,274 DEBUG [main] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:serviceInit(204)) - yarn.nodemanager.duration-to-track-stopped-containers :600000 2015-11-22 19:29:54,274 INFO [main] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:serviceInit(208)) - Initialized nodemanager with : physical-memory=4096 virtual-memory=8602 virtual-cores=8 2015-11-22 19:29:54,274 DEBUG [main] service.AbstractService (AbstractService.java:serviceInit(320)) - Config has been overridden during init 2015-11-22 19:29:54,274 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.MiniYARNCluster$ApplicationHistoryServerWrapper entered state INITED 2015-11-22 19:29:54,274 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer entered state INITED 2015-11-22 19:29:54,274 DEBUG [main] service.CompositeService (CompositeService.java:addService(73)) - Adding service org.apache.hadoop.yarn.server.timeline.MemoryTimelineStore 2015-11-22 19:29:54,274 DEBUG [main] service.CompositeService (CompositeService.java:addService(73)) - Adding service org.apache.hadoop.yarn.server.timeline.security.TimelineDelegationTokenSecretManagerService 2015-11-22 19:29:54,274 DEBUG [main] service.CompositeService (CompositeService.java:addService(73)) - Adding service org.apache.hadoop.yarn.server.timeline.TimelineDataManager 2015-11-22 19:29:54,274 WARN [main] applicationhistoryservice.ApplicationHistoryServer (ApplicationHistoryServer.java:createApplicationHistoryManager(218)) - The filesystem based application history store is deprecated. 2015-11-22 19:29:54,275 DEBUG [main] service.CompositeService (CompositeService.java:addService(73)) - Adding service ApplicationHistoryClientService 2015-11-22 19:29:54,275 DEBUG [main] service.CompositeService (CompositeService.java:addService(73)) - Adding service org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryManagerImpl 2015-11-22 19:29:54,275 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:init(159)) - ApplicationHistoryServer metrics system started (again) 2015-11-22 19:29:54,275 DEBUG [main] service.CompositeService (CompositeService.java:serviceInit(104)) - org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer: initing services, size=5 2015-11-22 19:29:54,275 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.timeline.MemoryTimelineStore entered state INITED 2015-11-22 19:29:54,275 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.timeline.security.TimelineDelegationTokenSecretManagerService entered state INITED 2015-11-22 19:29:54,275 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.timeline.TimelineDataManager entered state INITED 2015-11-22 19:29:54,275 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.yarn.server.timeline.TimelineDataManagerMetrics.getEntitiesOps with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[getEntities calls]) 2015-11-22 19:29:54,275 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.yarn.server.timeline.TimelineDataManagerMetrics.getEntitiesTotal with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Entities returned via getEntities]) 2015-11-22 19:29:54,276 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.yarn.server.timeline.TimelineDataManagerMetrics.getEntitiesTime with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[getEntities processing time]) 2015-11-22 19:29:54,276 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.yarn.server.timeline.TimelineDataManagerMetrics.getEntityOps with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[getEntity calls]) 2015-11-22 19:29:54,276 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.yarn.server.timeline.TimelineDataManagerMetrics.getEntityTime with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[getEntity processing time]) 2015-11-22 19:29:54,276 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.yarn.server.timeline.TimelineDataManagerMetrics.getEventsOps with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[getEvents calls]) 2015-11-22 19:29:54,276 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.yarn.server.timeline.TimelineDataManagerMetrics.getEventsTotal with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Events returned via getEvents]) 2015-11-22 19:29:54,276 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.yarn.server.timeline.TimelineDataManagerMetrics.getEventsTime with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[getEvents processing time]) 2015-11-22 19:29:54,276 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.yarn.server.timeline.TimelineDataManagerMetrics.postEntitiesOps with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[postEntities calls]) 2015-11-22 19:29:54,276 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.yarn.server.timeline.TimelineDataManagerMetrics.postEntitiesTotal with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Entities posted via postEntities]) 2015-11-22 19:29:54,276 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.yarn.server.timeline.TimelineDataManagerMetrics.postEntitiesTime with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[postEntities processing time]) 2015-11-22 19:29:54,277 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.yarn.server.timeline.TimelineDataManagerMetrics.putDomainOps with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[putDomain calls]) 2015-11-22 19:29:54,277 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.yarn.server.timeline.TimelineDataManagerMetrics.putDomainTime with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[putDomain processing time]) 2015-11-22 19:29:54,277 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.yarn.server.timeline.TimelineDataManagerMetrics.getDomainOps with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[getDomain calls]) 2015-11-22 19:29:54,277 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.yarn.server.timeline.TimelineDataManagerMetrics.getDomainTime with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[getDomain processing time]) 2015-11-22 19:29:54,277 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.yarn.server.timeline.TimelineDataManagerMetrics.getDomainsOps with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[getDomains calls]) 2015-11-22 19:29:54,277 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.yarn.server.timeline.TimelineDataManagerMetrics.getDomainsTotal with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Domains returned via getDomains]) 2015-11-22 19:29:54,277 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.yarn.server.timeline.TimelineDataManagerMetrics.getDomainsTime with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[getDomains processing time]) 2015-11-22 19:29:54,277 DEBUG [main] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForMethod(82)) - method public long org.apache.hadoop.yarn.server.timeline.TimelineDataManagerMetrics.totalOps() with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Total calls]) 2015-11-22 19:29:54,278 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:register(232)) - TimelineDataManagerMetrics, Metrics for TimelineDataManager 2015-11-22 19:29:54,278 DEBUG [main] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'PropertiesConfiguration' for key: source.source.start_mbeans 2015-11-22 19:29:54,278 DEBUG [main] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'MetricsConfig' for key: source.start_mbeans 2015-11-22 19:29:54,278 DEBUG [main] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'PropertiesConfiguration' for key: *.source.start_mbeans 2015-11-22 19:29:54,278 DEBUG [main] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateAttrCache(254)) - Updating attr cache... 2015-11-22 19:29:54,278 DEBUG [main] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateAttrCache(268)) - Done. # tags & metrics=28 2015-11-22 19:29:54,278 DEBUG [main] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateInfoCache(248)) - Updating info cache... 2015-11-22 19:29:54,278 DEBUG [main] impl.MetricsSystemImpl (MBeanInfoBuilder.java:get(109)) - [javax.management.MBeanAttributeInfo[description=Metrics context, name=tag.Context, type=java.lang.String, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Local hostname, name=tag.Hostname, type=java.lang.String, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=getEntities calls, name=GetEntitiesOps, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Entities returned via getEntities, name=GetEntitiesTotal, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of ops for getEntities processing time, name=GetEntitiesTimeNumOps, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Average time for getEntities processing time, name=GetEntitiesTimeAvgTime, type=java.lang.Double, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=getEntity calls, name=GetEntityOps, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of ops for getEntity processing time, name=GetEntityTimeNumOps, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Average time for getEntity processing time, name=GetEntityTimeAvgTime, type=java.lang.Double, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=getEvents calls, name=GetEventsOps, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Events returned via getEvents, name=GetEventsTotal, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of ops for getEvents processing time, name=GetEventsTimeNumOps, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Average time for getEvents processing time, name=GetEventsTimeAvgTime, type=java.lang.Double, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=postEntities calls, name=PostEntitiesOps, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Entities posted via postEntities, name=PostEntitiesTotal, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of ops for postEntities processing time, name=PostEntitiesTimeNumOps, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Average time for postEntities processing time, name=PostEntitiesTimeAvgTime, type=java.lang.Double, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=putDomain calls, name=PutDomainOps, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of ops for putDomain processing time, name=PutDomainTimeNumOps, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Average time for putDomain processing time, name=PutDomainTimeAvgTime, type=java.lang.Double, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=getDomain calls, name=GetDomainOps, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of ops for getDomain processing time, name=GetDomainTimeNumOps, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Average time for getDomain processing time, name=GetDomainTimeAvgTime, type=java.lang.Double, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=getDomains calls, name=GetDomainsOps, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Domains returned via getDomains, name=GetDomainsTotal, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of ops for getDomains processing time, name=GetDomainsTimeNumOps, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Average time for getDomains processing time, name=GetDomainsTimeAvgTime, type=java.lang.Double, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Total calls, name=TotalOps, type=java.lang.Long, read-only, descriptor={}]] 2015-11-22 19:29:54,279 DEBUG [main] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateInfoCache(250)) - Done 2015-11-22 19:29:54,279 DEBUG [main] util.MBeans (MBeans.java:register(67)) - Registered Hadoop:service=ApplicationHistoryServer,name=TimelineDataManagerMetrics 2015-11-22 19:29:54,279 DEBUG [main] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:startMBeans(227)) - MBean for source TimelineDataManagerMetrics registered. 2015-11-22 19:29:54,279 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:registerSource(269)) - Registered source TimelineDataManagerMetrics 2015-11-22 19:29:54,279 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: ApplicationHistoryClientService entered state INITED 2015-11-22 19:29:54,279 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryManagerImpl entered state INITED 2015-11-22 19:29:54,279 INFO [main] applicationhistoryservice.ApplicationHistoryManagerImpl (ApplicationHistoryManagerImpl.java:serviceInit(61)) - ApplicationHistory Init 2015-11-22 19:29:54,279 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.applicationhistoryservice.MemoryApplicationHistoryStore entered state INITED 2015-11-22 19:29:54,279 DEBUG [main] service.CompositeService (CompositeService.java:serviceStart(115)) - TestDistributedShell: starting services, size=3 2015-11-22 19:29:54,279 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:startResourceManager(326)) - Waiting for RM to start... 2015-11-22 19:29:54,280 INFO [RM-0] resourcemanager.ResourceManager (ResourceManager.java:transitionToActive(1024)) - Transitioning to active state 2015-11-22 19:29:54,280 DEBUG [RM-0] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.transitionToActive(ResourceManager.java:1026) 2015-11-22 19:29:54,280 DEBUG [RM-0] service.AbstractService (AbstractService.java:start(197)) - Service Dispatcher is started 2015-11-22 19:29:54,280 DEBUG [RM-0] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore is started 2015-11-22 19:29:54,281 DEBUG [RM-0] service.CompositeService (CompositeService.java:serviceStart(115)) - RMActiveServices: starting services, size=13 2015-11-22 19:29:54,281 DEBUG [RM-0] recovery.RMStateStore (RMStateStore.java:handleStoreEvent(1016)) - Processing event of type UPDATE_AMRM_TOKEN 2015-11-22 19:29:54,281 INFO [RM-0] recovery.RMStateStore (RMStateStore.java:transition(482)) - Updating AMRMToken 2015-11-22 19:29:54,281 INFO [RM-0] security.RMContainerTokenSecretManager (RMContainerTokenSecretManager.java:rollMasterKey(106)) - Rolling master-key for container-tokens 2015-11-22 19:29:54,281 INFO [RM-0] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:rollMasterKey(95)) - Rolling master-key for nm-tokens 2015-11-22 19:29:54,281 INFO [RM-0] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:29:54,281 INFO [RM-0] security.RMDelegationTokenSecretManager (RMDelegationTokenSecretManager.java:storeNewMasterKey(87)) - storing master key with keyID 1 2015-11-22 19:29:54,281 DEBUG [RM-0] recovery.RMStateStore (RMStateStore.java:handleStoreEvent(1016)) - Processing event of type STORE_MASTERKEY 2015-11-22 19:29:54,281 INFO [RM-0] recovery.RMStateStore (RMStateStore.java:transition(433)) - Storing RMDTMasterKey. 2015-11-22 19:29:54,282 DEBUG [RM-0] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.RMSecretManagerService is started 2015-11-22 19:29:54,282 INFO [Thread[Thread-3878,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2015-11-22 19:29:54,282 DEBUG [RM-0] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.ContainerAllocationExpirer is started 2015-11-22 19:29:54,282 INFO [Thread[Thread-3878,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:29:54,282 DEBUG [RM-0] service.AbstractService (AbstractService.java:start(197)) - Service AMLivelinessMonitor is started 2015-11-22 19:29:54,282 INFO [Thread[Thread-3878,5,main]] security.RMDelegationTokenSecretManager (RMDelegationTokenSecretManager.java:storeNewMasterKey(87)) - storing master key with keyID 2 2015-11-22 19:29:54,283 DEBUG [RM-0] service.AbstractService (AbstractService.java:start(197)) - Service AMLivelinessMonitor is started 2015-11-22 19:29:54,283 DEBUG [Thread[Thread-3878,5,main]] recovery.RMStateStore (RMStateStore.java:handleStoreEvent(1016)) - Processing event of type STORE_MASTERKEY 2015-11-22 19:29:54,283 INFO [Thread[Thread-3878,5,main]] recovery.RMStateStore (RMStateStore.java:transition(433)) - Storing RMDTMasterKey. 2015-11-22 19:29:54,284 INFO [RM-0] nodelabels.CommonNodeLabelsManager (CommonNodeLabelsManager.java:addToCluserNodeLabels(309)) - Add labels: [] 2015-11-22 19:29:54,284 INFO [RM-0] nodelabels.CommonNodeLabelsManager (CommonNodeLabelsManager.java:internalUpdateLabelsOnNodes(638)) - REPLACE labels on nodes: 2015-11-22 19:29:54,284 INFO [RM-0] nodelabels.CommonNodeLabelsManager (CommonNodeLabelsManager.java:internalUpdateLabelsOnNodes(640)) - NM=ip-172-31-20-42.ap-northeast-1.compute.internal:58694, labels=[x] 2015-11-22 19:29:54,285 INFO [RM-0] nodelabels.FileSystemNodeLabelsStore (FileSystemNodeLabelsStore.java:recover(279)) - Finished write mirror at:file:/tmp/hadoop-yarn-ubuntu/node-labels/nodelabel.mirror 2015-11-22 19:29:54,285 INFO [RM-0] nodelabels.FileSystemNodeLabelsStore (FileSystemNodeLabelsStore.java:recover(280)) - Finished create editlog file at:file:/tmp/hadoop-yarn-ubuntu/node-labels/nodelabel.editlog 2015-11-22 19:29:54,285 DEBUG [RM-0] service.AbstractService (AbstractService.java:enterState(452)) - Service: Dispatcher entered state INITED 2015-11-22 19:29:54,285 INFO [RM-0] event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering class org.apache.hadoop.yarn.nodelabels.event.NodeLabelsStoreEventType for class org.apache.hadoop.yarn.nodelabels.CommonNodeLabelsManager$ForwardingEventHandler 2015-11-22 19:29:54,285 DEBUG [RM-0] service.AbstractService (AbstractService.java:start(197)) - Service Dispatcher is started 2015-11-22 19:29:54,285 DEBUG [RM-0] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.nodelabels.CommonNodeLabelsManager is started 2015-11-22 19:29:54,286 DEBUG [RM-0] service.CompositeService (CompositeService.java:serviceStart(115)) - org.apache.hadoop.yarn.server.resourcemanager.NodesListManager: starting services, size=0 2015-11-22 19:29:54,286 DEBUG [RM-0] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.NodesListManager is started 2015-11-22 19:29:54,286 DEBUG [RM-0] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler is started 2015-11-22 19:29:54,287 DEBUG [RM-0] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher is started 2015-11-22 19:29:54,287 DEBUG [RM-0] service.AbstractService (AbstractService.java:start(197)) - Service NMLivelinessMonitor is started 2015-11-22 19:29:54,287 DEBUG [RM-0] ipc.YarnRPC (YarnRPC.java:create(59)) - Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC 2015-11-22 19:29:54,287 DEBUG [RM-0] ipc.HadoopYarnProtoRPC (HadoopYarnProtoRPC.java:getServer(62)) - Creating a HadoopYarnProtoRpc server for protocol interface org.apache.hadoop.yarn.server.api.ResourceTracker with 50 handlers 2015-11-22 19:29:54,287 INFO [RM-0] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:29:54,287 DEBUG [RM-0] ipc.Server (Server.java:getAuthMethods(2447)) - Server accepts auth methods:[SIMPLE] 2015-11-22 19:29:54,288 INFO [Socket Reader #1 for port 50902] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 50902 2015-11-22 19:29:54,288 DEBUG [RM-0] metrics.RpcMetrics (RpcMetrics.java:(74)) - Initialized MetricsRegistry{info=MetricsInfoImpl{name=rpc, description=rpc}, tags=[MetricsTag{info=MetricsInfoImpl{name=port, description=RPC port}, value=50902}], metrics=[]} 2015-11-22 19:29:54,288 DEBUG [RM-0] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.ipc.metrics.RpcMetrics.receivedBytes with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Number of received bytes]) 2015-11-22 19:29:54,289 DEBUG [RM-0] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.ipc.metrics.RpcMetrics.sentBytes with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Number of sent bytes]) 2015-11-22 19:29:54,289 DEBUG [RM-0] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.ipc.metrics.RpcMetrics.rpcQueueTime with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Queue time]) 2015-11-22 19:29:54,289 DEBUG [RM-0] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.ipc.metrics.RpcMetrics.rpcProcessingTime with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Processing time]) 2015-11-22 19:29:54,289 DEBUG [RM-0] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.ipc.metrics.RpcMetrics.rpcAuthenticationFailures with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Number of authentication failures]) 2015-11-22 19:29:54,289 DEBUG [RM-0] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.ipc.metrics.RpcMetrics.rpcAuthenticationSuccesses with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Number of authentication successes]) 2015-11-22 19:29:54,289 DEBUG [RM-0] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.ipc.metrics.RpcMetrics.rpcAuthorizationFailures with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Number of authorization failures]) 2015-11-22 19:29:54,289 DEBUG [RM-0] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.ipc.metrics.RpcMetrics.rpcAuthorizationSuccesses with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Number of authorization successes]) 2015-11-22 19:29:54,289 DEBUG [RM-0] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.ipc.metrics.RpcMetrics.rpcClientBackoff with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Number of client backoff requests]) 2015-11-22 19:29:54,289 DEBUG [RM-0] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.ipc.metrics.RpcMetrics.rpcSlowCalls with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Number of Slow RPC calls]) 2015-11-22 19:29:54,289 DEBUG [RM-0] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForMethod(82)) - method public int org.apache.hadoop.ipc.metrics.RpcMetrics.numOpenConnections() with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Number of open connections]) 2015-11-22 19:29:54,290 DEBUG [RM-0] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForMethod(82)) - method public int org.apache.hadoop.ipc.metrics.RpcMetrics.callQueueLength() with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Length of the call queue]) 2015-11-22 19:29:54,290 DEBUG [RM-0] impl.MetricsSystemImpl (MetricsSystemImpl.java:register(232)) - RpcActivityForPort50902, Aggregate RPC metrics 2015-11-22 19:29:54,290 DEBUG [RM-0] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'PropertiesConfiguration' for key: source.source.start_mbeans 2015-11-22 19:29:54,290 DEBUG [RM-0] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'MetricsConfig' for key: source.start_mbeans 2015-11-22 19:29:54,290 DEBUG [RM-0] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'PropertiesConfiguration' for key: *.source.start_mbeans 2015-11-22 19:29:54,290 DEBUG [RM-0] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateAttrCache(254)) - Updating attr cache... 2015-11-22 19:29:54,290 DEBUG [RM-0] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateAttrCache(268)) - Done. # tags & metrics=17 2015-11-22 19:29:54,290 DEBUG [RM-0] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateInfoCache(248)) - Updating info cache... 2015-11-22 19:29:54,290 DEBUG [RM-0] impl.MetricsSystemImpl (MBeanInfoBuilder.java:get(109)) - [javax.management.MBeanAttributeInfo[description=RPC port, name=tag.port, type=java.lang.String, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Metrics context, name=tag.Context, type=java.lang.String, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Local hostname, name=tag.Hostname, type=java.lang.String, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of received bytes, name=ReceivedBytes, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of sent bytes, name=SentBytes, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of ops for queue time, name=RpcQueueTimeNumOps, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Average time for queue time, name=RpcQueueTimeAvgTime, type=java.lang.Double, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of ops for processing time, name=RpcProcessingTimeNumOps, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Average time for processing time, name=RpcProcessingTimeAvgTime, type=java.lang.Double, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of authentication failures, name=RpcAuthenticationFailures, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of authentication successes, name=RpcAuthenticationSuccesses, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of authorization failures, name=RpcAuthorizationFailures, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of authorization successes, name=RpcAuthorizationSuccesses, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of client backoff requests, name=RpcClientBackoff, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of Slow RPC calls, name=RpcSlowCalls, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of open connections, name=NumOpenConnections, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Length of the call queue, name=CallQueueLength, type=java.lang.Integer, read-only, descriptor={}]] 2015-11-22 19:29:54,290 DEBUG [RM-0] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateInfoCache(250)) - Done 2015-11-22 19:29:54,291 DEBUG [RM-0] util.MBeans (MBeans.java:register(67)) - Registered Hadoop:service=ApplicationHistoryServer,name=RpcActivityForPort50902 2015-11-22 19:29:54,291 DEBUG [RM-0] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:startMBeans(227)) - MBean for source RpcActivityForPort50902 registered. 2015-11-22 19:29:54,291 DEBUG [RM-0] impl.MetricsSystemImpl (MetricsSystemImpl.java:registerSource(269)) - Registered source RpcActivityForPort50902 2015-11-22 19:29:54,291 DEBUG [RM-0] metrics.RpcDetailedMetrics (RpcDetailedMetrics.java:(47)) - MetricsInfoImpl{name=rpcdetailed, description=rpcdetailed} 2015-11-22 19:29:54,291 DEBUG [RM-0] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableRates org.apache.hadoop.ipc.metrics.RpcDetailedMetrics.rates with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[]) 2015-11-22 19:29:54,291 DEBUG [RM-0] impl.MetricsSystemImpl (MetricsSystemImpl.java:register(232)) - RpcDetailedActivityForPort50902, Per method RPC metrics 2015-11-22 19:29:54,291 DEBUG [RM-0] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'PropertiesConfiguration' for key: source.source.start_mbeans 2015-11-22 19:29:54,291 DEBUG [RM-0] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'MetricsConfig' for key: source.start_mbeans 2015-11-22 19:29:54,291 DEBUG [RM-0] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'PropertiesConfiguration' for key: *.source.start_mbeans 2015-11-22 19:29:54,291 DEBUG [RM-0] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateAttrCache(254)) - Updating attr cache... 2015-11-22 19:29:54,291 DEBUG [RM-0] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateAttrCache(268)) - Done. # tags & metrics=3 2015-11-22 19:29:54,292 DEBUG [RM-0] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateInfoCache(248)) - Updating info cache... 2015-11-22 19:29:54,292 DEBUG [RM-0] impl.MetricsSystemImpl (MBeanInfoBuilder.java:get(109)) - [javax.management.MBeanAttributeInfo[description=RPC port, name=tag.port, type=java.lang.String, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Metrics context, name=tag.Context, type=java.lang.String, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Local hostname, name=tag.Hostname, type=java.lang.String, read-only, descriptor={}]] 2015-11-22 19:29:54,292 DEBUG [RM-0] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateInfoCache(250)) - Done 2015-11-22 19:29:54,292 DEBUG [RM-0] util.MBeans (MBeans.java:register(67)) - Registered Hadoop:service=ApplicationHistoryServer,name=RpcDetailedActivityForPort50902 2015-11-22 19:29:54,292 DEBUG [RM-0] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:startMBeans(227)) - MBean for source RpcDetailedActivityForPort50902 registered. 2015-11-22 19:29:54,292 DEBUG [RM-0] impl.MetricsSystemImpl (MetricsSystemImpl.java:registerSource(269)) - Registered source RpcDetailedActivityForPort50902 2015-11-22 19:29:54,292 DEBUG [RM-0] ipc.Server (RPC.java:registerProtocolAndImpl(876)) - RpcKind = RPC_PROTOCOL_BUFFER Protocol Name = org.apache.hadoop.ipc.ProtocolMetaInfoPB version=1 ProtocolImpl=org.apache.hadoop.ipc.protobuf.ProtocolInfoProtos$ProtocolInfoService$2 protocolClass=org.apache.hadoop.ipc.ProtocolMetaInfoPB 2015-11-22 19:29:54,292 DEBUG [RM-0] ipc.Server (RPC.java:registerProtocolAndImpl(876)) - RpcKind = RPC_PROTOCOL_BUFFER Protocol Name = org.apache.hadoop.yarn.server.api.ResourceTrackerPB version=1 ProtocolImpl=org.apache.hadoop.yarn.proto.ResourceTracker$ResourceTrackerService$2 protocolClass=org.apache.hadoop.yarn.server.api.ResourceTrackerPB 2015-11-22 19:29:54,292 INFO [RM-0] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.server.api.ResourceTrackerPB to the server 2015-11-22 19:29:54,292 DEBUG [RM-0] ipc.Server (RPC.java:registerProtocolAndImpl(876)) - RpcKind = RPC_PROTOCOL_BUFFER Protocol Name = org.apache.hadoop.yarn.server.api.ResourceTrackerPB version=1 ProtocolImpl=org.apache.hadoop.yarn.proto.ResourceTracker$ResourceTrackerService$2 protocolClass=org.apache.hadoop.yarn.server.api.ResourceTrackerPB 2015-11-22 19:29:54,293 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:29:54,293 INFO [IPC Server listener on 50902] ipc.Server (Server.java:run(794)) - IPC Server listener on 50902: starting 2015-11-22 19:29:54,293 DEBUG [IPC Server handler 0 on 50902] ipc.Server (Server.java:run(2176)) - IPC Server handler 0 on 50902: starting 2015-11-22 19:29:54,293 DEBUG [IPC Server handler 2 on 50902] ipc.Server (Server.java:run(2176)) - IPC Server handler 2 on 50902: starting 2015-11-22 19:29:54,293 DEBUG [IPC Server handler 1 on 50902] ipc.Server (Server.java:run(2176)) - IPC Server handler 1 on 50902: starting 2015-11-22 19:29:54,294 DEBUG [IPC Server handler 3 on 50902] ipc.Server (Server.java:run(2176)) - IPC Server handler 3 on 50902: starting 2015-11-22 19:29:54,294 DEBUG [IPC Server handler 4 on 50902] ipc.Server (Server.java:run(2176)) - IPC Server handler 4 on 50902: starting 2015-11-22 19:29:54,294 DEBUG [IPC Server handler 5 on 50902] ipc.Server (Server.java:run(2176)) - IPC Server handler 5 on 50902: starting 2015-11-22 19:29:54,294 DEBUG [IPC Server handler 6 on 50902] ipc.Server (Server.java:run(2176)) - IPC Server handler 6 on 50902: starting 2015-11-22 19:29:54,294 DEBUG [IPC Server handler 7 on 50902] ipc.Server (Server.java:run(2176)) - IPC Server handler 7 on 50902: starting 2015-11-22 19:29:54,295 DEBUG [IPC Server handler 8 on 50902] ipc.Server (Server.java:run(2176)) - IPC Server handler 8 on 50902: starting 2015-11-22 19:29:54,295 DEBUG [IPC Server handler 9 on 50902] ipc.Server (Server.java:run(2176)) - IPC Server handler 9 on 50902: starting 2015-11-22 19:29:54,295 DEBUG [IPC Server handler 10 on 50902] ipc.Server (Server.java:run(2176)) - IPC Server handler 10 on 50902: starting 2015-11-22 19:29:54,295 DEBUG [IPC Server handler 11 on 50902] ipc.Server (Server.java:run(2176)) - IPC Server handler 11 on 50902: starting 2015-11-22 19:29:54,295 DEBUG [IPC Server handler 13 on 50902] ipc.Server (Server.java:run(2176)) - IPC Server handler 13 on 50902: starting 2015-11-22 19:29:54,295 DEBUG [IPC Server handler 14 on 50902] ipc.Server (Server.java:run(2176)) - IPC Server handler 14 on 50902: starting 2015-11-22 19:29:54,295 DEBUG [IPC Server handler 12 on 50902] ipc.Server (Server.java:run(2176)) - IPC Server handler 12 on 50902: starting 2015-11-22 19:29:54,295 DEBUG [IPC Server handler 15 on 50902] ipc.Server (Server.java:run(2176)) - IPC Server handler 15 on 50902: starting 2015-11-22 19:29:54,295 DEBUG [IPC Server handler 16 on 50902] ipc.Server (Server.java:run(2176)) - IPC Server handler 16 on 50902: starting 2015-11-22 19:29:54,295 DEBUG [IPC Server handler 17 on 50902] ipc.Server (Server.java:run(2176)) - IPC Server handler 17 on 50902: starting 2015-11-22 19:29:54,296 DEBUG [IPC Server handler 19 on 50902] ipc.Server (Server.java:run(2176)) - IPC Server handler 19 on 50902: starting 2015-11-22 19:29:54,296 DEBUG [IPC Server handler 18 on 50902] ipc.Server (Server.java:run(2176)) - IPC Server handler 18 on 50902: starting 2015-11-22 19:29:54,296 DEBUG [IPC Server handler 21 on 50902] ipc.Server (Server.java:run(2176)) - IPC Server handler 21 on 50902: starting 2015-11-22 19:29:54,296 DEBUG [IPC Server handler 20 on 50902] ipc.Server (Server.java:run(2176)) - IPC Server handler 20 on 50902: starting 2015-11-22 19:29:54,296 DEBUG [IPC Server handler 22 on 50902] ipc.Server (Server.java:run(2176)) - IPC Server handler 22 on 50902: starting 2015-11-22 19:29:54,296 DEBUG [IPC Server handler 23 on 50902] ipc.Server (Server.java:run(2176)) - IPC Server handler 23 on 50902: starting 2015-11-22 19:29:54,296 DEBUG [IPC Server handler 24 on 50902] ipc.Server (Server.java:run(2176)) - IPC Server handler 24 on 50902: starting 2015-11-22 19:29:54,296 DEBUG [IPC Server handler 25 on 50902] ipc.Server (Server.java:run(2176)) - IPC Server handler 25 on 50902: starting 2015-11-22 19:29:54,297 DEBUG [IPC Server handler 26 on 50902] ipc.Server (Server.java:run(2176)) - IPC Server handler 26 on 50902: starting 2015-11-22 19:29:54,297 DEBUG [IPC Server handler 27 on 50902] ipc.Server (Server.java:run(2176)) - IPC Server handler 27 on 50902: starting 2015-11-22 19:29:54,297 DEBUG [IPC Server handler 28 on 50902] ipc.Server (Server.java:run(2176)) - IPC Server handler 28 on 50902: starting 2015-11-22 19:29:54,297 DEBUG [IPC Server handler 29 on 50902] ipc.Server (Server.java:run(2176)) - IPC Server handler 29 on 50902: starting 2015-11-22 19:29:54,297 DEBUG [IPC Server handler 30 on 50902] ipc.Server (Server.java:run(2176)) - IPC Server handler 30 on 50902: starting 2015-11-22 19:29:54,297 DEBUG [IPC Server handler 31 on 50902] ipc.Server (Server.java:run(2176)) - IPC Server handler 31 on 50902: starting 2015-11-22 19:29:54,298 DEBUG [IPC Server handler 33 on 50902] ipc.Server (Server.java:run(2176)) - IPC Server handler 33 on 50902: starting 2015-11-22 19:29:54,298 DEBUG [IPC Server handler 34 on 50902] ipc.Server (Server.java:run(2176)) - IPC Server handler 34 on 50902: starting 2015-11-22 19:29:54,298 DEBUG [IPC Server handler 32 on 50902] ipc.Server (Server.java:run(2176)) - IPC Server handler 32 on 50902: starting 2015-11-22 19:29:54,298 DEBUG [IPC Server handler 35 on 50902] ipc.Server (Server.java:run(2176)) - IPC Server handler 35 on 50902: starting 2015-11-22 19:29:54,298 DEBUG [IPC Server handler 36 on 50902] ipc.Server (Server.java:run(2176)) - IPC Server handler 36 on 50902: starting 2015-11-22 19:29:54,298 DEBUG [IPC Server handler 37 on 50902] ipc.Server (Server.java:run(2176)) - IPC Server handler 37 on 50902: starting 2015-11-22 19:29:54,298 DEBUG [IPC Server handler 38 on 50902] ipc.Server (Server.java:run(2176)) - IPC Server handler 38 on 50902: starting 2015-11-22 19:29:54,299 DEBUG [IPC Server handler 39 on 50902] ipc.Server (Server.java:run(2176)) - IPC Server handler 39 on 50902: starting 2015-11-22 19:29:54,299 DEBUG [IPC Server handler 40 on 50902] ipc.Server (Server.java:run(2176)) - IPC Server handler 40 on 50902: starting 2015-11-22 19:29:54,299 DEBUG [IPC Server handler 41 on 50902] ipc.Server (Server.java:run(2176)) - IPC Server handler 41 on 50902: starting 2015-11-22 19:29:54,299 DEBUG [IPC Server handler 42 on 50902] ipc.Server (Server.java:run(2176)) - IPC Server handler 42 on 50902: starting 2015-11-22 19:29:54,299 DEBUG [IPC Server handler 43 on 50902] ipc.Server (Server.java:run(2176)) - IPC Server handler 43 on 50902: starting 2015-11-22 19:29:54,299 DEBUG [IPC Server handler 45 on 50902] ipc.Server (Server.java:run(2176)) - IPC Server handler 45 on 50902: starting 2015-11-22 19:29:54,300 DEBUG [IPC Server handler 46 on 50902] ipc.Server (Server.java:run(2176)) - IPC Server handler 46 on 50902: starting 2015-11-22 19:29:54,300 DEBUG [IPC Server handler 44 on 50902] ipc.Server (Server.java:run(2176)) - IPC Server handler 44 on 50902: starting 2015-11-22 19:29:54,300 DEBUG [IPC Server handler 47 on 50902] ipc.Server (Server.java:run(2176)) - IPC Server handler 47 on 50902: starting 2015-11-22 19:29:54,300 DEBUG [IPC Server handler 48 on 50902] ipc.Server (Server.java:run(2176)) - IPC Server handler 48 on 50902: starting 2015-11-22 19:29:54,300 DEBUG [RM-0] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.ResourceTrackerService is started 2015-11-22 19:29:54,300 DEBUG [IPC Server handler 49 on 50902] ipc.Server (Server.java:run(2176)) - IPC Server handler 49 on 50902: starting 2015-11-22 19:29:54,300 DEBUG [RM-0] ipc.YarnRPC (YarnRPC.java:create(59)) - Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC 2015-11-22 19:29:54,301 DEBUG [RM-0] ipc.HadoopYarnProtoRPC (HadoopYarnProtoRPC.java:getServer(62)) - Creating a HadoopYarnProtoRpc server for protocol interface org.apache.hadoop.yarn.api.ApplicationMasterProtocol with 50 handlers 2015-11-22 19:29:54,301 INFO [RM-0] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:29:54,301 DEBUG [RM-0] ipc.Server (Server.java:getAuthMethods(2447)) - Server accepts auth methods:[TOKEN] 2015-11-22 19:29:54,301 INFO [Socket Reader #1 for port 53664] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 53664 2015-11-22 19:29:54,301 DEBUG [RM-0] metrics.RpcMetrics (RpcMetrics.java:(74)) - Initialized MetricsRegistry{info=MetricsInfoImpl{name=rpc, description=rpc}, tags=[MetricsTag{info=MetricsInfoImpl{name=port, description=RPC port}, value=53664}], metrics=[]} 2015-11-22 19:29:54,302 DEBUG [RM-0] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.ipc.metrics.RpcMetrics.receivedBytes with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Number of received bytes]) 2015-11-22 19:29:54,302 DEBUG [RM-0] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.ipc.metrics.RpcMetrics.sentBytes with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Number of sent bytes]) 2015-11-22 19:29:54,302 DEBUG [RM-0] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.ipc.metrics.RpcMetrics.rpcQueueTime with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Queue time]) 2015-11-22 19:29:54,302 DEBUG [RM-0] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.ipc.metrics.RpcMetrics.rpcProcessingTime with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Processing time]) 2015-11-22 19:29:54,302 DEBUG [RM-0] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.ipc.metrics.RpcMetrics.rpcAuthenticationFailures with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Number of authentication failures]) 2015-11-22 19:29:54,302 DEBUG [RM-0] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.ipc.metrics.RpcMetrics.rpcAuthenticationSuccesses with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Number of authentication successes]) 2015-11-22 19:29:54,302 DEBUG [RM-0] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.ipc.metrics.RpcMetrics.rpcAuthorizationFailures with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Number of authorization failures]) 2015-11-22 19:29:54,302 DEBUG [RM-0] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.ipc.metrics.RpcMetrics.rpcAuthorizationSuccesses with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Number of authorization successes]) 2015-11-22 19:29:54,302 DEBUG [RM-0] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.ipc.metrics.RpcMetrics.rpcClientBackoff with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Number of client backoff requests]) 2015-11-22 19:29:54,302 DEBUG [RM-0] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.ipc.metrics.RpcMetrics.rpcSlowCalls with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Number of Slow RPC calls]) 2015-11-22 19:29:54,303 DEBUG [RM-0] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForMethod(82)) - method public int org.apache.hadoop.ipc.metrics.RpcMetrics.numOpenConnections() with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Number of open connections]) 2015-11-22 19:29:54,303 DEBUG [RM-0] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForMethod(82)) - method public int org.apache.hadoop.ipc.metrics.RpcMetrics.callQueueLength() with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Length of the call queue]) 2015-11-22 19:29:54,303 DEBUG [RM-0] impl.MetricsSystemImpl (MetricsSystemImpl.java:register(232)) - RpcActivityForPort53664, Aggregate RPC metrics 2015-11-22 19:29:54,303 DEBUG [RM-0] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'PropertiesConfiguration' for key: source.source.start_mbeans 2015-11-22 19:29:54,303 DEBUG [RM-0] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'MetricsConfig' for key: source.start_mbeans 2015-11-22 19:29:54,303 DEBUG [RM-0] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'PropertiesConfiguration' for key: *.source.start_mbeans 2015-11-22 19:29:54,303 DEBUG [RM-0] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateAttrCache(254)) - Updating attr cache... 2015-11-22 19:29:54,303 DEBUG [RM-0] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateAttrCache(268)) - Done. # tags & metrics=17 2015-11-22 19:29:54,304 DEBUG [RM-0] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateInfoCache(248)) - Updating info cache... 2015-11-22 19:29:54,304 DEBUG [RM-0] impl.MetricsSystemImpl (MBeanInfoBuilder.java:get(109)) - [javax.management.MBeanAttributeInfo[description=RPC port, name=tag.port, type=java.lang.String, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Metrics context, name=tag.Context, type=java.lang.String, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Local hostname, name=tag.Hostname, type=java.lang.String, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of received bytes, name=ReceivedBytes, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of sent bytes, name=SentBytes, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of ops for queue time, name=RpcQueueTimeNumOps, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Average time for queue time, name=RpcQueueTimeAvgTime, type=java.lang.Double, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of ops for processing time, name=RpcProcessingTimeNumOps, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Average time for processing time, name=RpcProcessingTimeAvgTime, type=java.lang.Double, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of authentication failures, name=RpcAuthenticationFailures, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of authentication successes, name=RpcAuthenticationSuccesses, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of authorization failures, name=RpcAuthorizationFailures, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of authorization successes, name=RpcAuthorizationSuccesses, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of client backoff requests, name=RpcClientBackoff, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of Slow RPC calls, name=RpcSlowCalls, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of open connections, name=NumOpenConnections, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Length of the call queue, name=CallQueueLength, type=java.lang.Integer, read-only, descriptor={}]] 2015-11-22 19:29:54,304 DEBUG [RM-0] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateInfoCache(250)) - Done 2015-11-22 19:29:54,304 DEBUG [RM-0] util.MBeans (MBeans.java:register(67)) - Registered Hadoop:service=ApplicationHistoryServer,name=RpcActivityForPort53664 2015-11-22 19:29:54,304 DEBUG [RM-0] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:startMBeans(227)) - MBean for source RpcActivityForPort53664 registered. 2015-11-22 19:29:54,304 DEBUG [RM-0] impl.MetricsSystemImpl (MetricsSystemImpl.java:registerSource(269)) - Registered source RpcActivityForPort53664 2015-11-22 19:29:54,304 DEBUG [RM-0] metrics.RpcDetailedMetrics (RpcDetailedMetrics.java:(47)) - MetricsInfoImpl{name=rpcdetailed, description=rpcdetailed} 2015-11-22 19:29:54,304 DEBUG [RM-0] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableRates org.apache.hadoop.ipc.metrics.RpcDetailedMetrics.rates with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[]) 2015-11-22 19:29:54,305 DEBUG [RM-0] impl.MetricsSystemImpl (MetricsSystemImpl.java:register(232)) - RpcDetailedActivityForPort53664, Per method RPC metrics 2015-11-22 19:29:54,305 DEBUG [RM-0] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'PropertiesConfiguration' for key: source.source.start_mbeans 2015-11-22 19:29:54,305 DEBUG [RM-0] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'MetricsConfig' for key: source.start_mbeans 2015-11-22 19:29:54,305 DEBUG [RM-0] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'PropertiesConfiguration' for key: *.source.start_mbeans 2015-11-22 19:29:54,305 DEBUG [RM-0] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateAttrCache(254)) - Updating attr cache... 2015-11-22 19:29:54,305 DEBUG [RM-0] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateAttrCache(268)) - Done. # tags & metrics=3 2015-11-22 19:29:54,305 DEBUG [RM-0] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateInfoCache(248)) - Updating info cache... 2015-11-22 19:29:54,305 DEBUG [RM-0] impl.MetricsSystemImpl (MBeanInfoBuilder.java:get(109)) - [javax.management.MBeanAttributeInfo[description=RPC port, name=tag.port, type=java.lang.String, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Metrics context, name=tag.Context, type=java.lang.String, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Local hostname, name=tag.Hostname, type=java.lang.String, read-only, descriptor={}]] 2015-11-22 19:29:54,305 DEBUG [RM-0] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateInfoCache(250)) - Done 2015-11-22 19:29:54,305 DEBUG [RM-0] util.MBeans (MBeans.java:register(67)) - Registered Hadoop:service=ApplicationHistoryServer,name=RpcDetailedActivityForPort53664 2015-11-22 19:29:54,305 DEBUG [RM-0] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:startMBeans(227)) - MBean for source RpcDetailedActivityForPort53664 registered. 2015-11-22 19:29:54,306 DEBUG [RM-0] impl.MetricsSystemImpl (MetricsSystemImpl.java:registerSource(269)) - Registered source RpcDetailedActivityForPort53664 2015-11-22 19:29:54,306 DEBUG [RM-0] ipc.Server (RPC.java:registerProtocolAndImpl(876)) - RpcKind = RPC_PROTOCOL_BUFFER Protocol Name = org.apache.hadoop.ipc.ProtocolMetaInfoPB version=1 ProtocolImpl=org.apache.hadoop.ipc.protobuf.ProtocolInfoProtos$ProtocolInfoService$2 protocolClass=org.apache.hadoop.ipc.ProtocolMetaInfoPB 2015-11-22 19:29:54,306 DEBUG [RM-0] ipc.Server (RPC.java:registerProtocolAndImpl(876)) - RpcKind = RPC_PROTOCOL_BUFFER Protocol Name = org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB version=1 ProtocolImpl=org.apache.hadoop.yarn.proto.ApplicationMasterProtocol$ApplicationMasterProtocolService$2 protocolClass=org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB 2015-11-22 19:29:54,306 INFO [RM-0] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB to the server 2015-11-22 19:29:54,306 DEBUG [RM-0] ipc.Server (RPC.java:registerProtocolAndImpl(876)) - RpcKind = RPC_PROTOCOL_BUFFER Protocol Name = org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB version=1 ProtocolImpl=org.apache.hadoop.yarn.proto.ApplicationMasterProtocol$ApplicationMasterProtocolService$2 protocolClass=org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB 2015-11-22 19:29:54,306 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:29:54,307 INFO [IPC Server listener on 53664] ipc.Server (Server.java:run(794)) - IPC Server listener on 53664: starting 2015-11-22 19:29:54,307 DEBUG [IPC Server handler 1 on 53664] ipc.Server (Server.java:run(2176)) - IPC Server handler 1 on 53664: starting 2015-11-22 19:29:54,308 DEBUG [IPC Server handler 0 on 53664] ipc.Server (Server.java:run(2176)) - IPC Server handler 0 on 53664: starting 2015-11-22 19:29:54,309 DEBUG [IPC Server handler 8 on 53664] ipc.Server (Server.java:run(2176)) - IPC Server handler 8 on 53664: starting 2015-11-22 19:29:54,309 DEBUG [IPC Server handler 9 on 53664] ipc.Server (Server.java:run(2176)) - IPC Server handler 9 on 53664: starting 2015-11-22 19:29:54,308 DEBUG [IPC Server handler 7 on 53664] ipc.Server (Server.java:run(2176)) - IPC Server handler 7 on 53664: starting 2015-11-22 19:29:54,309 DEBUG [IPC Server handler 15 on 53664] ipc.Server (Server.java:run(2176)) - IPC Server handler 15 on 53664: starting 2015-11-22 19:29:54,308 DEBUG [IPC Server handler 6 on 53664] ipc.Server (Server.java:run(2176)) - IPC Server handler 6 on 53664: starting 2015-11-22 19:29:54,308 DEBUG [IPC Server handler 5 on 53664] ipc.Server (Server.java:run(2176)) - IPC Server handler 5 on 53664: starting 2015-11-22 19:29:54,308 DEBUG [IPC Server handler 4 on 53664] ipc.Server (Server.java:run(2176)) - IPC Server handler 4 on 53664: starting 2015-11-22 19:29:54,308 DEBUG [IPC Server handler 3 on 53664] ipc.Server (Server.java:run(2176)) - IPC Server handler 3 on 53664: starting 2015-11-22 19:29:54,308 DEBUG [IPC Server handler 2 on 53664] ipc.Server (Server.java:run(2176)) - IPC Server handler 2 on 53664: starting 2015-11-22 19:29:54,310 DEBUG [IPC Server handler 11 on 53664] ipc.Server (Server.java:run(2176)) - IPC Server handler 11 on 53664: starting 2015-11-22 19:29:54,310 DEBUG [IPC Server handler 19 on 53664] ipc.Server (Server.java:run(2176)) - IPC Server handler 19 on 53664: starting 2015-11-22 19:29:54,310 DEBUG [IPC Server handler 10 on 53664] ipc.Server (Server.java:run(2176)) - IPC Server handler 10 on 53664: starting 2015-11-22 19:29:54,310 DEBUG [IPC Server handler 18 on 53664] ipc.Server (Server.java:run(2176)) - IPC Server handler 18 on 53664: starting 2015-11-22 19:29:54,310 DEBUG [IPC Server handler 17 on 53664] ipc.Server (Server.java:run(2176)) - IPC Server handler 17 on 53664: starting 2015-11-22 19:29:54,309 DEBUG [IPC Server handler 16 on 53664] ipc.Server (Server.java:run(2176)) - IPC Server handler 16 on 53664: starting 2015-11-22 19:29:54,309 DEBUG [IPC Server handler 14 on 53664] ipc.Server (Server.java:run(2176)) - IPC Server handler 14 on 53664: starting 2015-11-22 19:29:54,309 DEBUG [IPC Server handler 13 on 53664] ipc.Server (Server.java:run(2176)) - IPC Server handler 13 on 53664: starting 2015-11-22 19:29:54,309 DEBUG [IPC Server handler 12 on 53664] ipc.Server (Server.java:run(2176)) - IPC Server handler 12 on 53664: starting 2015-11-22 19:29:54,311 DEBUG [IPC Server handler 25 on 53664] ipc.Server (Server.java:run(2176)) - IPC Server handler 25 on 53664: starting 2015-11-22 19:29:54,311 DEBUG [IPC Server handler 24 on 53664] ipc.Server (Server.java:run(2176)) - IPC Server handler 24 on 53664: starting 2015-11-22 19:29:54,310 DEBUG [IPC Server handler 23 on 53664] ipc.Server (Server.java:run(2176)) - IPC Server handler 23 on 53664: starting 2015-11-22 19:29:54,310 DEBUG [IPC Server handler 22 on 53664] ipc.Server (Server.java:run(2176)) - IPC Server handler 22 on 53664: starting 2015-11-22 19:29:54,310 DEBUG [IPC Server handler 20 on 53664] ipc.Server (Server.java:run(2176)) - IPC Server handler 20 on 53664: starting 2015-11-22 19:29:54,310 DEBUG [IPC Server handler 21 on 53664] ipc.Server (Server.java:run(2176)) - IPC Server handler 21 on 53664: starting 2015-11-22 19:29:54,311 DEBUG [IPC Server handler 29 on 53664] ipc.Server (Server.java:run(2176)) - IPC Server handler 29 on 53664: starting 2015-11-22 19:29:54,311 DEBUG [IPC Server handler 27 on 53664] ipc.Server (Server.java:run(2176)) - IPC Server handler 27 on 53664: starting 2015-11-22 19:29:54,311 DEBUG [IPC Server handler 28 on 53664] ipc.Server (Server.java:run(2176)) - IPC Server handler 28 on 53664: starting 2015-11-22 19:29:54,311 DEBUG [IPC Server handler 26 on 53664] ipc.Server (Server.java:run(2176)) - IPC Server handler 26 on 53664: starting 2015-11-22 19:29:54,312 DEBUG [IPC Server handler 30 on 53664] ipc.Server (Server.java:run(2176)) - IPC Server handler 30 on 53664: starting 2015-11-22 19:29:54,312 DEBUG [IPC Server handler 32 on 53664] ipc.Server (Server.java:run(2176)) - IPC Server handler 32 on 53664: starting 2015-11-22 19:29:54,312 DEBUG [IPC Server handler 31 on 53664] ipc.Server (Server.java:run(2176)) - IPC Server handler 31 on 53664: starting 2015-11-22 19:29:54,312 DEBUG [IPC Server handler 34 on 53664] ipc.Server (Server.java:run(2176)) - IPC Server handler 34 on 53664: starting 2015-11-22 19:29:54,312 DEBUG [IPC Server handler 33 on 53664] ipc.Server (Server.java:run(2176)) - IPC Server handler 33 on 53664: starting 2015-11-22 19:29:54,312 DEBUG [IPC Server handler 36 on 53664] ipc.Server (Server.java:run(2176)) - IPC Server handler 36 on 53664: starting 2015-11-22 19:29:54,312 DEBUG [IPC Server handler 35 on 53664] ipc.Server (Server.java:run(2176)) - IPC Server handler 35 on 53664: starting 2015-11-22 19:29:54,312 DEBUG [IPC Server handler 38 on 53664] ipc.Server (Server.java:run(2176)) - IPC Server handler 38 on 53664: starting 2015-11-22 19:29:54,312 DEBUG [IPC Server handler 37 on 53664] ipc.Server (Server.java:run(2176)) - IPC Server handler 37 on 53664: starting 2015-11-22 19:29:54,313 DEBUG [IPC Server handler 41 on 53664] ipc.Server (Server.java:run(2176)) - IPC Server handler 41 on 53664: starting 2015-11-22 19:29:54,313 DEBUG [IPC Server handler 40 on 53664] ipc.Server (Server.java:run(2176)) - IPC Server handler 40 on 53664: starting 2015-11-22 19:29:54,313 DEBUG [IPC Server handler 42 on 53664] ipc.Server (Server.java:run(2176)) - IPC Server handler 42 on 53664: starting 2015-11-22 19:29:54,313 DEBUG [IPC Server handler 39 on 53664] ipc.Server (Server.java:run(2176)) - IPC Server handler 39 on 53664: starting 2015-11-22 19:29:54,313 DEBUG [IPC Server handler 44 on 53664] ipc.Server (Server.java:run(2176)) - IPC Server handler 44 on 53664: starting 2015-11-22 19:29:54,314 DEBUG [IPC Server handler 48 on 53664] ipc.Server (Server.java:run(2176)) - IPC Server handler 48 on 53664: starting 2015-11-22 19:29:54,314 DEBUG [IPC Server handler 49 on 53664] ipc.Server (Server.java:run(2176)) - IPC Server handler 49 on 53664: starting 2015-11-22 19:29:54,313 DEBUG [IPC Server handler 43 on 53664] ipc.Server (Server.java:run(2176)) - IPC Server handler 43 on 53664: starting 2015-11-22 19:29:54,313 DEBUG [IPC Server handler 47 on 53664] ipc.Server (Server.java:run(2176)) - IPC Server handler 47 on 53664: starting 2015-11-22 19:29:54,313 DEBUG [IPC Server handler 46 on 53664] ipc.Server (Server.java:run(2176)) - IPC Server handler 46 on 53664: starting 2015-11-22 19:29:54,313 DEBUG [IPC Server handler 45 on 53664] ipc.Server (Server.java:run(2176)) - IPC Server handler 45 on 53664: starting 2015-11-22 19:29:54,314 DEBUG [RM-0] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService is started 2015-11-22 19:29:54,314 DEBUG [RM-0] ipc.YarnRPC (YarnRPC.java:create(59)) - Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC 2015-11-22 19:29:54,314 DEBUG [RM-0] ipc.HadoopYarnProtoRPC (HadoopYarnProtoRPC.java:getServer(62)) - Creating a HadoopYarnProtoRpc server for protocol interface org.apache.hadoop.yarn.api.ApplicationClientProtocol with 50 handlers 2015-11-22 19:29:54,314 INFO [RM-0] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:29:54,314 DEBUG [RM-0] ipc.Server (Server.java:getAuthMethods(2440)) - TOKEN authentication enabled for secret manager 2015-11-22 19:29:54,315 DEBUG [RM-0] ipc.Server (Server.java:getAuthMethods(2447)) - Server accepts auth methods:[TOKEN, SIMPLE] 2015-11-22 19:29:54,315 INFO [Socket Reader #1 for port 38037] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 38037 2015-11-22 19:29:54,315 DEBUG [RM-0] metrics.RpcMetrics (RpcMetrics.java:(74)) - Initialized MetricsRegistry{info=MetricsInfoImpl{name=rpc, description=rpc}, tags=[MetricsTag{info=MetricsInfoImpl{name=port, description=RPC port}, value=38037}], metrics=[]} 2015-11-22 19:29:54,316 DEBUG [RM-0] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.ipc.metrics.RpcMetrics.receivedBytes with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Number of received bytes]) 2015-11-22 19:29:54,316 DEBUG [RM-0] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.ipc.metrics.RpcMetrics.sentBytes with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Number of sent bytes]) 2015-11-22 19:29:54,316 DEBUG [RM-0] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.ipc.metrics.RpcMetrics.rpcQueueTime with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Queue time]) 2015-11-22 19:29:54,316 DEBUG [RM-0] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.ipc.metrics.RpcMetrics.rpcProcessingTime with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Processing time]) 2015-11-22 19:29:54,316 DEBUG [RM-0] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.ipc.metrics.RpcMetrics.rpcAuthenticationFailures with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Number of authentication failures]) 2015-11-22 19:29:54,316 DEBUG [RM-0] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.ipc.metrics.RpcMetrics.rpcAuthenticationSuccesses with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Number of authentication successes]) 2015-11-22 19:29:54,316 DEBUG [RM-0] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.ipc.metrics.RpcMetrics.rpcAuthorizationFailures with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Number of authorization failures]) 2015-11-22 19:29:54,316 DEBUG [RM-0] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.ipc.metrics.RpcMetrics.rpcAuthorizationSuccesses with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Number of authorization successes]) 2015-11-22 19:29:54,317 DEBUG [RM-0] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.ipc.metrics.RpcMetrics.rpcClientBackoff with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Number of client backoff requests]) 2015-11-22 19:29:54,317 DEBUG [RM-0] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.ipc.metrics.RpcMetrics.rpcSlowCalls with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Number of Slow RPC calls]) 2015-11-22 19:29:54,317 DEBUG [RM-0] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForMethod(82)) - method public int org.apache.hadoop.ipc.metrics.RpcMetrics.numOpenConnections() with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Number of open connections]) 2015-11-22 19:29:54,317 DEBUG [RM-0] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForMethod(82)) - method public int org.apache.hadoop.ipc.metrics.RpcMetrics.callQueueLength() with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Length of the call queue]) 2015-11-22 19:29:54,317 DEBUG [RM-0] impl.MetricsSystemImpl (MetricsSystemImpl.java:register(232)) - RpcActivityForPort38037, Aggregate RPC metrics 2015-11-22 19:29:54,317 DEBUG [RM-0] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'PropertiesConfiguration' for key: source.source.start_mbeans 2015-11-22 19:29:54,317 DEBUG [RM-0] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'MetricsConfig' for key: source.start_mbeans 2015-11-22 19:29:54,317 DEBUG [RM-0] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'PropertiesConfiguration' for key: *.source.start_mbeans 2015-11-22 19:29:54,318 DEBUG [RM-0] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateAttrCache(254)) - Updating attr cache... 2015-11-22 19:29:54,318 DEBUG [RM-0] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateAttrCache(268)) - Done. # tags & metrics=17 2015-11-22 19:29:54,318 DEBUG [RM-0] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateInfoCache(248)) - Updating info cache... 2015-11-22 19:29:54,318 DEBUG [RM-0] impl.MetricsSystemImpl (MBeanInfoBuilder.java:get(109)) - [javax.management.MBeanAttributeInfo[description=RPC port, name=tag.port, type=java.lang.String, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Metrics context, name=tag.Context, type=java.lang.String, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Local hostname, name=tag.Hostname, type=java.lang.String, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of received bytes, name=ReceivedBytes, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of sent bytes, name=SentBytes, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of ops for queue time, name=RpcQueueTimeNumOps, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Average time for queue time, name=RpcQueueTimeAvgTime, type=java.lang.Double, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of ops for processing time, name=RpcProcessingTimeNumOps, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Average time for processing time, name=RpcProcessingTimeAvgTime, type=java.lang.Double, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of authentication failures, name=RpcAuthenticationFailures, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of authentication successes, name=RpcAuthenticationSuccesses, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of authorization failures, name=RpcAuthorizationFailures, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of authorization successes, name=RpcAuthorizationSuccesses, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of client backoff requests, name=RpcClientBackoff, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of Slow RPC calls, name=RpcSlowCalls, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of open connections, name=NumOpenConnections, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Length of the call queue, name=CallQueueLength, type=java.lang.Integer, read-only, descriptor={}]] 2015-11-22 19:29:54,318 DEBUG [RM-0] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateInfoCache(250)) - Done 2015-11-22 19:29:54,318 DEBUG [RM-0] util.MBeans (MBeans.java:register(67)) - Registered Hadoop:service=ApplicationHistoryServer,name=RpcActivityForPort38037 2015-11-22 19:29:54,318 DEBUG [RM-0] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:startMBeans(227)) - MBean for source RpcActivityForPort38037 registered. 2015-11-22 19:29:54,318 DEBUG [RM-0] impl.MetricsSystemImpl (MetricsSystemImpl.java:registerSource(269)) - Registered source RpcActivityForPort38037 2015-11-22 19:29:54,318 DEBUG [RM-0] metrics.RpcDetailedMetrics (RpcDetailedMetrics.java:(47)) - MetricsInfoImpl{name=rpcdetailed, description=rpcdetailed} 2015-11-22 19:29:54,319 DEBUG [RM-0] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableRates org.apache.hadoop.ipc.metrics.RpcDetailedMetrics.rates with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[]) 2015-11-22 19:29:54,319 DEBUG [RM-0] impl.MetricsSystemImpl (MetricsSystemImpl.java:register(232)) - RpcDetailedActivityForPort38037, Per method RPC metrics 2015-11-22 19:29:54,319 DEBUG [RM-0] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'PropertiesConfiguration' for key: source.source.start_mbeans 2015-11-22 19:29:54,319 DEBUG [RM-0] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'MetricsConfig' for key: source.start_mbeans 2015-11-22 19:29:54,319 DEBUG [RM-0] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'PropertiesConfiguration' for key: *.source.start_mbeans 2015-11-22 19:29:54,319 DEBUG [RM-0] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateAttrCache(254)) - Updating attr cache... 2015-11-22 19:29:54,319 DEBUG [RM-0] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateAttrCache(268)) - Done. # tags & metrics=3 2015-11-22 19:29:54,319 DEBUG [RM-0] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateInfoCache(248)) - Updating info cache... 2015-11-22 19:29:54,320 DEBUG [RM-0] impl.MetricsSystemImpl (MBeanInfoBuilder.java:get(109)) - [javax.management.MBeanAttributeInfo[description=RPC port, name=tag.port, type=java.lang.String, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Metrics context, name=tag.Context, type=java.lang.String, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Local hostname, name=tag.Hostname, type=java.lang.String, read-only, descriptor={}]] 2015-11-22 19:29:54,320 DEBUG [RM-0] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateInfoCache(250)) - Done 2015-11-22 19:29:54,320 DEBUG [RM-0] util.MBeans (MBeans.java:register(67)) - Registered Hadoop:service=ApplicationHistoryServer,name=RpcDetailedActivityForPort38037 2015-11-22 19:29:54,320 DEBUG [RM-0] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:startMBeans(227)) - MBean for source RpcDetailedActivityForPort38037 registered. 2015-11-22 19:29:54,320 DEBUG [RM-0] impl.MetricsSystemImpl (MetricsSystemImpl.java:registerSource(269)) - Registered source RpcDetailedActivityForPort38037 2015-11-22 19:29:54,320 DEBUG [RM-0] ipc.Server (RPC.java:registerProtocolAndImpl(876)) - RpcKind = RPC_PROTOCOL_BUFFER Protocol Name = org.apache.hadoop.ipc.ProtocolMetaInfoPB version=1 ProtocolImpl=org.apache.hadoop.ipc.protobuf.ProtocolInfoProtos$ProtocolInfoService$2 protocolClass=org.apache.hadoop.ipc.ProtocolMetaInfoPB 2015-11-22 19:29:54,321 DEBUG [RM-0] ipc.Server (RPC.java:registerProtocolAndImpl(876)) - RpcKind = RPC_PROTOCOL_BUFFER Protocol Name = org.apache.hadoop.yarn.api.ApplicationClientProtocolPB version=1 ProtocolImpl=org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2 protocolClass=org.apache.hadoop.yarn.api.ApplicationClientProtocolPB 2015-11-22 19:29:54,321 INFO [RM-0] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.api.ApplicationClientProtocolPB to the server 2015-11-22 19:29:54,321 DEBUG [RM-0] ipc.Server (RPC.java:registerProtocolAndImpl(876)) - RpcKind = RPC_PROTOCOL_BUFFER Protocol Name = org.apache.hadoop.yarn.api.ApplicationClientProtocolPB version=1 ProtocolImpl=org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2 protocolClass=org.apache.hadoop.yarn.api.ApplicationClientProtocolPB 2015-11-22 19:29:54,321 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:29:54,321 DEBUG [IPC Server handler 0 on 38037] ipc.Server (Server.java:run(2176)) - IPC Server handler 0 on 38037: starting 2015-11-22 19:29:54,321 INFO [IPC Server listener on 38037] ipc.Server (Server.java:run(794)) - IPC Server listener on 38037: starting 2015-11-22 19:29:54,322 DEBUG [IPC Server handler 2 on 38037] ipc.Server (Server.java:run(2176)) - IPC Server handler 2 on 38037: starting 2015-11-22 19:29:54,322 DEBUG [IPC Server handler 3 on 38037] ipc.Server (Server.java:run(2176)) - IPC Server handler 3 on 38037: starting 2015-11-22 19:29:54,322 DEBUG [IPC Server handler 1 on 38037] ipc.Server (Server.java:run(2176)) - IPC Server handler 1 on 38037: starting 2015-11-22 19:29:54,322 DEBUG [IPC Server handler 4 on 38037] ipc.Server (Server.java:run(2176)) - IPC Server handler 4 on 38037: starting 2015-11-22 19:29:54,323 DEBUG [IPC Server handler 5 on 38037] ipc.Server (Server.java:run(2176)) - IPC Server handler 5 on 38037: starting 2015-11-22 19:29:54,323 DEBUG [IPC Server handler 6 on 38037] ipc.Server (Server.java:run(2176)) - IPC Server handler 6 on 38037: starting 2015-11-22 19:29:54,323 DEBUG [IPC Server handler 7 on 38037] ipc.Server (Server.java:run(2176)) - IPC Server handler 7 on 38037: starting 2015-11-22 19:29:54,323 DEBUG [IPC Server handler 8 on 38037] ipc.Server (Server.java:run(2176)) - IPC Server handler 8 on 38037: starting 2015-11-22 19:29:54,323 DEBUG [IPC Server handler 10 on 38037] ipc.Server (Server.java:run(2176)) - IPC Server handler 10 on 38037: starting 2015-11-22 19:29:54,323 DEBUG [IPC Server handler 11 on 38037] ipc.Server (Server.java:run(2176)) - IPC Server handler 11 on 38037: starting 2015-11-22 19:29:54,323 DEBUG [IPC Server handler 12 on 38037] ipc.Server (Server.java:run(2176)) - IPC Server handler 12 on 38037: starting 2015-11-22 19:29:54,323 DEBUG [IPC Server handler 9 on 38037] ipc.Server (Server.java:run(2176)) - IPC Server handler 9 on 38037: starting 2015-11-22 19:29:54,324 DEBUG [IPC Server handler 13 on 38037] ipc.Server (Server.java:run(2176)) - IPC Server handler 13 on 38037: starting 2015-11-22 19:29:54,324 DEBUG [IPC Server handler 15 on 38037] ipc.Server (Server.java:run(2176)) - IPC Server handler 15 on 38037: starting 2015-11-22 19:29:54,324 DEBUG [IPC Server handler 14 on 38037] ipc.Server (Server.java:run(2176)) - IPC Server handler 14 on 38037: starting 2015-11-22 19:29:54,324 DEBUG [IPC Server handler 16 on 38037] ipc.Server (Server.java:run(2176)) - IPC Server handler 16 on 38037: starting 2015-11-22 19:29:54,324 DEBUG [IPC Server handler 17 on 38037] ipc.Server (Server.java:run(2176)) - IPC Server handler 17 on 38037: starting 2015-11-22 19:29:54,324 DEBUG [IPC Server handler 18 on 38037] ipc.Server (Server.java:run(2176)) - IPC Server handler 18 on 38037: starting 2015-11-22 19:29:54,324 DEBUG [IPC Server handler 19 on 38037] ipc.Server (Server.java:run(2176)) - IPC Server handler 19 on 38037: starting 2015-11-22 19:29:54,325 DEBUG [IPC Server handler 20 on 38037] ipc.Server (Server.java:run(2176)) - IPC Server handler 20 on 38037: starting 2015-11-22 19:29:54,325 DEBUG [IPC Server handler 22 on 38037] ipc.Server (Server.java:run(2176)) - IPC Server handler 22 on 38037: starting 2015-11-22 19:29:54,325 DEBUG [IPC Server handler 21 on 38037] ipc.Server (Server.java:run(2176)) - IPC Server handler 21 on 38037: starting 2015-11-22 19:29:54,326 DEBUG [IPC Server handler 24 on 38037] ipc.Server (Server.java:run(2176)) - IPC Server handler 24 on 38037: starting 2015-11-22 19:29:54,326 DEBUG [IPC Server handler 23 on 38037] ipc.Server (Server.java:run(2176)) - IPC Server handler 23 on 38037: starting 2015-11-22 19:29:54,326 DEBUG [IPC Server handler 25 on 38037] ipc.Server (Server.java:run(2176)) - IPC Server handler 25 on 38037: starting 2015-11-22 19:29:54,326 DEBUG [IPC Server handler 26 on 38037] ipc.Server (Server.java:run(2176)) - IPC Server handler 26 on 38037: starting 2015-11-22 19:29:54,326 DEBUG [IPC Server handler 27 on 38037] ipc.Server (Server.java:run(2176)) - IPC Server handler 27 on 38037: starting 2015-11-22 19:29:54,326 DEBUG [IPC Server handler 28 on 38037] ipc.Server (Server.java:run(2176)) - IPC Server handler 28 on 38037: starting 2015-11-22 19:29:54,326 DEBUG [IPC Server handler 29 on 38037] ipc.Server (Server.java:run(2176)) - IPC Server handler 29 on 38037: starting 2015-11-22 19:29:54,326 DEBUG [IPC Server handler 30 on 38037] ipc.Server (Server.java:run(2176)) - IPC Server handler 30 on 38037: starting 2015-11-22 19:29:54,326 DEBUG [IPC Server handler 31 on 38037] ipc.Server (Server.java:run(2176)) - IPC Server handler 31 on 38037: starting 2015-11-22 19:29:54,327 DEBUG [IPC Server handler 32 on 38037] ipc.Server (Server.java:run(2176)) - IPC Server handler 32 on 38037: starting 2015-11-22 19:29:54,327 DEBUG [IPC Server handler 33 on 38037] ipc.Server (Server.java:run(2176)) - IPC Server handler 33 on 38037: starting 2015-11-22 19:29:54,327 DEBUG [IPC Server handler 34 on 38037] ipc.Server (Server.java:run(2176)) - IPC Server handler 34 on 38037: starting 2015-11-22 19:29:54,327 DEBUG [IPC Server handler 35 on 38037] ipc.Server (Server.java:run(2176)) - IPC Server handler 35 on 38037: starting 2015-11-22 19:29:54,327 DEBUG [IPC Server handler 36 on 38037] ipc.Server (Server.java:run(2176)) - IPC Server handler 36 on 38037: starting 2015-11-22 19:29:54,327 DEBUG [IPC Server handler 37 on 38037] ipc.Server (Server.java:run(2176)) - IPC Server handler 37 on 38037: starting 2015-11-22 19:29:54,327 DEBUG [IPC Server handler 38 on 38037] ipc.Server (Server.java:run(2176)) - IPC Server handler 38 on 38037: starting 2015-11-22 19:29:54,327 DEBUG [IPC Server handler 39 on 38037] ipc.Server (Server.java:run(2176)) - IPC Server handler 39 on 38037: starting 2015-11-22 19:29:54,327 DEBUG [IPC Server handler 40 on 38037] ipc.Server (Server.java:run(2176)) - IPC Server handler 40 on 38037: starting 2015-11-22 19:29:54,328 DEBUG [IPC Server handler 41 on 38037] ipc.Server (Server.java:run(2176)) - IPC Server handler 41 on 38037: starting 2015-11-22 19:29:54,328 DEBUG [IPC Server handler 42 on 38037] ipc.Server (Server.java:run(2176)) - IPC Server handler 42 on 38037: starting 2015-11-22 19:29:54,328 DEBUG [IPC Server handler 43 on 38037] ipc.Server (Server.java:run(2176)) - IPC Server handler 43 on 38037: starting 2015-11-22 19:29:54,328 DEBUG [IPC Server handler 44 on 38037] ipc.Server (Server.java:run(2176)) - IPC Server handler 44 on 38037: starting 2015-11-22 19:29:54,328 DEBUG [IPC Server handler 45 on 38037] ipc.Server (Server.java:run(2176)) - IPC Server handler 45 on 38037: starting 2015-11-22 19:29:54,328 DEBUG [IPC Server handler 46 on 38037] ipc.Server (Server.java:run(2176)) - IPC Server handler 46 on 38037: starting 2015-11-22 19:29:54,328 DEBUG [IPC Server handler 47 on 38037] ipc.Server (Server.java:run(2176)) - IPC Server handler 47 on 38037: starting 2015-11-22 19:29:54,328 DEBUG [IPC Server handler 48 on 38037] ipc.Server (Server.java:run(2176)) - IPC Server handler 48 on 38037: starting 2015-11-22 19:29:54,328 DEBUG [IPC Server handler 49 on 38037] ipc.Server (Server.java:run(2176)) - IPC Server handler 49 on 38037: starting 2015-11-22 19:29:54,328 DEBUG [RM-0] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.ClientRMService is started 2015-11-22 19:29:54,329 DEBUG [RM-0] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher is started 2015-11-22 19:29:54,329 DEBUG [RM-0] service.AbstractService (AbstractService.java:start(197)) - Service RMActiveServices is started 2015-11-22 19:29:54,329 INFO [RM-0] resourcemanager.ResourceManager (ResourceManager.java:transitionToActive(1040)) - Transitioned to active state 2015-11-22 19:29:54,329 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap=null 2015-11-22 19:29:54,329 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=null 2015-11-22 19:29:54,329 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:54,329 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/=org.mortbay.jetty.servlet.DefaultServlet-929987349} 2015-11-22 19:29:54,330 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-929987349=org.mortbay.jetty.servlet.DefaultServlet-929987349} 2015-11-22 19:29:54,330 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={NoCacheFilter=NoCacheFilter} 2015-11-22 19:29:54,330 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15)] 2015-11-22 19:29:54,330 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:54,330 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/=org.mortbay.jetty.servlet.DefaultServlet-929987349} 2015-11-22 19:29:54,330 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-929987349=org.mortbay.jetty.servlet.DefaultServlet-929987349} 2015-11-22 19:29:54,330 INFO [RM-0] server.AuthenticationFilter (AuthenticationFilter.java:constructSecretProvider(304)) - Unable to initialize FileSignerSecretProvider, falling back to use random secrets. 2015-11-22 19:29:54,331 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container Server@7ff1e948 + org.mortbay.thread.QueuedThreadPool@7e23a01a as threadpool 2015-11-22 19:29:54,331 INFO [RM-0] http.HttpRequestLog (HttpRequestLog.java:getRequestLog(80)) - Http request log for http.requests.resourcemanager is not defined 2015-11-22 19:29:54,331 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container Server@7ff1e948 + ContextHandlerCollection@3b039370 as handler 2015-11-22 19:29:54,332 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ContextHandlerCollection@3b039370 + org.mortbay.jetty.webapp.WebAppContext@5b1e5299{/,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes/webapps/cluster} as handler 2015-11-22 19:29:54,332 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 + NoCacheFilter as filter 2015-11-22 19:29:54,332 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 + (F=NoCacheFilter,[/*],[],15) as filterMapping 2015-11-22 19:29:54,332 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 + org.mortbay.jetty.servlet.DefaultServlet-929987349 as servlet 2015-11-22 19:29:54,332 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 + (S=org.mortbay.jetty.servlet.DefaultServlet-929987349,[/]) as servletMapping 2015-11-22 19:29:54,332 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container SecurityHandler@48150f6a + ServletHandler@38de1a90 as handler 2015-11-22 19:29:54,332 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container SessionHandler@6f2a8d0b + SecurityHandler@48150f6a as handler 2015-11-22 19:29:54,333 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container SessionHandler@6f2a8d0b + org.mortbay.jetty.servlet.HashSessionManager@468980ed as sessionManager 2015-11-22 19:29:54,333 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container org.mortbay.jetty.webapp.WebAppContext@5b1e5299{/,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes/webapps/cluster} + SessionHandler@6f2a8d0b as handler 2015-11-22 19:29:54,333 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container org.mortbay.jetty.webapp.WebAppContext@5b1e5299{/,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes/webapps/cluster} + ErrorPageErrorHandler@e4b0836 as error 2015-11-22 19:29:54,333 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ContextHandlerCollection@3b039370 + org.mortbay.jetty.servlet.Context@6ec538ca{/logs,null} as handler 2015-11-22 19:29:54,333 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container org.mortbay.jetty.servlet.Context@6ec538ca{/logs,null} + ServletHandler@553cb2cc as handler 2015-11-22 19:29:54,333 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@553cb2cc + org.apache.hadoop.http.AdminAuthorizedServlet-1597562478 as servlet 2015-11-22 19:29:54,334 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@553cb2cc + (S=org.apache.hadoop.http.AdminAuthorizedServlet-1597562478,[/*]) as servletMapping 2015-11-22 19:29:54,334 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap=null 2015-11-22 19:29:54,334 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=null 2015-11-22 19:29:54,334 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:54,334 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/*=org.apache.hadoop.http.AdminAuthorizedServlet-1597562478} 2015-11-22 19:29:54,334 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.apache.hadoop.http.AdminAuthorizedServlet-1597562478=org.apache.hadoop.http.AdminAuthorizedServlet-1597562478} 2015-11-22 19:29:54,334 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 + NoCacheFilter as filter 2015-11-22 19:29:54,334 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 + (F=NoCacheFilter,[/*],[],15) as filterMapping 2015-11-22 19:29:54,334 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={NoCacheFilter=NoCacheFilter} 2015-11-22 19:29:54,334 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15)] 2015-11-22 19:29:54,334 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:54,334 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/=org.mortbay.jetty.servlet.DefaultServlet-929987349} 2015-11-22 19:29:54,334 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-929987349=org.mortbay.jetty.servlet.DefaultServlet-929987349} 2015-11-22 19:29:54,335 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ContextHandlerCollection@3b039370 + org.mortbay.jetty.servlet.Context@9647707{/static,null} as handler 2015-11-22 19:29:54,335 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container org.mortbay.jetty.servlet.Context@9647707{/static,null} + ServletHandler@4ea00d89 as handler 2015-11-22 19:29:54,335 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@4ea00d89 + org.mortbay.jetty.servlet.DefaultServlet-2126150723 as servlet 2015-11-22 19:29:54,335 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@4ea00d89 + (S=org.mortbay.jetty.servlet.DefaultServlet-2126150723,[/*]) as servletMapping 2015-11-22 19:29:54,335 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap=null 2015-11-22 19:29:54,335 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=null 2015-11-22 19:29:54,335 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:54,335 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/*=org.mortbay.jetty.servlet.DefaultServlet-2126150723} 2015-11-22 19:29:54,335 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-2126150723=org.mortbay.jetty.servlet.DefaultServlet-2126150723} 2015-11-22 19:29:54,335 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 + safety as filter 2015-11-22 19:29:54,335 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 + (F=safety,[/*],[],15) as filterMapping 2015-11-22 19:29:54,335 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={safety=safety, NoCacheFilter=NoCacheFilter} 2015-11-22 19:29:54,335 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15)] 2015-11-22 19:29:54,336 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:54,336 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/=org.mortbay.jetty.servlet.DefaultServlet-929987349} 2015-11-22 19:29:54,336 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-929987349=org.mortbay.jetty.servlet.DefaultServlet-929987349} 2015-11-22 19:29:54,336 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@4ea00d89 + safety as filter 2015-11-22 19:29:54,336 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@4ea00d89 + (F=safety,[/*],[],15) as filterMapping 2015-11-22 19:29:54,336 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={safety=safety} 2015-11-22 19:29:54,336 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=safety,[/*],[],15)] 2015-11-22 19:29:54,336 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:54,336 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/*=org.mortbay.jetty.servlet.DefaultServlet-2126150723} 2015-11-22 19:29:54,336 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-2126150723=org.mortbay.jetty.servlet.DefaultServlet-2126150723} 2015-11-22 19:29:54,336 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@553cb2cc + safety as filter 2015-11-22 19:29:54,336 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@553cb2cc + (F=safety,[/*],[],15) as filterMapping 2015-11-22 19:29:54,336 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={safety=safety} 2015-11-22 19:29:54,336 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=safety,[/*],[],15)] 2015-11-22 19:29:54,336 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:54,337 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/*=org.apache.hadoop.http.AdminAuthorizedServlet-1597562478} 2015-11-22 19:29:54,337 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.apache.hadoop.http.AdminAuthorizedServlet-1597562478=org.apache.hadoop.http.AdminAuthorizedServlet-1597562478} 2015-11-22 19:29:54,337 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter) 2015-11-22 19:29:54,337 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 + RMAuthenticationFilter as filter 2015-11-22 19:29:54,337 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 + (F=RMAuthenticationFilter,[*.html, *.jsp],[],15) as filterMapping 2015-11-22 19:29:54,337 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter} 2015-11-22 19:29:54,337 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15)] 2015-11-22 19:29:54,337 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:54,337 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/=org.mortbay.jetty.servlet.DefaultServlet-929987349} 2015-11-22 19:29:54,338 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-929987349=org.mortbay.jetty.servlet.DefaultServlet-929987349} 2015-11-22 19:29:54,338 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context cluster 2015-11-22 19:29:54,338 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@4ea00d89 + RMAuthenticationFilter as filter 2015-11-22 19:29:54,338 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@4ea00d89 + (F=RMAuthenticationFilter,[/*],[],15) as filterMapping 2015-11-22 19:29:54,338 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={safety=safety, RMAuthenticationFilter=RMAuthenticationFilter} 2015-11-22 19:29:54,338 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=safety,[/*],[],15), (F=RMAuthenticationFilter,[/*],[],15)] 2015-11-22 19:29:54,338 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:54,338 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/*=org.mortbay.jetty.servlet.DefaultServlet-2126150723} 2015-11-22 19:29:54,338 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-2126150723=org.mortbay.jetty.servlet.DefaultServlet-2126150723} 2015-11-22 19:29:54,338 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context static 2015-11-22 19:29:54,338 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@553cb2cc + RMAuthenticationFilter as filter 2015-11-22 19:29:54,338 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@553cb2cc + (F=RMAuthenticationFilter,[/*],[],15) as filterMapping 2015-11-22 19:29:54,338 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={safety=safety, RMAuthenticationFilter=RMAuthenticationFilter} 2015-11-22 19:29:54,338 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=safety,[/*],[],15), (F=RMAuthenticationFilter,[/*],[],15)] 2015-11-22 19:29:54,339 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:54,339 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/*=org.apache.hadoop.http.AdminAuthorizedServlet-1597562478} 2015-11-22 19:29:54,339 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.apache.hadoop.http.AdminAuthorizedServlet-1597562478=org.apache.hadoop.http.AdminAuthorizedServlet-1597562478} 2015-11-22 19:29:54,339 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context logs 2015-11-22 19:29:54,339 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 + static_user_filter as filter 2015-11-22 19:29:54,339 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 + (F=static_user_filter,[*.html, *.jsp],[],15) as filterMapping 2015-11-22 19:29:54,339 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:54,339 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15)] 2015-11-22 19:29:54,339 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:54,339 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/=org.mortbay.jetty.servlet.DefaultServlet-929987349} 2015-11-22 19:29:54,339 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-929987349=org.mortbay.jetty.servlet.DefaultServlet-929987349} 2015-11-22 19:29:54,339 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context cluster 2015-11-22 19:29:54,339 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@4ea00d89 + static_user_filter as filter 2015-11-22 19:29:54,339 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@4ea00d89 + (F=static_user_filter,[/*],[],15) as filterMapping 2015-11-22 19:29:54,339 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:54,339 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=safety,[/*],[],15), (F=RMAuthenticationFilter,[/*],[],15), (F=static_user_filter,[/*],[],15)] 2015-11-22 19:29:54,340 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:54,340 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/*=org.mortbay.jetty.servlet.DefaultServlet-2126150723} 2015-11-22 19:29:54,340 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-2126150723=org.mortbay.jetty.servlet.DefaultServlet-2126150723} 2015-11-22 19:29:54,340 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2015-11-22 19:29:54,340 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@553cb2cc + static_user_filter as filter 2015-11-22 19:29:54,340 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@553cb2cc + (F=static_user_filter,[/*],[],15) as filterMapping 2015-11-22 19:29:54,340 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:54,340 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=safety,[/*],[],15), (F=RMAuthenticationFilter,[/*],[],15), (F=static_user_filter,[/*],[],15)] 2015-11-22 19:29:54,340 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:54,340 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/*=org.apache.hadoop.http.AdminAuthorizedServlet-1597562478} 2015-11-22 19:29:54,340 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.apache.hadoop.http.AdminAuthorizedServlet-1597562478=org.apache.hadoop.http.AdminAuthorizedServlet-1597562478} 2015-11-22 19:29:54,340 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2015-11-22 19:29:54,341 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 + Timeline Authentication Filter as filter 2015-11-22 19:29:54,341 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 + (F=Timeline Authentication Filter,[/*],[],15) as filterMapping 2015-11-22 19:29:54,341 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:54,341 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15)] 2015-11-22 19:29:54,341 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:54,341 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/=org.mortbay.jetty.servlet.DefaultServlet-929987349} 2015-11-22 19:29:54,341 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-929987349=org.mortbay.jetty.servlet.DefaultServlet-929987349} 2015-11-22 19:29:54,341 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@4ea00d89 + Timeline Authentication Filter as filter 2015-11-22 19:29:54,341 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@4ea00d89 + (F=Timeline Authentication Filter,[/*],[],15) as filterMapping 2015-11-22 19:29:54,341 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:54,341 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=safety,[/*],[],15), (F=RMAuthenticationFilter,[/*],[],15), (F=static_user_filter,[/*],[],15), (F=Timeline Authentication Filter,[/*],[],15)] 2015-11-22 19:29:54,341 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:54,342 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/*=org.mortbay.jetty.servlet.DefaultServlet-2126150723} 2015-11-22 19:29:54,342 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-2126150723=org.mortbay.jetty.servlet.DefaultServlet-2126150723} 2015-11-22 19:29:54,342 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@553cb2cc + Timeline Authentication Filter as filter 2015-11-22 19:29:54,342 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@553cb2cc + (F=Timeline Authentication Filter,[/*],[],15) as filterMapping 2015-11-22 19:29:54,342 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:54,342 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=safety,[/*],[],15), (F=RMAuthenticationFilter,[/*],[],15), (F=static_user_filter,[/*],[],15), (F=Timeline Authentication Filter,[/*],[],15)] 2015-11-22 19:29:54,342 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:54,342 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/*=org.apache.hadoop.http.AdminAuthorizedServlet-1597562478} 2015-11-22 19:29:54,342 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.apache.hadoop.http.AdminAuthorizedServlet-1597562478=org.apache.hadoop.http.AdminAuthorizedServlet-1597562478} 2015-11-22 19:29:54,342 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'Timeline Authentication Filter' (class=org.apache.hadoop.yarn.server.timeline.security.TimelineAuthenticationFilter) 2015-11-22 19:29:54,342 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 + stacks as servlet 2015-11-22 19:29:54,342 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 + (S=stacks,[/stacks]) as servletMapping 2015-11-22 19:29:54,342 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:54,342 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15)] 2015-11-22 19:29:54,343 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:54,343 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/stacks=stacks, /=org.mortbay.jetty.servlet.DefaultServlet-929987349} 2015-11-22 19:29:54,343 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-929987349=org.mortbay.jetty.servlet.DefaultServlet-929987349, stacks=stacks} 2015-11-22 19:29:54,343 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 + (F=RMAuthenticationFilter,[/stacks],[],15) as filterMapping 2015-11-22 19:29:54,343 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:54,343 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15), (F=RMAuthenticationFilter,[/stacks],[],15)] 2015-11-22 19:29:54,343 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:54,343 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/stacks=stacks, /=org.mortbay.jetty.servlet.DefaultServlet-929987349} 2015-11-22 19:29:54,343 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-929987349=org.mortbay.jetty.servlet.DefaultServlet-929987349, stacks=stacks} 2015-11-22 19:29:54,343 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 + (F=static_user_filter,[/stacks],[],15) as filterMapping 2015-11-22 19:29:54,343 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:54,343 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15), (F=RMAuthenticationFilter,[/stacks],[],15), (F=static_user_filter,[/stacks],[],15)] 2015-11-22 19:29:54,343 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:54,344 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/stacks=stacks, /=org.mortbay.jetty.servlet.DefaultServlet-929987349} 2015-11-22 19:29:54,344 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-929987349=org.mortbay.jetty.servlet.DefaultServlet-929987349, stacks=stacks} 2015-11-22 19:29:54,344 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 + logLevel as servlet 2015-11-22 19:29:54,344 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 + (S=logLevel,[/logLevel]) as servletMapping 2015-11-22 19:29:54,344 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:54,344 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15), (F=RMAuthenticationFilter,[/stacks],[],15), (F=static_user_filter,[/stacks],[],15)] 2015-11-22 19:29:54,344 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:54,344 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/stacks=stacks, /logLevel=logLevel, /=org.mortbay.jetty.servlet.DefaultServlet-929987349} 2015-11-22 19:29:54,344 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={logLevel=logLevel, org.mortbay.jetty.servlet.DefaultServlet-929987349=org.mortbay.jetty.servlet.DefaultServlet-929987349, stacks=stacks} 2015-11-22 19:29:54,344 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 + (F=RMAuthenticationFilter,[/logLevel],[],15) as filterMapping 2015-11-22 19:29:54,344 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:54,344 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15), (F=RMAuthenticationFilter,[/stacks],[],15), (F=static_user_filter,[/stacks],[],15), (F=RMAuthenticationFilter,[/logLevel],[],15)] 2015-11-22 19:29:54,344 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:54,344 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/stacks=stacks, /logLevel=logLevel, /=org.mortbay.jetty.servlet.DefaultServlet-929987349} 2015-11-22 19:29:54,345 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={logLevel=logLevel, org.mortbay.jetty.servlet.DefaultServlet-929987349=org.mortbay.jetty.servlet.DefaultServlet-929987349, stacks=stacks} 2015-11-22 19:29:54,345 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 + (F=static_user_filter,[/logLevel],[],15) as filterMapping 2015-11-22 19:29:54,345 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:54,345 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15), (F=RMAuthenticationFilter,[/stacks],[],15), (F=static_user_filter,[/stacks],[],15), (F=RMAuthenticationFilter,[/logLevel],[],15), (F=static_user_filter,[/logLevel],[],15)] 2015-11-22 19:29:54,345 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:54,345 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/stacks=stacks, /logLevel=logLevel, /=org.mortbay.jetty.servlet.DefaultServlet-929987349} 2015-11-22 19:29:54,345 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={logLevel=logLevel, org.mortbay.jetty.servlet.DefaultServlet-929987349=org.mortbay.jetty.servlet.DefaultServlet-929987349, stacks=stacks} 2015-11-22 19:29:54,345 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 + metrics as servlet 2015-11-22 19:29:54,345 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 + (S=metrics,[/metrics]) as servletMapping 2015-11-22 19:29:54,345 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:54,345 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15), (F=RMAuthenticationFilter,[/stacks],[],15), (F=static_user_filter,[/stacks],[],15), (F=RMAuthenticationFilter,[/logLevel],[],15), (F=static_user_filter,[/logLevel],[],15)] 2015-11-22 19:29:54,345 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:54,345 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/stacks=stacks, /logLevel=logLevel, /metrics=metrics, /=org.mortbay.jetty.servlet.DefaultServlet-929987349} 2015-11-22 19:29:54,346 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={logLevel=logLevel, org.mortbay.jetty.servlet.DefaultServlet-929987349=org.mortbay.jetty.servlet.DefaultServlet-929987349, stacks=stacks, metrics=metrics} 2015-11-22 19:29:54,346 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 + (F=RMAuthenticationFilter,[/metrics],[],15) as filterMapping 2015-11-22 19:29:54,346 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:54,346 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15), (F=RMAuthenticationFilter,[/stacks],[],15), (F=static_user_filter,[/stacks],[],15), (F=RMAuthenticationFilter,[/logLevel],[],15), (F=static_user_filter,[/logLevel],[],15), (F=RMAuthenticationFilter,[/metrics],[],15)] 2015-11-22 19:29:54,346 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:54,346 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/stacks=stacks, /logLevel=logLevel, /metrics=metrics, /=org.mortbay.jetty.servlet.DefaultServlet-929987349} 2015-11-22 19:29:54,346 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={logLevel=logLevel, org.mortbay.jetty.servlet.DefaultServlet-929987349=org.mortbay.jetty.servlet.DefaultServlet-929987349, stacks=stacks, metrics=metrics} 2015-11-22 19:29:54,346 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 + (F=static_user_filter,[/metrics],[],15) as filterMapping 2015-11-22 19:29:54,346 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:54,346 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15), (F=RMAuthenticationFilter,[/stacks],[],15), (F=static_user_filter,[/stacks],[],15), (F=RMAuthenticationFilter,[/logLevel],[],15), (F=static_user_filter,[/logLevel],[],15), (F=RMAuthenticationFilter,[/metrics],[],15), (F=static_user_filter,[/metrics],[],15)] 2015-11-22 19:29:54,346 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:54,346 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/stacks=stacks, /logLevel=logLevel, /metrics=metrics, /=org.mortbay.jetty.servlet.DefaultServlet-929987349} 2015-11-22 19:29:54,346 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={logLevel=logLevel, org.mortbay.jetty.servlet.DefaultServlet-929987349=org.mortbay.jetty.servlet.DefaultServlet-929987349, stacks=stacks, metrics=metrics} 2015-11-22 19:29:54,346 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 + jmx as servlet 2015-11-22 19:29:54,347 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 + (S=jmx,[/jmx]) as servletMapping 2015-11-22 19:29:54,347 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:54,347 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15), (F=RMAuthenticationFilter,[/stacks],[],15), (F=static_user_filter,[/stacks],[],15), (F=RMAuthenticationFilter,[/logLevel],[],15), (F=static_user_filter,[/logLevel],[],15), (F=RMAuthenticationFilter,[/metrics],[],15), (F=static_user_filter,[/metrics],[],15)] 2015-11-22 19:29:54,347 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:54,347 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/jmx=jmx, /stacks=stacks, /logLevel=logLevel, /metrics=metrics, /=org.mortbay.jetty.servlet.DefaultServlet-929987349} 2015-11-22 19:29:54,347 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={logLevel=logLevel, jmx=jmx, org.mortbay.jetty.servlet.DefaultServlet-929987349=org.mortbay.jetty.servlet.DefaultServlet-929987349, stacks=stacks, metrics=metrics} 2015-11-22 19:29:54,347 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 + (F=RMAuthenticationFilter,[/jmx],[],15) as filterMapping 2015-11-22 19:29:54,347 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:54,347 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15), (F=RMAuthenticationFilter,[/stacks],[],15), (F=static_user_filter,[/stacks],[],15), (F=RMAuthenticationFilter,[/logLevel],[],15), (F=static_user_filter,[/logLevel],[],15), (F=RMAuthenticationFilter,[/metrics],[],15), (F=static_user_filter,[/metrics],[],15), (F=RMAuthenticationFilter,[/jmx],[],15)] 2015-11-22 19:29:54,347 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:54,347 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/jmx=jmx, /stacks=stacks, /logLevel=logLevel, /metrics=metrics, /=org.mortbay.jetty.servlet.DefaultServlet-929987349} 2015-11-22 19:29:54,347 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={logLevel=logLevel, jmx=jmx, org.mortbay.jetty.servlet.DefaultServlet-929987349=org.mortbay.jetty.servlet.DefaultServlet-929987349, stacks=stacks, metrics=metrics} 2015-11-22 19:29:54,347 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 + (F=static_user_filter,[/jmx],[],15) as filterMapping 2015-11-22 19:29:54,347 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:54,348 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15), (F=RMAuthenticationFilter,[/stacks],[],15), (F=static_user_filter,[/stacks],[],15), (F=RMAuthenticationFilter,[/logLevel],[],15), (F=static_user_filter,[/logLevel],[],15), (F=RMAuthenticationFilter,[/metrics],[],15), (F=static_user_filter,[/metrics],[],15), (F=RMAuthenticationFilter,[/jmx],[],15), (F=static_user_filter,[/jmx],[],15)] 2015-11-22 19:29:54,348 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:54,348 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/jmx=jmx, /stacks=stacks, /logLevel=logLevel, /metrics=metrics, /=org.mortbay.jetty.servlet.DefaultServlet-929987349} 2015-11-22 19:29:54,348 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={logLevel=logLevel, jmx=jmx, org.mortbay.jetty.servlet.DefaultServlet-929987349=org.mortbay.jetty.servlet.DefaultServlet-929987349, stacks=stacks, metrics=metrics} 2015-11-22 19:29:54,348 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 + conf as servlet 2015-11-22 19:29:54,348 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 + (S=conf,[/conf]) as servletMapping 2015-11-22 19:29:54,348 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:54,348 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15), (F=RMAuthenticationFilter,[/stacks],[],15), (F=static_user_filter,[/stacks],[],15), (F=RMAuthenticationFilter,[/logLevel],[],15), (F=static_user_filter,[/logLevel],[],15), (F=RMAuthenticationFilter,[/metrics],[],15), (F=static_user_filter,[/metrics],[],15), (F=RMAuthenticationFilter,[/jmx],[],15), (F=static_user_filter,[/jmx],[],15)] 2015-11-22 19:29:54,348 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:54,348 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/jmx=jmx, /conf=conf, /stacks=stacks, /logLevel=logLevel, /metrics=metrics, /=org.mortbay.jetty.servlet.DefaultServlet-929987349} 2015-11-22 19:29:54,348 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={logLevel=logLevel, jmx=jmx, org.mortbay.jetty.servlet.DefaultServlet-929987349=org.mortbay.jetty.servlet.DefaultServlet-929987349, stacks=stacks, conf=conf, metrics=metrics} 2015-11-22 19:29:54,349 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 + (F=RMAuthenticationFilter,[/conf],[],15) as filterMapping 2015-11-22 19:29:54,349 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:54,349 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15), (F=RMAuthenticationFilter,[/stacks],[],15), (F=static_user_filter,[/stacks],[],15), (F=RMAuthenticationFilter,[/logLevel],[],15), (F=static_user_filter,[/logLevel],[],15), (F=RMAuthenticationFilter,[/metrics],[],15), (F=static_user_filter,[/metrics],[],15), (F=RMAuthenticationFilter,[/jmx],[],15), (F=static_user_filter,[/jmx],[],15), (F=RMAuthenticationFilter,[/conf],[],15)] 2015-11-22 19:29:54,349 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:54,349 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/jmx=jmx, /conf=conf, /stacks=stacks, /logLevel=logLevel, /metrics=metrics, /=org.mortbay.jetty.servlet.DefaultServlet-929987349} 2015-11-22 19:29:54,349 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={logLevel=logLevel, jmx=jmx, org.mortbay.jetty.servlet.DefaultServlet-929987349=org.mortbay.jetty.servlet.DefaultServlet-929987349, stacks=stacks, conf=conf, metrics=metrics} 2015-11-22 19:29:54,349 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 + (F=static_user_filter,[/conf],[],15) as filterMapping 2015-11-22 19:29:54,349 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:54,349 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15), (F=RMAuthenticationFilter,[/stacks],[],15), (F=static_user_filter,[/stacks],[],15), (F=RMAuthenticationFilter,[/logLevel],[],15), (F=static_user_filter,[/logLevel],[],15), (F=RMAuthenticationFilter,[/metrics],[],15), (F=static_user_filter,[/metrics],[],15), (F=RMAuthenticationFilter,[/jmx],[],15), (F=static_user_filter,[/jmx],[],15), (F=RMAuthenticationFilter,[/conf],[],15), (F=static_user_filter,[/conf],[],15)] 2015-11-22 19:29:54,349 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:54,349 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/jmx=jmx, /conf=conf, /stacks=stacks, /logLevel=logLevel, /metrics=metrics, /=org.mortbay.jetty.servlet.DefaultServlet-929987349} 2015-11-22 19:29:54,350 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={logLevel=logLevel, jmx=jmx, org.mortbay.jetty.servlet.DefaultServlet-929987349=org.mortbay.jetty.servlet.DefaultServlet-929987349, stacks=stacks, conf=conf, metrics=metrics} 2015-11-22 19:29:54,350 INFO [RM-0] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /cluster/* 2015-11-22 19:29:54,350 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 + (F=RMAuthenticationFilter,[/cluster/*],[],15) as filterMapping 2015-11-22 19:29:54,350 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:54,350 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15), (F=RMAuthenticationFilter,[/stacks],[],15), (F=static_user_filter,[/stacks],[],15), (F=RMAuthenticationFilter,[/logLevel],[],15), (F=static_user_filter,[/logLevel],[],15), (F=RMAuthenticationFilter,[/metrics],[],15), (F=static_user_filter,[/metrics],[],15), (F=RMAuthenticationFilter,[/jmx],[],15), (F=static_user_filter,[/jmx],[],15), (F=RMAuthenticationFilter,[/conf],[],15), (F=static_user_filter,[/conf],[],15), (F=RMAuthenticationFilter,[/cluster/*],[],15)] 2015-11-22 19:29:54,350 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:54,350 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/jmx=jmx, /conf=conf, /stacks=stacks, /logLevel=logLevel, /metrics=metrics, /=org.mortbay.jetty.servlet.DefaultServlet-929987349} 2015-11-22 19:29:54,350 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={logLevel=logLevel, jmx=jmx, org.mortbay.jetty.servlet.DefaultServlet-929987349=org.mortbay.jetty.servlet.DefaultServlet-929987349, stacks=stacks, conf=conf, metrics=metrics} 2015-11-22 19:29:54,350 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 + (F=static_user_filter,[/cluster/*],[],15) as filterMapping 2015-11-22 19:29:54,351 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:54,351 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15), (F=RMAuthenticationFilter,[/stacks],[],15), (F=static_user_filter,[/stacks],[],15), (F=RMAuthenticationFilter,[/logLevel],[],15), (F=static_user_filter,[/logLevel],[],15), (F=RMAuthenticationFilter,[/metrics],[],15), (F=static_user_filter,[/metrics],[],15), (F=RMAuthenticationFilter,[/jmx],[],15), (F=static_user_filter,[/jmx],[],15), (F=RMAuthenticationFilter,[/conf],[],15), (F=static_user_filter,[/conf],[],15), (F=RMAuthenticationFilter,[/cluster/*],[],15), (F=static_user_filter,[/cluster/*],[],15)] 2015-11-22 19:29:54,351 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:54,351 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/jmx=jmx, /conf=conf, /stacks=stacks, /logLevel=logLevel, /metrics=metrics, /=org.mortbay.jetty.servlet.DefaultServlet-929987349} 2015-11-22 19:29:54,351 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={logLevel=logLevel, jmx=jmx, org.mortbay.jetty.servlet.DefaultServlet-929987349=org.mortbay.jetty.servlet.DefaultServlet-929987349, stacks=stacks, conf=conf, metrics=metrics} 2015-11-22 19:29:54,351 INFO [RM-0] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /ws/* 2015-11-22 19:29:54,351 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 + (F=RMAuthenticationFilter,[/ws/*],[],15) as filterMapping 2015-11-22 19:29:54,351 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:54,351 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15), (F=RMAuthenticationFilter,[/stacks],[],15), (F=static_user_filter,[/stacks],[],15), (F=RMAuthenticationFilter,[/logLevel],[],15), (F=static_user_filter,[/logLevel],[],15), (F=RMAuthenticationFilter,[/metrics],[],15), (F=static_user_filter,[/metrics],[],15), (F=RMAuthenticationFilter,[/jmx],[],15), (F=static_user_filter,[/jmx],[],15), (F=RMAuthenticationFilter,[/conf],[],15), (F=static_user_filter,[/conf],[],15), (F=RMAuthenticationFilter,[/cluster/*],[],15), (F=static_user_filter,[/cluster/*],[],15), (F=RMAuthenticationFilter,[/ws/*],[],15)] 2015-11-22 19:29:54,352 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:54,352 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/jmx=jmx, /conf=conf, /stacks=stacks, /logLevel=logLevel, /metrics=metrics, /=org.mortbay.jetty.servlet.DefaultServlet-929987349} 2015-11-22 19:29:54,352 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={logLevel=logLevel, jmx=jmx, org.mortbay.jetty.servlet.DefaultServlet-929987349=org.mortbay.jetty.servlet.DefaultServlet-929987349, stacks=stacks, conf=conf, metrics=metrics} 2015-11-22 19:29:54,352 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 + (F=static_user_filter,[/ws/*],[],15) as filterMapping 2015-11-22 19:29:54,352 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:54,352 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15), (F=RMAuthenticationFilter,[/stacks],[],15), (F=static_user_filter,[/stacks],[],15), (F=RMAuthenticationFilter,[/logLevel],[],15), (F=static_user_filter,[/logLevel],[],15), (F=RMAuthenticationFilter,[/metrics],[],15), (F=static_user_filter,[/metrics],[],15), (F=RMAuthenticationFilter,[/jmx],[],15), (F=static_user_filter,[/jmx],[],15), (F=RMAuthenticationFilter,[/conf],[],15), (F=static_user_filter,[/conf],[],15), (F=RMAuthenticationFilter,[/cluster/*],[],15), (F=static_user_filter,[/cluster/*],[],15), (F=RMAuthenticationFilter,[/ws/*],[],15), (F=static_user_filter,[/ws/*],[],15)] 2015-11-22 19:29:54,352 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:54,352 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/jmx=jmx, /conf=conf, /stacks=stacks, /logLevel=logLevel, /metrics=metrics, /=org.mortbay.jetty.servlet.DefaultServlet-929987349} 2015-11-22 19:29:54,352 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={logLevel=logLevel, jmx=jmx, org.mortbay.jetty.servlet.DefaultServlet-929987349=org.mortbay.jetty.servlet.DefaultServlet-929987349, stacks=stacks, conf=conf, metrics=metrics} 2015-11-22 19:29:54,353 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container Server@7ff1e948 + HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:0 as connector 2015-11-22 19:29:54,353 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 + proxy as servlet 2015-11-22 19:29:54,353 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 + (S=proxy,[/proxy/*]) as servletMapping 2015-11-22 19:29:54,353 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:54,353 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15), (F=RMAuthenticationFilter,[/stacks],[],15), (F=static_user_filter,[/stacks],[],15), (F=RMAuthenticationFilter,[/logLevel],[],15), (F=static_user_filter,[/logLevel],[],15), (F=RMAuthenticationFilter,[/metrics],[],15), (F=static_user_filter,[/metrics],[],15), (F=RMAuthenticationFilter,[/jmx],[],15), (F=static_user_filter,[/jmx],[],15), (F=RMAuthenticationFilter,[/conf],[],15), (F=static_user_filter,[/conf],[],15), (F=RMAuthenticationFilter,[/cluster/*],[],15), (F=static_user_filter,[/cluster/*],[],15), (F=RMAuthenticationFilter,[/ws/*],[],15), (F=static_user_filter,[/ws/*],[],15)] 2015-11-22 19:29:54,353 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:54,353 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/jmx=jmx, /proxy/*=proxy, /conf=conf, /stacks=stacks, /logLevel=logLevel, /metrics=metrics, /=org.mortbay.jetty.servlet.DefaultServlet-929987349} 2015-11-22 19:29:54,353 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={proxy=proxy, logLevel=logLevel, jmx=jmx, org.mortbay.jetty.servlet.DefaultServlet-929987349=org.mortbay.jetty.servlet.DefaultServlet-929987349, stacks=stacks, conf=conf, metrics=metrics} 2015-11-22 19:29:54,353 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 + (F=RMAuthenticationFilter,[/proxy/*],[],15) as filterMapping 2015-11-22 19:29:54,354 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:54,354 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15), (F=RMAuthenticationFilter,[/stacks],[],15), (F=static_user_filter,[/stacks],[],15), (F=RMAuthenticationFilter,[/logLevel],[],15), (F=static_user_filter,[/logLevel],[],15), (F=RMAuthenticationFilter,[/metrics],[],15), (F=static_user_filter,[/metrics],[],15), (F=RMAuthenticationFilter,[/jmx],[],15), (F=static_user_filter,[/jmx],[],15), (F=RMAuthenticationFilter,[/conf],[],15), (F=static_user_filter,[/conf],[],15), (F=RMAuthenticationFilter,[/cluster/*],[],15), (F=static_user_filter,[/cluster/*],[],15), (F=RMAuthenticationFilter,[/ws/*],[],15), (F=static_user_filter,[/ws/*],[],15), (F=RMAuthenticationFilter,[/proxy/*],[],15)] 2015-11-22 19:29:54,354 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:54,354 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/jmx=jmx, /proxy/*=proxy, /conf=conf, /stacks=stacks, /logLevel=logLevel, /metrics=metrics, /=org.mortbay.jetty.servlet.DefaultServlet-929987349} 2015-11-22 19:29:54,354 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={proxy=proxy, logLevel=logLevel, jmx=jmx, org.mortbay.jetty.servlet.DefaultServlet-929987349=org.mortbay.jetty.servlet.DefaultServlet-929987349, stacks=stacks, conf=conf, metrics=metrics} 2015-11-22 19:29:54,354 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 + (F=static_user_filter,[/proxy/*],[],15) as filterMapping 2015-11-22 19:29:54,354 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:54,354 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15), (F=RMAuthenticationFilter,[/stacks],[],15), (F=static_user_filter,[/stacks],[],15), (F=RMAuthenticationFilter,[/logLevel],[],15), (F=static_user_filter,[/logLevel],[],15), (F=RMAuthenticationFilter,[/metrics],[],15), (F=static_user_filter,[/metrics],[],15), (F=RMAuthenticationFilter,[/jmx],[],15), (F=static_user_filter,[/jmx],[],15), (F=RMAuthenticationFilter,[/conf],[],15), (F=static_user_filter,[/conf],[],15), (F=RMAuthenticationFilter,[/cluster/*],[],15), (F=static_user_filter,[/cluster/*],[],15), (F=RMAuthenticationFilter,[/ws/*],[],15), (F=static_user_filter,[/ws/*],[],15), (F=RMAuthenticationFilter,[/proxy/*],[],15), (F=static_user_filter,[/proxy/*],[],15)] 2015-11-22 19:29:54,354 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:54,355 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/jmx=jmx, /proxy/*=proxy, /conf=conf, /stacks=stacks, /logLevel=logLevel, /metrics=metrics, /=org.mortbay.jetty.servlet.DefaultServlet-929987349} 2015-11-22 19:29:54,355 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={proxy=proxy, logLevel=logLevel, jmx=jmx, org.mortbay.jetty.servlet.DefaultServlet-929987349=org.mortbay.jetty.servlet.DefaultServlet-929987349, stacks=stacks, conf=conf, metrics=metrics} 2015-11-22 19:29:54,355 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 + guice as filter 2015-11-22 19:29:54,355 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 + (F=guice,[/*],[],15) as filterMapping 2015-11-22 19:29:54,355 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter, guice=guice} 2015-11-22 19:29:54,355 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15), (F=RMAuthenticationFilter,[/stacks],[],15), (F=static_user_filter,[/stacks],[],15), (F=RMAuthenticationFilter,[/logLevel],[],15), (F=static_user_filter,[/logLevel],[],15), (F=RMAuthenticationFilter,[/metrics],[],15), (F=static_user_filter,[/metrics],[],15), (F=RMAuthenticationFilter,[/jmx],[],15), (F=static_user_filter,[/jmx],[],15), (F=RMAuthenticationFilter,[/conf],[],15), (F=static_user_filter,[/conf],[],15), (F=RMAuthenticationFilter,[/cluster/*],[],15), (F=static_user_filter,[/cluster/*],[],15), (F=RMAuthenticationFilter,[/ws/*],[],15), (F=static_user_filter,[/ws/*],[],15), (F=RMAuthenticationFilter,[/proxy/*],[],15), (F=static_user_filter,[/proxy/*],[],15), (F=guice,[/*],[],15)] 2015-11-22 19:29:54,355 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:54,355 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/jmx=jmx, /proxy/*=proxy, /conf=conf, /stacks=stacks, /logLevel=logLevel, /metrics=metrics, /=org.mortbay.jetty.servlet.DefaultServlet-929987349} 2015-11-22 19:29:54,355 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={proxy=proxy, logLevel=logLevel, jmx=jmx, org.mortbay.jetty.servlet.DefaultServlet-929987349=org.mortbay.jetty.servlet.DefaultServlet-929987349, stacks=stacks, conf=conf, metrics=metrics} 2015-11-22 19:29:54,357 DEBUG [RM-0] webapp.Router (Router.java:add(85)) - adding /([])->class org.apache.hadoop.yarn.server.resourcemanager.webapp.RmController#index 2015-11-22 19:29:54,357 DEBUG [RM-0] webapp.Router (Router.java:load(267)) - trying: org.apache.hadoop.yarn.server.resourcemanager.webapp.RmView 2015-11-22 19:29:54,357 DEBUG [RM-0] webapp.Router (Router.java:load(271)) - found org.apache.hadoop.yarn.server.resourcemanager.webapp.RmView 2015-11-22 19:29:54,357 DEBUG [RM-0] webapp.Router (Router.java:add(85)) - adding /nodes([:node.state])->class org.apache.hadoop.yarn.server.resourcemanager.webapp.RmController#nodes 2015-11-22 19:29:54,357 DEBUG [RM-0] webapp.Router (Router.java:load(267)) - trying: org.apache.hadoop.yarn.server.resourcemanager.webapp.RmView 2015-11-22 19:29:54,357 DEBUG [RM-0] webapp.Router (Router.java:load(271)) - found org.apache.hadoop.yarn.server.resourcemanager.webapp.RmView 2015-11-22 19:29:54,358 DEBUG [RM-0] webapp.Router (Router.java:add(85)) - adding /apps([:app.state])->class org.apache.hadoop.yarn.server.resourcemanager.webapp.RmController#index 2015-11-22 19:29:54,358 DEBUG [RM-0] webapp.Router (Router.java:load(267)) - trying: org.apache.hadoop.yarn.server.resourcemanager.webapp.RmView 2015-11-22 19:29:54,358 DEBUG [RM-0] webapp.Router (Router.java:load(271)) - found org.apache.hadoop.yarn.server.resourcemanager.webapp.RmView 2015-11-22 19:29:54,358 DEBUG [RM-0] webapp.Router (Router.java:add(85)) - adding /cluster([])->class org.apache.hadoop.yarn.server.resourcemanager.webapp.RmController#about 2015-11-22 19:29:54,358 DEBUG [RM-0] webapp.Router (Router.java:load(267)) - trying: org.apache.hadoop.yarn.server.resourcemanager.webapp.RmView 2015-11-22 19:29:54,358 DEBUG [RM-0] webapp.Router (Router.java:load(271)) - found org.apache.hadoop.yarn.server.resourcemanager.webapp.RmView 2015-11-22 19:29:54,358 DEBUG [RM-0] webapp.Router (Router.java:add(85)) - adding /app([:app.id])->class org.apache.hadoop.yarn.server.resourcemanager.webapp.RmController#app 2015-11-22 19:29:54,358 DEBUG [RM-0] webapp.Router (Router.java:load(267)) - trying: org.apache.hadoop.yarn.server.resourcemanager.webapp.RmView 2015-11-22 19:29:54,358 DEBUG [RM-0] webapp.Router (Router.java:load(271)) - found org.apache.hadoop.yarn.server.resourcemanager.webapp.RmView 2015-11-22 19:29:54,358 DEBUG [RM-0] webapp.Router (Router.java:add(85)) - adding /scheduler([])->class org.apache.hadoop.yarn.server.resourcemanager.webapp.RmController#scheduler 2015-11-22 19:29:54,358 DEBUG [RM-0] webapp.Router (Router.java:load(267)) - trying: org.apache.hadoop.yarn.server.resourcemanager.webapp.RmView 2015-11-22 19:29:54,358 DEBUG [RM-0] webapp.Router (Router.java:load(271)) - found org.apache.hadoop.yarn.server.resourcemanager.webapp.RmView 2015-11-22 19:29:54,358 DEBUG [RM-0] webapp.Router (Router.java:add(85)) - adding /queue([:queue.name])->class org.apache.hadoop.yarn.server.resourcemanager.webapp.RmController#queue 2015-11-22 19:29:54,359 DEBUG [RM-0] webapp.Router (Router.java:load(267)) - trying: org.apache.hadoop.yarn.server.resourcemanager.webapp.RmView 2015-11-22 19:29:54,359 DEBUG [RM-0] webapp.Router (Router.java:load(271)) - found org.apache.hadoop.yarn.server.resourcemanager.webapp.RmView 2015-11-22 19:29:54,359 DEBUG [RM-0] webapp.Router (Router.java:add(85)) - adding /nodelabels([])->class org.apache.hadoop.yarn.server.resourcemanager.webapp.RmController#nodelabels 2015-11-22 19:29:54,359 DEBUG [RM-0] webapp.Router (Router.java:load(267)) - trying: org.apache.hadoop.yarn.server.resourcemanager.webapp.RmView 2015-11-22 19:29:54,359 DEBUG [RM-0] webapp.Router (Router.java:load(271)) - found org.apache.hadoop.yarn.server.resourcemanager.webapp.RmView 2015-11-22 19:29:54,359 DEBUG [RM-0] webapp.Router (Router.java:add(85)) - adding /appattempt([:appattempt.id])->class org.apache.hadoop.yarn.server.resourcemanager.webapp.RmController#appattempt 2015-11-22 19:29:54,359 DEBUG [RM-0] webapp.Router (Router.java:load(267)) - trying: org.apache.hadoop.yarn.server.resourcemanager.webapp.RmView 2015-11-22 19:29:54,359 DEBUG [RM-0] webapp.Router (Router.java:load(271)) - found org.apache.hadoop.yarn.server.resourcemanager.webapp.RmView 2015-11-22 19:29:54,359 DEBUG [RM-0] webapp.Router (Router.java:add(85)) - adding /container([:container.id])->class org.apache.hadoop.yarn.server.resourcemanager.webapp.RmController#container 2015-11-22 19:29:54,359 DEBUG [RM-0] webapp.Router (Router.java:load(267)) - trying: org.apache.hadoop.yarn.server.resourcemanager.webapp.RmView 2015-11-22 19:29:54,359 DEBUG [RM-0] webapp.Router (Router.java:load(271)) - found org.apache.hadoop.yarn.server.resourcemanager.webapp.RmView 2015-11-22 19:29:54,359 DEBUG [RM-0] webapp.Router (Router.java:add(85)) - adding /errors-and-warnings([])->class org.apache.hadoop.yarn.server.resourcemanager.webapp.RmController#errorsAndWarnings 2015-11-22 19:29:54,360 DEBUG [RM-0] webapp.Router (Router.java:load(267)) - trying: org.apache.hadoop.yarn.server.resourcemanager.webapp.RmView 2015-11-22 19:29:54,360 DEBUG [RM-0] webapp.Router (Router.java:load(271)) - found org.apache.hadoop.yarn.server.resourcemanager.webapp.RmView 2015-11-22 19:29:54,360 DEBUG [RM-0] webapp.Router (Router.java:add(85)) - adding /logaggregationstatus([:app.id])->class org.apache.hadoop.yarn.server.resourcemanager.webapp.RmController#logaggregationstatus 2015-11-22 19:29:54,360 DEBUG [RM-0] webapp.Router (Router.java:load(267)) - trying: org.apache.hadoop.yarn.server.resourcemanager.webapp.RmView 2015-11-22 19:29:54,360 DEBUG [RM-0] webapp.Router (Router.java:load(271)) - found org.apache.hadoop.yarn.server.resourcemanager.webapp.RmView 2015-11-22 19:29:54,362 INFO [RM-0] webapp.WebApps (WebApps.java:build(288)) - Registered webapp guice modules 2015-11-22 19:29:54,363 INFO [RM-0] http.HttpServer2 (HttpServer2.java:openListeners(917)) - Jetty bound to port 40017 2015-11-22 19:29:54,363 INFO [RM-0] mortbay.log (Slf4jLog.java:info(67)) - jetty-6.1.26 2015-11-22 19:29:54,363 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - started org.mortbay.thread.QueuedThreadPool@7e23a01a 2015-11-22 19:29:54,365 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Thread Context class loader is: ContextLoader@cluster([]) / sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,365 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Parent class loader is: sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,365 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Parent class loader is: sun.misc.Launcher$ExtClassLoader@2de8284b 2015-11-22 19:29:54,365 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Try webapp=file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes/webapps/cluster/, exists=true, directory=true 2015-11-22 19:29:54,365 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - webapp=file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes/webapps/cluster/ 2015-11-22 19:29:54,365 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Created temp dir /tmp/Jetty_ip.172.31.20.42_ap.northeast.1_compute_internal_40017_cluster____oa9zzc for org.mortbay.jetty.webapp.WebAppContext@5b1e5299{/,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes/webapps/cluster} 2015-11-22 19:29:54,366 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Configuring web-jetty.xml 2015-11-22 19:29:54,366 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - TLD search of file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/surefire/surefirebooter2438562254715847001.jar 2015-11-22 19:29:54,366 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - TLD search of file:/usr/lib/jvm/java-8-oracle/jre/lib/ext/zipfs.jar 2015-11-22 19:29:54,366 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - TLD search of file:/usr/lib/jvm/java-8-oracle/jre/lib/ext/sunjce_provider.jar 2015-11-22 19:29:54,366 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - TLD search of file:/usr/lib/jvm/java-8-oracle/jre/lib/ext/sunpkcs11.jar 2015-11-22 19:29:54,366 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - TLD search of file:/usr/lib/jvm/java-8-oracle/jre/lib/ext/sunec.jar 2015-11-22 19:29:54,367 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - TLD search of file:/usr/lib/jvm/java-8-oracle/jre/lib/ext/dnsns.jar 2015-11-22 19:29:54,367 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - TLD search of file:/usr/lib/jvm/java-8-oracle/jre/lib/ext/localedata.jar 2015-11-22 19:29:54,367 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - TLD search of file:/usr/lib/jvm/java-8-oracle/jre/lib/ext/jfxrt.jar 2015-11-22 19:29:54,372 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - TLD search of file:/usr/lib/jvm/java-8-oracle/jre/lib/ext/cldrdata.jar 2015-11-22 19:29:54,373 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - TLD search of file:/usr/lib/jvm/java-8-oracle/jre/lib/ext/nashorn.jar 2015-11-22 19:29:54,374 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.org.apache.xerces.internal.jaxp.SAXParserFactoryImpl from null 2015-11-22 19:29:54,374 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.org.apache.xerces.internal.impl.dv.dtd.DTDDVFactoryImpl from null 2015-11-22 19:29:54,374 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - getResource(javax/servlet/jsp/resources/web-jsptaglibrary_1_1.dtd)=jar:file:/home/ubuntu/.m2/repository/javax/servlet/jsp/jsp-api/2.1/jsp-api-2.1.jar!/javax/servlet/jsp/resources/web-jsptaglibrary_1_1.dtd 2015-11-22 19:29:54,374 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - getResource(javax/servlet/jsp/resources/web-jsptaglibrary_1_2.dtd)=jar:file:/home/ubuntu/.m2/repository/javax/servlet/jsp/jsp-api/2.1/jsp-api-2.1.jar!/javax/servlet/jsp/resources/web-jsptaglibrary_1_2.dtd 2015-11-22 19:29:54,375 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - getResource(javax/servlet/jsp/resources/web-jsptaglibrary_2_0.xsd)=jar:file:/home/ubuntu/.m2/repository/javax/servlet/jsp/jsp-api/2.1/jsp-api-2.1.jar!/javax/servlet/jsp/resources/web-jsptaglibrary_2_0.xsd 2015-11-22 19:29:54,375 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - getResource(javax/servlet/jsp/resources/web-jsptaglibrary_1_1.dtd)=jar:file:/home/ubuntu/.m2/repository/javax/servlet/jsp/jsp-api/2.1/jsp-api-2.1.jar!/javax/servlet/jsp/resources/web-jsptaglibrary_1_1.dtd 2015-11-22 19:29:54,375 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - getResource(javax/servlet/jsp/resources/web-jsptaglibrary_1_2.dtd)=jar:file:/home/ubuntu/.m2/repository/javax/servlet/jsp/jsp-api/2.1/jsp-api-2.1.jar!/javax/servlet/jsp/resources/web-jsptaglibrary_1_2.dtd 2015-11-22 19:29:54,375 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - getResource(javax/servlet/jsp/resources/web-jsptaglibrary_2_0.xsd)=jar:file:/home/ubuntu/.m2/repository/javax/servlet/jsp/jsp-api/2.1/jsp-api-2.1.jar!/javax/servlet/jsp/resources/web-jsptaglibrary_2_0.xsd 2015-11-22 19:29:54,375 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container Server@7ff1e948 + org.mortbay.jetty.servlet.HashSessionIdManager@122752d1 as sessionIdManager 2015-11-22 19:29:54,375 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Init SecureRandom. 2015-11-22 19:29:54,375 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - started org.mortbay.jetty.servlet.HashSessionIdManager@122752d1 2015-11-22 19:29:54,375 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - started org.mortbay.jetty.servlet.HashSessionManager@468980ed 2015-11-22 19:29:54,376 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter, guice=guice} 2015-11-22 19:29:54,376 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15), (F=RMAuthenticationFilter,[/stacks],[],15), (F=static_user_filter,[/stacks],[],15), (F=RMAuthenticationFilter,[/logLevel],[],15), (F=static_user_filter,[/logLevel],[],15), (F=RMAuthenticationFilter,[/metrics],[],15), (F=static_user_filter,[/metrics],[],15), (F=RMAuthenticationFilter,[/jmx],[],15), (F=static_user_filter,[/jmx],[],15), (F=RMAuthenticationFilter,[/conf],[],15), (F=static_user_filter,[/conf],[],15), (F=RMAuthenticationFilter,[/cluster/*],[],15), (F=static_user_filter,[/cluster/*],[],15), (F=RMAuthenticationFilter,[/ws/*],[],15), (F=static_user_filter,[/ws/*],[],15), (F=RMAuthenticationFilter,[/proxy/*],[],15), (F=static_user_filter,[/proxy/*],[],15), (F=guice,[/*],[],15)] 2015-11-22 19:29:54,376 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:54,376 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/jmx=jmx, /proxy/*=proxy, /conf=conf, /stacks=stacks, /logLevel=logLevel, /metrics=metrics, /=org.mortbay.jetty.servlet.DefaultServlet-929987349} 2015-11-22 19:29:54,376 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={proxy=proxy, logLevel=logLevel, jmx=jmx, org.mortbay.jetty.servlet.DefaultServlet-929987349=org.mortbay.jetty.servlet.DefaultServlet-929987349, stacks=stacks, conf=conf, metrics=metrics} 2015-11-22 19:29:54,376 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - starting ServletHandler@38de1a90 2015-11-22 19:29:54,376 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - started ServletHandler@38de1a90 2015-11-22 19:29:54,376 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - starting SecurityHandler@48150f6a 2015-11-22 19:29:54,376 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - started SecurityHandler@48150f6a 2015-11-22 19:29:54,376 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - starting SessionHandler@6f2a8d0b 2015-11-22 19:29:54,376 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - started SessionHandler@6f2a8d0b 2015-11-22 19:29:54,376 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - starting org.mortbay.jetty.webapp.WebAppContext@5b1e5299{/,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes/webapps/cluster} 2015-11-22 19:29:54,377 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - starting ErrorPageErrorHandler@e4b0836 2015-11-22 19:29:54,377 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - started ErrorPageErrorHandler@e4b0836 2015-11-22 19:29:54,377 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class org.apache.hadoop.http.NoCacheFilter from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,377 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Holding class org.apache.hadoop.http.NoCacheFilter 2015-11-22 19:29:54,377 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - started NoCacheFilter 2015-11-22 19:29:54,377 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class org.apache.hadoop.http.NoCacheFilter from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,377 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Holding class org.apache.hadoop.http.NoCacheFilter 2015-11-22 19:29:54,377 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - started NoCacheFilter 2015-11-22 19:29:54,377 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class org.apache.hadoop.http.HttpServer2$QuotingInputFilter from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,377 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Holding class org.apache.hadoop.http.HttpServer2$QuotingInputFilter 2015-11-22 19:29:54,377 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - started safety 2015-11-22 19:29:54,377 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,377 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Holding class org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter 2015-11-22 19:29:54,378 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class org.apache.hadoop.security.token.delegation.web.PseudoDelegationTokenAuthenticationHandler from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,378 INFO [RM-0] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:29:54,378 INFO [Thread[Thread-4047,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2015-11-22 19:29:54,379 INFO [Thread[Thread-4047,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:29:54,379 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - started RMAuthenticationFilter 2015-11-22 19:29:54,379 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,379 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Holding class org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter 2015-11-22 19:29:54,379 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - started static_user_filter 2015-11-22 19:29:54,380 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class org.apache.hadoop.yarn.server.timeline.security.TimelineAuthenticationFilter from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,380 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Holding class org.apache.hadoop.yarn.server.timeline.security.TimelineAuthenticationFilter 2015-11-22 19:29:54,380 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class org.apache.hadoop.security.token.delegation.web.PseudoDelegationTokenAuthenticationHandler from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,380 INFO [RM-0] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:29:54,380 DEBUG [RM-0] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:stopThreads(619)) - Stopping expired delegation token remover thread 2015-11-22 19:29:54,380 INFO [Thread[Thread-4049,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2015-11-22 19:29:54,380 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - started Timeline Authentication Filter 2015-11-22 19:29:54,380 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.google.inject.servlet.GuiceFilter from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,381 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Holding class com.google.inject.servlet.GuiceFilter 2015-11-22 19:29:54,381 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.server.impl.container.WebApplicationProviderImpl from sun.misc.Launcher$AppClassLoader@42a57993 Nov 22, 2015 7:29:54 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.resourcemanager.webapp.JAXBContextResolver as a provider class Nov 22, 2015 7:29:54 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices as a root resource class Nov 22, 2015 7:29:54 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a provider class Nov 22, 2015 7:29:54 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate INFO: Initiating Jersey application, version 'Jersey: 1.9 09/02/2011 11:17 AM' 2015-11-22 19:29:54,384 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.server.impl.model.method.dispatch.VoidVoidDispatchProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,384 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.server.impl.model.method.dispatch.HttpReqResDispatchProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,384 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.server.impl.model.method.dispatch.MultipartFormDispatchProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,384 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.server.impl.model.method.dispatch.FormDispatchProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,385 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.server.impl.model.method.dispatch.EntityParamDispatchProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,385 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded interface javax.annotation.PostConstruct 2015-11-22 19:29:54,385 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded interface javax.annotation.PostConstruct from null 2015-11-22 19:29:54,385 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded interface javax.annotation.PreDestroy 2015-11-22 19:29:54,385 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded interface javax.annotation.PreDestroy from null 2015-11-22 19:29:54,387 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.xml.SAXParserContextProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,387 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.xml.XMLStreamReaderContextProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,387 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.xml.DocumentBuilderFactoryProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,387 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.xml.TransformerFactoryProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,390 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.api.container.filter.GZIPContentEncodingFilter from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,390 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.server.impl.container.filter.NormalizeFilter from sun.misc.Launcher$AppClassLoader@42a57993 Nov 22, 2015 7:29:54 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.resourcemanager.webapp.JAXBContextResolver to GuiceManagedComponentProvider with the scope "Singleton" 2015-11-22 19:29:54,396 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - getResource(META-INF/services/javax.xml.bind.JAXBContext)=jar:file:/home/ubuntu/.m2/repository/com/sun/xml/bind/jaxb-impl/2.2.3-1/jaxb-impl-2.2.3-1.jar!/META-INF/services/javax.xml.bind.JAXBContext 2015-11-22 19:29:54,397 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.xml.bind.v2.ContextFactory from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,440 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - getResource(META-INF/services/javax.xml.bind.JAXBContext)=jar:file:/home/ubuntu/.m2/repository/com/sun/xml/bind/jaxb-impl/2.2.3-1/jaxb-impl-2.2.3-1.jar!/META-INF/services/javax.xml.bind.JAXBContext 2015-11-22 19:29:54,440 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.xml.bind.v2.ContextFactory from sun.misc.Launcher$AppClassLoader@42a57993 Nov 22, 2015 7:29:54 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.webapp.GenericExceptionHandler to GuiceManagedComponentProvider with the scope "Singleton" 2015-11-22 19:29:54,452 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.StringProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,452 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.ByteArrayProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,452 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.FileProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,452 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.InputStreamProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,453 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.DataSourceProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,453 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.RenderedImageProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,453 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.MimeMultipartProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,453 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.FormProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,453 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.FormMultivaluedMapProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,454 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.XMLRootElementProvider$App from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,454 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.XMLRootElementProvider$Text from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,454 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.XMLRootElementProvider$General from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,454 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.XMLJAXBElementProvider$App from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,454 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.XMLJAXBElementProvider$Text from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,454 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.XMLJAXBElementProvider$General from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,455 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.XMLListElementProvider$App from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,455 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.XMLListElementProvider$Text from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,455 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.XMLListElementProvider$General from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,455 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.ReaderProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,455 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.DocumentProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,455 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.SourceProvider$StreamSourceReader from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,456 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.SourceProvider$SAXSourceReader from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,456 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.SourceProvider$DOMSourceReader from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,456 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.XMLRootObjectProvider$App from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,456 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.XMLRootObjectProvider$Text from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,456 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.XMLRootObjectProvider$General from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,456 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.EntityHolderReader from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,457 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.json.impl.provider.entity.JSONRootElementProvider$App from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,457 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.json.impl.provider.entity.JSONRootElementProvider$General from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,457 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.json.impl.provider.entity.JSONJAXBElementProvider$App from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,457 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.json.impl.provider.entity.JSONJAXBElementProvider$General from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,457 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.json.impl.provider.entity.JSONListElementProvider$App from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,458 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.json.impl.provider.entity.JSONListElementProvider$General from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,458 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.json.impl.provider.entity.JSONArrayProvider$App from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,458 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.json.impl.provider.entity.JSONArrayProvider$General from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,458 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.json.impl.provider.entity.JSONObjectProvider$App from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,458 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.json.impl.provider.entity.JSONObjectProvider$General from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,458 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.json.impl.provider.entity.JacksonProviderProxy from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,485 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.StreamingOutputProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,486 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.SourceProvider$SourceWriter from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,486 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.json.impl.provider.entity.JSONWithPaddingProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,486 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.server.impl.template.ViewableMessageBodyWriter from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,489 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.server.impl.model.parameter.multivalued.StringReaderProviders$TypeFromStringEnum from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,489 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.server.impl.model.parameter.multivalued.StringReaderProviders$TypeValueOf from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,489 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.server.impl.model.parameter.multivalued.StringReaderProviders$TypeFromString from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,489 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.server.impl.model.parameter.multivalued.StringReaderProviders$StringConstructor from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,489 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.server.impl.model.parameter.multivalued.StringReaderProviders$DateProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,490 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.server.impl.model.parameter.multivalued.JAXBStringReaderProviders$RootElementProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,500 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - getResource(META-INF/services/javax.xml.bind.JAXBContext)=jar:file:/home/ubuntu/.m2/repository/com/sun/xml/bind/jaxb-impl/2.2.3-1/jaxb-impl-2.2.3-1.jar!/META-INF/services/javax.xml.bind.JAXBContext 2015-11-22 19:29:54,500 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.xml.bind.v2.ContextFactory from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:54,500 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.research.ws.wadl.ObjectFactory from sun.misc.Launcher$AppClassLoader@42a57993 Nov 22, 2015 7:29:54 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices to GuiceManagedComponentProvider with the scope "Singleton" 2015-11-22 19:29:54,521 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - started guice 2015-11-22 19:29:54,521 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - started conf 2015-11-22 19:29:54,521 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - started stacks 2015-11-22 19:29:54,521 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - started jmx 2015-11-22 19:29:54,521 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - started logLevel 2015-11-22 19:29:54,521 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - started metrics 2015-11-22 19:29:54,521 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - started proxy 2015-11-22 19:29:54,522 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - started org.mortbay.jetty.servlet.DefaultServlet$NIOResourceCache@290906b6 2015-11-22 19:29:54,522 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - resource base = file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes/webapps/cluster/ 2015-11-22 19:29:54,522 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - started org.mortbay.jetty.servlet.DefaultServlet-929987349 2015-11-22 19:29:54,522 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - started org.mortbay.jetty.webapp.WebAppContext@5b1e5299{/,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes/webapps/cluster} 2015-11-22 19:29:54,522 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container org.mortbay.jetty.servlet.Context@6ec538ca{/logs,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/log} + ErrorHandler@4557bc67 as errorHandler 2015-11-22 19:29:54,522 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:54,522 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=safety,[/*],[],15), (F=RMAuthenticationFilter,[/*],[],15), (F=static_user_filter,[/*],[],15), (F=Timeline Authentication Filter,[/*],[],15)] 2015-11-22 19:29:54,522 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:54,522 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/*=org.apache.hadoop.http.AdminAuthorizedServlet-1597562478} 2015-11-22 19:29:54,522 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.apache.hadoop.http.AdminAuthorizedServlet-1597562478=org.apache.hadoop.http.AdminAuthorizedServlet-1597562478} 2015-11-22 19:29:54,522 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - starting ServletHandler@553cb2cc 2015-11-22 19:29:54,523 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - started ServletHandler@553cb2cc 2015-11-22 19:29:54,523 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - starting org.mortbay.jetty.servlet.Context@6ec538ca{/logs,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/log} 2015-11-22 19:29:54,523 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - starting ErrorHandler@4557bc67 2015-11-22 19:29:54,523 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - started ErrorHandler@4557bc67 2015-11-22 19:29:54,523 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Holding class org.apache.hadoop.http.AdminAuthorizedServlet 2015-11-22 19:29:54,523 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - started org.apache.hadoop.http.AdminAuthorizedServlet-1597562478 2015-11-22 19:29:54,523 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - started org.mortbay.jetty.servlet.Context@6ec538ca{/logs,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/log} 2015-11-22 19:29:54,523 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Container org.mortbay.jetty.servlet.Context@9647707{/static,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes/webapps/static/} + ErrorHandler@3c22152a as errorHandler 2015-11-22 19:29:54,523 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:54,523 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=safety,[/*],[],15), (F=RMAuthenticationFilter,[/*],[],15), (F=static_user_filter,[/*],[],15), (F=Timeline Authentication Filter,[/*],[],15)] 2015-11-22 19:29:54,523 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:54,523 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/*=org.mortbay.jetty.servlet.DefaultServlet-2126150723} 2015-11-22 19:29:54,523 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-2126150723=org.mortbay.jetty.servlet.DefaultServlet-2126150723} 2015-11-22 19:29:54,523 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - starting ServletHandler@4ea00d89 2015-11-22 19:29:54,524 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - started ServletHandler@4ea00d89 2015-11-22 19:29:54,524 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - starting org.mortbay.jetty.servlet.Context@9647707{/static,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes/webapps/static/} 2015-11-22 19:29:54,524 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - starting ErrorHandler@3c22152a 2015-11-22 19:29:54,524 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - started ErrorHandler@3c22152a 2015-11-22 19:29:54,524 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - Holding class org.mortbay.jetty.servlet.DefaultServlet 2015-11-22 19:29:54,524 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - started org.mortbay.jetty.servlet.DefaultServlet-2126150723 2015-11-22 19:29:54,524 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - started org.mortbay.jetty.servlet.Context@9647707{/static,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes/webapps/static/} 2015-11-22 19:29:54,524 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - starting ContextHandlerCollection@3b039370 2015-11-22 19:29:54,524 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - started ContextHandlerCollection@3b039370 2015-11-22 19:29:54,524 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - starting Server@7ff1e948 2015-11-22 19:29:54,524 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - started org.mortbay.jetty.nio.SelectChannelConnector$1@1b6186d1 2015-11-22 19:29:54,524 INFO [RM-0] mortbay.log (Slf4jLog.java:info(67)) - Started HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:40017 2015-11-22 19:29:54,525 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - started HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:40017 2015-11-22 19:29:54,525 DEBUG [RM-0] mortbay.log (Slf4jLog.java:debug(40)) - started Server@7ff1e948 2015-11-22 19:29:54,525 INFO [RM-0] webapp.WebApps (WebApps.java:start(307)) - Web app cluster started at 40017 2015-11-22 19:29:54,526 DEBUG [RM-0] service.CompositeService (CompositeService.java:serviceStart(115)) - ResourceManager: starting services, size=4 2015-11-22 19:29:54,526 DEBUG [RM-0] service.AbstractService (AbstractService.java:start(197)) - Service Dispatcher is started 2015-11-22 19:29:54,526 DEBUG [RM-0] ipc.YarnRPC (YarnRPC.java:create(59)) - Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC 2015-11-22 19:29:54,526 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeLabelsUpdateSchedulerEvent.EventType: NODE_LABELS_UPDATE 2015-11-22 19:29:54,526 DEBUG [RM-0] ipc.HadoopYarnProtoRPC (HadoopYarnProtoRPC.java:getServer(62)) - Creating a HadoopYarnProtoRpc server for protocol interface org.apache.hadoop.yarn.server.api.ResourceManagerAdministrationProtocol with 1 handlers 2015-11-22 19:29:54,527 INFO [RM-0] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:29:54,527 DEBUG [RM-0] ipc.Server (Server.java:getAuthMethods(2447)) - Server accepts auth methods:[SIMPLE] 2015-11-22 19:29:54,527 INFO [Socket Reader #1 for port 38415] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 38415 2015-11-22 19:29:54,528 DEBUG [RM-0] metrics.RpcMetrics (RpcMetrics.java:(74)) - Initialized MetricsRegistry{info=MetricsInfoImpl{name=rpc, description=rpc}, tags=[MetricsTag{info=MetricsInfoImpl{name=port, description=RPC port}, value=38415}], metrics=[]} 2015-11-22 19:29:54,528 DEBUG [RM-0] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.ipc.metrics.RpcMetrics.receivedBytes with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Number of received bytes]) 2015-11-22 19:29:54,528 DEBUG [RM-0] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.ipc.metrics.RpcMetrics.sentBytes with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Number of sent bytes]) 2015-11-22 19:29:54,528 DEBUG [RM-0] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.ipc.metrics.RpcMetrics.rpcQueueTime with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Queue time]) 2015-11-22 19:29:54,528 DEBUG [RM-0] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.ipc.metrics.RpcMetrics.rpcProcessingTime with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Processing time]) 2015-11-22 19:29:54,528 DEBUG [RM-0] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.ipc.metrics.RpcMetrics.rpcAuthenticationFailures with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Number of authentication failures]) 2015-11-22 19:29:54,528 DEBUG [RM-0] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.ipc.metrics.RpcMetrics.rpcAuthenticationSuccesses with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Number of authentication successes]) 2015-11-22 19:29:54,529 DEBUG [RM-0] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.ipc.metrics.RpcMetrics.rpcAuthorizationFailures with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Number of authorization failures]) 2015-11-22 19:29:54,529 DEBUG [RM-0] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.ipc.metrics.RpcMetrics.rpcAuthorizationSuccesses with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Number of authorization successes]) 2015-11-22 19:29:54,529 DEBUG [RM-0] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.ipc.metrics.RpcMetrics.rpcClientBackoff with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Number of client backoff requests]) 2015-11-22 19:29:54,529 DEBUG [RM-0] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.ipc.metrics.RpcMetrics.rpcSlowCalls with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Number of Slow RPC calls]) 2015-11-22 19:29:54,529 DEBUG [RM-0] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForMethod(82)) - method public int org.apache.hadoop.ipc.metrics.RpcMetrics.numOpenConnections() with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Number of open connections]) 2015-11-22 19:29:54,529 DEBUG [RM-0] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForMethod(82)) - method public int org.apache.hadoop.ipc.metrics.RpcMetrics.callQueueLength() with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Length of the call queue]) 2015-11-22 19:29:54,529 DEBUG [RM-0] impl.MetricsSystemImpl (MetricsSystemImpl.java:register(232)) - RpcActivityForPort38415, Aggregate RPC metrics 2015-11-22 19:29:54,529 DEBUG [RM-0] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'PropertiesConfiguration' for key: source.source.start_mbeans 2015-11-22 19:29:54,529 DEBUG [RM-0] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'MetricsConfig' for key: source.start_mbeans 2015-11-22 19:29:54,529 DEBUG [RM-0] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'PropertiesConfiguration' for key: *.source.start_mbeans 2015-11-22 19:29:54,529 DEBUG [RM-0] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateAttrCache(254)) - Updating attr cache... 2015-11-22 19:29:54,530 DEBUG [RM-0] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateAttrCache(268)) - Done. # tags & metrics=17 2015-11-22 19:29:54,530 DEBUG [RM-0] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateInfoCache(248)) - Updating info cache... 2015-11-22 19:29:54,530 DEBUG [RM-0] impl.MetricsSystemImpl (MBeanInfoBuilder.java:get(109)) - [javax.management.MBeanAttributeInfo[description=RPC port, name=tag.port, type=java.lang.String, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Metrics context, name=tag.Context, type=java.lang.String, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Local hostname, name=tag.Hostname, type=java.lang.String, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of received bytes, name=ReceivedBytes, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of sent bytes, name=SentBytes, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of ops for queue time, name=RpcQueueTimeNumOps, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Average time for queue time, name=RpcQueueTimeAvgTime, type=java.lang.Double, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of ops for processing time, name=RpcProcessingTimeNumOps, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Average time for processing time, name=RpcProcessingTimeAvgTime, type=java.lang.Double, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of authentication failures, name=RpcAuthenticationFailures, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of authentication successes, name=RpcAuthenticationSuccesses, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of authorization failures, name=RpcAuthorizationFailures, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of authorization successes, name=RpcAuthorizationSuccesses, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of client backoff requests, name=RpcClientBackoff, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of Slow RPC calls, name=RpcSlowCalls, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of open connections, name=NumOpenConnections, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Length of the call queue, name=CallQueueLength, type=java.lang.Integer, read-only, descriptor={}]] 2015-11-22 19:29:54,530 DEBUG [RM-0] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateInfoCache(250)) - Done 2015-11-22 19:29:54,530 DEBUG [RM-0] util.MBeans (MBeans.java:register(67)) - Registered Hadoop:service=ApplicationHistoryServer,name=RpcActivityForPort38415 2015-11-22 19:29:54,530 DEBUG [RM-0] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:startMBeans(227)) - MBean for source RpcActivityForPort38415 registered. 2015-11-22 19:29:54,530 DEBUG [RM-0] impl.MetricsSystemImpl (MetricsSystemImpl.java:registerSource(269)) - Registered source RpcActivityForPort38415 2015-11-22 19:29:54,530 DEBUG [RM-0] metrics.RpcDetailedMetrics (RpcDetailedMetrics.java:(47)) - MetricsInfoImpl{name=rpcdetailed, description=rpcdetailed} 2015-11-22 19:29:54,530 DEBUG [RM-0] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableRates org.apache.hadoop.ipc.metrics.RpcDetailedMetrics.rates with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[]) 2015-11-22 19:29:54,530 DEBUG [RM-0] impl.MetricsSystemImpl (MetricsSystemImpl.java:register(232)) - RpcDetailedActivityForPort38415, Per method RPC metrics 2015-11-22 19:29:54,530 DEBUG [RM-0] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'PropertiesConfiguration' for key: source.source.start_mbeans 2015-11-22 19:29:54,531 DEBUG [RM-0] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'MetricsConfig' for key: source.start_mbeans 2015-11-22 19:29:54,531 DEBUG [RM-0] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'PropertiesConfiguration' for key: *.source.start_mbeans 2015-11-22 19:29:54,531 DEBUG [RM-0] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateAttrCache(254)) - Updating attr cache... 2015-11-22 19:29:54,531 DEBUG [RM-0] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateAttrCache(268)) - Done. # tags & metrics=3 2015-11-22 19:29:54,531 DEBUG [RM-0] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateInfoCache(248)) - Updating info cache... 2015-11-22 19:29:54,531 DEBUG [RM-0] impl.MetricsSystemImpl (MBeanInfoBuilder.java:get(109)) - [javax.management.MBeanAttributeInfo[description=RPC port, name=tag.port, type=java.lang.String, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Metrics context, name=tag.Context, type=java.lang.String, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Local hostname, name=tag.Hostname, type=java.lang.String, read-only, descriptor={}]] 2015-11-22 19:29:54,531 DEBUG [RM-0] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateInfoCache(250)) - Done 2015-11-22 19:29:54,531 DEBUG [RM-0] util.MBeans (MBeans.java:register(67)) - Registered Hadoop:service=ApplicationHistoryServer,name=RpcDetailedActivityForPort38415 2015-11-22 19:29:54,531 DEBUG [RM-0] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:startMBeans(227)) - MBean for source RpcDetailedActivityForPort38415 registered. 2015-11-22 19:29:54,531 DEBUG [RM-0] impl.MetricsSystemImpl (MetricsSystemImpl.java:registerSource(269)) - Registered source RpcDetailedActivityForPort38415 2015-11-22 19:29:54,531 DEBUG [RM-0] ipc.Server (RPC.java:registerProtocolAndImpl(876)) - RpcKind = RPC_PROTOCOL_BUFFER Protocol Name = org.apache.hadoop.ipc.ProtocolMetaInfoPB version=1 ProtocolImpl=org.apache.hadoop.ipc.protobuf.ProtocolInfoProtos$ProtocolInfoService$2 protocolClass=org.apache.hadoop.ipc.ProtocolMetaInfoPB 2015-11-22 19:29:54,531 DEBUG [RM-0] ipc.Server (RPC.java:registerProtocolAndImpl(876)) - RpcKind = RPC_PROTOCOL_BUFFER Protocol Name = org.apache.hadoop.yarn.server.api.ResourceManagerAdministrationProtocolPB version=1 ProtocolImpl=org.apache.hadoop.yarn.proto.ResourceManagerAdministrationProtocol$ResourceManagerAdministrationProtocolService$2 protocolClass=org.apache.hadoop.yarn.server.api.ResourceManagerAdministrationProtocolPB 2015-11-22 19:29:54,532 INFO [RM-0] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.server.api.ResourceManagerAdministrationProtocolPB to the server 2015-11-22 19:29:54,532 DEBUG [RM-0] ipc.Server (RPC.java:registerProtocolAndImpl(876)) - RpcKind = RPC_PROTOCOL_BUFFER Protocol Name = org.apache.hadoop.yarn.server.api.ResourceManagerAdministrationProtocolPB version=1 ProtocolImpl=org.apache.hadoop.yarn.proto.ResourceManagerAdministrationProtocol$ResourceManagerAdministrationProtocolService$2 protocolClass=org.apache.hadoop.yarn.server.api.ResourceManagerAdministrationProtocolPB 2015-11-22 19:29:54,532 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:29:54,532 INFO [IPC Server listener on 38415] ipc.Server (Server.java:run(794)) - IPC Server listener on 38415: starting 2015-11-22 19:29:54,533 DEBUG [RM-0] service.CompositeService (CompositeService.java:serviceStart(115)) - org.apache.hadoop.yarn.server.resourcemanager.AdminService: starting services, size=0 2015-11-22 19:29:54,533 DEBUG [RM-0] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.AdminService is started 2015-11-22 19:29:54,534 DEBUG [RM-0] service.CompositeService (CompositeService.java:serviceStart(115)) - org.apache.hadoop.yarn.server.resourcemanager.ahs.RMApplicationHistoryWriter: starting services, size=0 2015-11-22 19:29:54,534 DEBUG [RM-0] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.ahs.RMApplicationHistoryWriter is started 2015-11-22 19:29:54,534 DEBUG [RM-0] service.CompositeService (CompositeService.java:serviceStart(115)) - org.apache.hadoop.yarn.server.resourcemanager.metrics.SystemMetricsPublisher: starting services, size=0 2015-11-22 19:29:54,534 DEBUG [RM-0] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.resourcemanager.metrics.SystemMetricsPublisher is started 2015-11-22 19:29:54,534 DEBUG [RM-0] service.AbstractService (AbstractService.java:start(197)) - Service ResourceManager is started 2015-11-22 19:29:54,533 DEBUG [IPC Server handler 0 on 38415] ipc.Server (Server.java:run(2176)) - IPC Server handler 0 on 38415: starting 2015-11-22 19:29:54,739 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu sending #935 2015-11-22 19:29:54,739 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:close(1208)) - closing ipc connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028: null java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:392) at org.apache.hadoop.ipc.Client$Connection.receiveRpcResponse(Client.java:1110) at org.apache.hadoop.ipc.Client$Connection.run(Client.java:1005) 2015-11-22 19:29:54,739 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:close(1217)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu: closed 2015-11-22 19:29:54,739 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu] ipc.Client (Client.java:run(1018)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:52028 from ubuntu: stopped, remaining connections 0 2015-11-22 19:29:54,743 DEBUG [Thread-3684] retry.RetryInvocationHandler (RetryInvocationHandler.java:invoke(151)) - Exception while invoking getApplicationReport of class ApplicationClientProtocolPBClientImpl over null. Retrying after sleeping for 30000ms. java.io.EOFException: End of File Exception between local host is: "ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42"; destination host is: "ip-172-31-20-42.ap-northeast-1.compute.internal":52028; : java.io.EOFException; For more details see: http://wiki.apache.org/hadoop/EOFException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:422) at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:792) at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:765) at org.apache.hadoop.ipc.Client.call(Client.java:1452) at org.apache.hadoop.ipc.Client.call(Client.java:1385) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:230) at com.sun.proxy.$Proxy87.getApplicationReport(Unknown Source) at org.apache.hadoop.yarn.api.impl.pb.client.ApplicationClientProtocolPBClientImpl.getApplicationReport(ApplicationClientProtocolPBClientImpl.java:220) at sun.reflect.GeneratedMethodAccessor64.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:256) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:104) at com.sun.proxy.$Proxy88.getApplicationReport(Unknown Source) at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.getApplicationReport(YarnClientImpl.java:446) at org.apache.hadoop.yarn.applications.distributedshell.Client.monitorApplication(Client.java:740) at org.apache.hadoop.yarn.applications.distributedshell.Client.run(Client.java:715) at org.apache.hadoop.yarn.applications.distributedshell.TestDistributedShell.testDSShellWithCustomLogPropertyFile(TestDistributedShell.java:502) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74) Caused by: java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:392) at org.apache.hadoop.ipc.Client$Connection.receiveRpcResponse(Client.java:1110) at org.apache.hadoop.ipc.Client$Connection.run(Client.java:1005) 2015-11-22 19:29:55,780 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:startResourceManager(339)) - MiniYARN ResourceManager address: ip-172-31-20-42.ap-northeast-1.compute.internal:38037 2015-11-22 19:29:55,780 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:startResourceManager(341)) - MiniYARN ResourceManager web address: ip-172-31-20-42.ap-northeast-1.compute.internal:40017 2015-11-22 19:29:55,780 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(459)) - MiniYARN ResourceManager address: ip-172-31-20-42.ap-northeast-1.compute.internal:38037 2015-11-22 19:29:55,780 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(461)) - MiniYARN ResourceManager web address: ip-172-31-20-42.ap-northeast-1.compute.internal:40017 2015-11-22 19:29:55,780 DEBUG [main] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.MiniYARNCluster$ResourceManagerWrapper_0 is started 2015-11-22 19:29:55,781 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(575)) - Waiting for NM 0 to start... 2015-11-22 19:29:55,781 DEBUG [Thread-4056] service.CompositeService (CompositeService.java:serviceStart(115)) - NodeManager: starting services, size=7 2015-11-22 19:29:55,781 DEBUG [Thread-4056] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.nodemanager.DeletionService is started 2015-11-22 19:29:55,781 DEBUG [Thread-4056] service.CompositeService (CompositeService.java:serviceStart(115)) - org.apache.hadoop.yarn.server.nodemanager.NodeHealthCheckerService: starting services, size=1 2015-11-22 19:29:55,781 DEBUG [Thread-4056] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService is started 2015-11-22 19:29:55,781 DEBUG [Thread-4056] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.nodemanager.NodeHealthCheckerService is started 2015-11-22 19:29:55,781 DEBUG [Thread-4056] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.nodemanager.NodeResourceMonitorImpl is started 2015-11-22 19:29:55,782 DEBUG [Thread-4056] ipc.YarnRPC (YarnRPC.java:create(59)) - Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC 2015-11-22 19:29:55,782 DEBUG [Thread-4056] ipc.HadoopYarnProtoRPC (HadoopYarnProtoRPC.java:getServer(62)) - Creating a HadoopYarnProtoRpc server for protocol interface org.apache.hadoop.yarn.api.ContainerManagementProtocol with 20 handlers 2015-11-22 19:29:55,783 INFO [Thread-4056] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:29:55,783 DEBUG [Thread-4056] ipc.Server (Server.java:getAuthMethods(2447)) - Server accepts auth methods:[TOKEN] 2015-11-22 19:29:55,783 INFO [Socket Reader #1 for port 35735] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 35735 2015-11-22 19:29:55,784 DEBUG [Thread-4056] metrics.RpcMetrics (RpcMetrics.java:(74)) - Initialized MetricsRegistry{info=MetricsInfoImpl{name=rpc, description=rpc}, tags=[MetricsTag{info=MetricsInfoImpl{name=port, description=RPC port}, value=35735}], metrics=[]} 2015-11-22 19:29:55,784 DEBUG [Thread-4056] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.ipc.metrics.RpcMetrics.receivedBytes with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Number of received bytes]) 2015-11-22 19:29:55,784 DEBUG [Thread-4056] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.ipc.metrics.RpcMetrics.sentBytes with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Number of sent bytes]) 2015-11-22 19:29:55,785 DEBUG [Thread-4056] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.ipc.metrics.RpcMetrics.rpcQueueTime with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Queue time]) 2015-11-22 19:29:55,785 DEBUG [Thread-4056] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.ipc.metrics.RpcMetrics.rpcProcessingTime with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Processing time]) 2015-11-22 19:29:55,785 DEBUG [Thread-4056] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.ipc.metrics.RpcMetrics.rpcAuthenticationFailures with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Number of authentication failures]) 2015-11-22 19:29:55,785 DEBUG [Thread-4056] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.ipc.metrics.RpcMetrics.rpcAuthenticationSuccesses with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Number of authentication successes]) 2015-11-22 19:29:55,785 DEBUG [Thread-4056] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.ipc.metrics.RpcMetrics.rpcAuthorizationFailures with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Number of authorization failures]) 2015-11-22 19:29:55,785 DEBUG [Thread-4056] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.ipc.metrics.RpcMetrics.rpcAuthorizationSuccesses with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Number of authorization successes]) 2015-11-22 19:29:55,785 DEBUG [Thread-4056] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.ipc.metrics.RpcMetrics.rpcClientBackoff with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Number of client backoff requests]) 2015-11-22 19:29:55,786 DEBUG [Thread-4056] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.ipc.metrics.RpcMetrics.rpcSlowCalls with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Number of Slow RPC calls]) 2015-11-22 19:29:55,786 DEBUG [Thread-4056] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForMethod(82)) - method public int org.apache.hadoop.ipc.metrics.RpcMetrics.numOpenConnections() with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Number of open connections]) 2015-11-22 19:29:55,786 DEBUG [Thread-4056] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForMethod(82)) - method public int org.apache.hadoop.ipc.metrics.RpcMetrics.callQueueLength() with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Length of the call queue]) 2015-11-22 19:29:55,786 DEBUG [Thread-4056] impl.MetricsSystemImpl (MetricsSystemImpl.java:register(232)) - RpcActivityForPort35735, Aggregate RPC metrics 2015-11-22 19:29:55,786 DEBUG [Thread-4056] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'PropertiesConfiguration' for key: source.source.start_mbeans 2015-11-22 19:29:55,786 DEBUG [Thread-4056] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'MetricsConfig' for key: source.start_mbeans 2015-11-22 19:29:55,786 DEBUG [Thread-4056] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'PropertiesConfiguration' for key: *.source.start_mbeans 2015-11-22 19:29:55,786 DEBUG [Thread-4056] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateAttrCache(254)) - Updating attr cache... 2015-11-22 19:29:55,787 DEBUG [Thread-4056] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateAttrCache(268)) - Done. # tags & metrics=17 2015-11-22 19:29:55,787 DEBUG [Thread-4056] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateInfoCache(248)) - Updating info cache... 2015-11-22 19:29:55,787 DEBUG [Thread-4056] impl.MetricsSystemImpl (MBeanInfoBuilder.java:get(109)) - [javax.management.MBeanAttributeInfo[description=RPC port, name=tag.port, type=java.lang.String, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Metrics context, name=tag.Context, type=java.lang.String, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Local hostname, name=tag.Hostname, type=java.lang.String, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of received bytes, name=ReceivedBytes, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of sent bytes, name=SentBytes, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of ops for queue time, name=RpcQueueTimeNumOps, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Average time for queue time, name=RpcQueueTimeAvgTime, type=java.lang.Double, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of ops for processing time, name=RpcProcessingTimeNumOps, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Average time for processing time, name=RpcProcessingTimeAvgTime, type=java.lang.Double, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of authentication failures, name=RpcAuthenticationFailures, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of authentication successes, name=RpcAuthenticationSuccesses, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of authorization failures, name=RpcAuthorizationFailures, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of authorization successes, name=RpcAuthorizationSuccesses, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of client backoff requests, name=RpcClientBackoff, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of Slow RPC calls, name=RpcSlowCalls, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of open connections, name=NumOpenConnections, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Length of the call queue, name=CallQueueLength, type=java.lang.Integer, read-only, descriptor={}]] 2015-11-22 19:29:55,787 DEBUG [Thread-4056] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateInfoCache(250)) - Done 2015-11-22 19:29:55,787 DEBUG [Thread-4056] util.MBeans (MBeans.java:register(67)) - Registered Hadoop:service=ApplicationHistoryServer,name=RpcActivityForPort35735 2015-11-22 19:29:55,787 DEBUG [Thread-4056] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:startMBeans(227)) - MBean for source RpcActivityForPort35735 registered. 2015-11-22 19:29:55,787 DEBUG [Thread-4056] impl.MetricsSystemImpl (MetricsSystemImpl.java:registerSource(269)) - Registered source RpcActivityForPort35735 2015-11-22 19:29:55,788 DEBUG [Thread-4056] metrics.RpcDetailedMetrics (RpcDetailedMetrics.java:(47)) - MetricsInfoImpl{name=rpcdetailed, description=rpcdetailed} 2015-11-22 19:29:55,788 DEBUG [Thread-4056] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableRates org.apache.hadoop.ipc.metrics.RpcDetailedMetrics.rates with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[]) 2015-11-22 19:29:55,788 DEBUG [Thread-4056] impl.MetricsSystemImpl (MetricsSystemImpl.java:register(232)) - RpcDetailedActivityForPort35735, Per method RPC metrics 2015-11-22 19:29:55,788 DEBUG [Thread-4056] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'PropertiesConfiguration' for key: source.source.start_mbeans 2015-11-22 19:29:55,788 DEBUG [Thread-4056] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'MetricsConfig' for key: source.start_mbeans 2015-11-22 19:29:55,788 DEBUG [Thread-4056] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'PropertiesConfiguration' for key: *.source.start_mbeans 2015-11-22 19:29:55,788 DEBUG [Thread-4056] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateAttrCache(254)) - Updating attr cache... 2015-11-22 19:29:55,788 DEBUG [Thread-4056] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateAttrCache(268)) - Done. # tags & metrics=3 2015-11-22 19:29:55,789 DEBUG [Thread-4056] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateInfoCache(248)) - Updating info cache... 2015-11-22 19:29:55,789 DEBUG [Thread-4056] impl.MetricsSystemImpl (MBeanInfoBuilder.java:get(109)) - [javax.management.MBeanAttributeInfo[description=RPC port, name=tag.port, type=java.lang.String, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Metrics context, name=tag.Context, type=java.lang.String, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Local hostname, name=tag.Hostname, type=java.lang.String, read-only, descriptor={}]] 2015-11-22 19:29:55,789 DEBUG [Thread-4056] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateInfoCache(250)) - Done 2015-11-22 19:29:55,789 DEBUG [Thread-4056] util.MBeans (MBeans.java:register(67)) - Registered Hadoop:service=ApplicationHistoryServer,name=RpcDetailedActivityForPort35735 2015-11-22 19:29:55,789 DEBUG [Thread-4056] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:startMBeans(227)) - MBean for source RpcDetailedActivityForPort35735 registered. 2015-11-22 19:29:55,789 DEBUG [Thread-4056] impl.MetricsSystemImpl (MetricsSystemImpl.java:registerSource(269)) - Registered source RpcDetailedActivityForPort35735 2015-11-22 19:29:55,790 DEBUG [Thread-4056] ipc.Server (RPC.java:registerProtocolAndImpl(876)) - RpcKind = RPC_PROTOCOL_BUFFER Protocol Name = org.apache.hadoop.ipc.ProtocolMetaInfoPB version=1 ProtocolImpl=org.apache.hadoop.ipc.protobuf.ProtocolInfoProtos$ProtocolInfoService$2 protocolClass=org.apache.hadoop.ipc.ProtocolMetaInfoPB 2015-11-22 19:29:55,790 DEBUG [Thread-4056] ipc.Server (RPC.java:registerProtocolAndImpl(876)) - RpcKind = RPC_PROTOCOL_BUFFER Protocol Name = org.apache.hadoop.yarn.api.ContainerManagementProtocolPB version=1 ProtocolImpl=org.apache.hadoop.yarn.proto.ContainerManagementProtocol$ContainerManagementProtocolService$2 protocolClass=org.apache.hadoop.yarn.api.ContainerManagementProtocolPB 2015-11-22 19:29:55,790 INFO [Thread-4056] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.api.ContainerManagementProtocolPB to the server 2015-11-22 19:29:55,790 DEBUG [Thread-4056] ipc.Server (RPC.java:registerProtocolAndImpl(876)) - RpcKind = RPC_PROTOCOL_BUFFER Protocol Name = org.apache.hadoop.yarn.api.ContainerManagementProtocolPB version=1 ProtocolImpl=org.apache.hadoop.yarn.proto.ContainerManagementProtocol$ContainerManagementProtocolService$2 protocolClass=org.apache.hadoop.yarn.api.ContainerManagementProtocolPB 2015-11-22 19:29:55,790 INFO [Thread-4056] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:serviceStart(464)) - Blocking new container-requests as container manager rpc server is still starting. 2015-11-22 19:29:55,790 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:29:55,791 DEBUG [IPC Server handler 0 on 35735] ipc.Server (Server.java:run(2176)) - IPC Server handler 0 on 35735: starting 2015-11-22 19:29:55,791 INFO [IPC Server listener on 35735] ipc.Server (Server.java:run(794)) - IPC Server listener on 35735: starting 2015-11-22 19:29:55,791 DEBUG [IPC Server handler 2 on 35735] ipc.Server (Server.java:run(2176)) - IPC Server handler 2 on 35735: starting 2015-11-22 19:29:55,796 DEBUG [IPC Server handler 3 on 35735] ipc.Server (Server.java:run(2176)) - IPC Server handler 3 on 35735: starting 2015-11-22 19:29:55,796 DEBUG [IPC Server handler 4 on 35735] ipc.Server (Server.java:run(2176)) - IPC Server handler 4 on 35735: starting 2015-11-22 19:29:55,796 DEBUG [IPC Server handler 5 on 35735] ipc.Server (Server.java:run(2176)) - IPC Server handler 5 on 35735: starting 2015-11-22 19:29:55,796 DEBUG [IPC Server handler 1 on 35735] ipc.Server (Server.java:run(2176)) - IPC Server handler 1 on 35735: starting 2015-11-22 19:29:55,796 DEBUG [IPC Server handler 6 on 35735] ipc.Server (Server.java:run(2176)) - IPC Server handler 6 on 35735: starting 2015-11-22 19:29:55,797 DEBUG [IPC Server handler 7 on 35735] ipc.Server (Server.java:run(2176)) - IPC Server handler 7 on 35735: starting 2015-11-22 19:29:55,797 DEBUG [IPC Server handler 8 on 35735] ipc.Server (Server.java:run(2176)) - IPC Server handler 8 on 35735: starting 2015-11-22 19:29:55,797 DEBUG [IPC Server handler 9 on 35735] ipc.Server (Server.java:run(2176)) - IPC Server handler 9 on 35735: starting 2015-11-22 19:29:55,800 DEBUG [IPC Server handler 10 on 35735] ipc.Server (Server.java:run(2176)) - IPC Server handler 10 on 35735: starting 2015-11-22 19:29:55,800 DEBUG [IPC Server handler 12 on 35735] ipc.Server (Server.java:run(2176)) - IPC Server handler 12 on 35735: starting 2015-11-22 19:29:55,800 DEBUG [IPC Server handler 13 on 35735] ipc.Server (Server.java:run(2176)) - IPC Server handler 13 on 35735: starting 2015-11-22 19:29:55,800 DEBUG [IPC Server handler 14 on 35735] ipc.Server (Server.java:run(2176)) - IPC Server handler 14 on 35735: starting 2015-11-22 19:29:55,801 DEBUG [IPC Server handler 15 on 35735] ipc.Server (Server.java:run(2176)) - IPC Server handler 15 on 35735: starting 2015-11-22 19:29:55,801 DEBUG [IPC Server handler 16 on 35735] ipc.Server (Server.java:run(2176)) - IPC Server handler 16 on 35735: starting 2015-11-22 19:29:55,801 DEBUG [IPC Server handler 11 on 35735] ipc.Server (Server.java:run(2176)) - IPC Server handler 11 on 35735: starting 2015-11-22 19:29:55,801 DEBUG [IPC Server handler 17 on 35735] ipc.Server (Server.java:run(2176)) - IPC Server handler 17 on 35735: starting 2015-11-22 19:29:55,801 DEBUG [IPC Server handler 18 on 35735] ipc.Server (Server.java:run(2176)) - IPC Server handler 18 on 35735: starting 2015-11-22 19:29:55,801 DEBUG [Thread-4056] security.NMTokenSecretManagerInNM (NMTokenSecretManagerInNM.java:setNodeId(243)) - updating nodeId : ip-172-31-20-42.ap-northeast-1.compute.internal:35735 2015-11-22 19:29:55,801 INFO [Thread-4056] security.NMContainerTokenSecretManager (NMContainerTokenSecretManager.java:setNodeId(260)) - Updating node address : ip-172-31-20-42.ap-northeast-1.compute.internal:35735 2015-11-22 19:29:55,802 DEBUG [Thread-4056] service.CompositeService (CompositeService.java:serviceStart(115)) - org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl: starting services, size=7 2015-11-22 19:29:55,801 DEBUG [IPC Server handler 19 on 35735] ipc.Server (Server.java:run(2176)) - IPC Server handler 19 on 35735: starting 2015-11-22 19:29:55,802 DEBUG [Thread-4056] ipc.YarnRPC (YarnRPC.java:create(59)) - Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC 2015-11-22 19:29:55,802 DEBUG [Thread-4056] ipc.HadoopYarnProtoRPC (HadoopYarnProtoRPC.java:getServer(62)) - Creating a HadoopYarnProtoRpc server for protocol interface org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocol with 5 handlers 2015-11-22 19:29:55,802 INFO [Thread-4056] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:29:55,802 DEBUG [Thread-4056] ipc.Server (Server.java:getAuthMethods(2447)) - Server accepts auth methods:[SIMPLE] 2015-11-22 19:29:55,802 INFO [Socket Reader #1 for port 40868] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 40868 2015-11-22 19:29:55,803 DEBUG [Thread-4056] metrics.RpcMetrics (RpcMetrics.java:(74)) - Initialized MetricsRegistry{info=MetricsInfoImpl{name=rpc, description=rpc}, tags=[MetricsTag{info=MetricsInfoImpl{name=port, description=RPC port}, value=40868}], metrics=[]} 2015-11-22 19:29:55,803 DEBUG [Thread-4056] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.ipc.metrics.RpcMetrics.receivedBytes with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Number of received bytes]) 2015-11-22 19:29:55,803 DEBUG [Thread-4056] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.ipc.metrics.RpcMetrics.sentBytes with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Number of sent bytes]) 2015-11-22 19:29:55,803 DEBUG [Thread-4056] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.ipc.metrics.RpcMetrics.rpcQueueTime with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Queue time]) 2015-11-22 19:29:55,803 DEBUG [Thread-4056] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.ipc.metrics.RpcMetrics.rpcProcessingTime with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Processing time]) 2015-11-22 19:29:55,804 DEBUG [Thread-4056] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.ipc.metrics.RpcMetrics.rpcAuthenticationFailures with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Number of authentication failures]) 2015-11-22 19:29:55,804 DEBUG [Thread-4056] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.ipc.metrics.RpcMetrics.rpcAuthenticationSuccesses with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Number of authentication successes]) 2015-11-22 19:29:55,804 DEBUG [Thread-4056] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.ipc.metrics.RpcMetrics.rpcAuthorizationFailures with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Number of authorization failures]) 2015-11-22 19:29:55,804 DEBUG [Thread-4056] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.ipc.metrics.RpcMetrics.rpcAuthorizationSuccesses with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Number of authorization successes]) 2015-11-22 19:29:55,804 DEBUG [Thread-4056] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.ipc.metrics.RpcMetrics.rpcClientBackoff with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Number of client backoff requests]) 2015-11-22 19:29:55,804 DEBUG [Thread-4056] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.ipc.metrics.RpcMetrics.rpcSlowCalls with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Number of Slow RPC calls]) 2015-11-22 19:29:55,804 DEBUG [Thread-4056] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForMethod(82)) - method public int org.apache.hadoop.ipc.metrics.RpcMetrics.numOpenConnections() with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Number of open connections]) 2015-11-22 19:29:55,804 DEBUG [Thread-4056] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForMethod(82)) - method public int org.apache.hadoop.ipc.metrics.RpcMetrics.callQueueLength() with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Length of the call queue]) 2015-11-22 19:29:55,804 DEBUG [Thread-4056] impl.MetricsSystemImpl (MetricsSystemImpl.java:register(232)) - RpcActivityForPort40868, Aggregate RPC metrics 2015-11-22 19:29:55,805 DEBUG [Thread-4056] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'PropertiesConfiguration' for key: source.source.start_mbeans 2015-11-22 19:29:55,805 DEBUG [Thread-4056] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'MetricsConfig' for key: source.start_mbeans 2015-11-22 19:29:55,805 DEBUG [Thread-4056] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'PropertiesConfiguration' for key: *.source.start_mbeans 2015-11-22 19:29:55,805 DEBUG [Thread-4056] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateAttrCache(254)) - Updating attr cache... 2015-11-22 19:29:55,805 DEBUG [Thread-4056] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateAttrCache(268)) - Done. # tags & metrics=17 2015-11-22 19:29:55,805 DEBUG [Thread-4056] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateInfoCache(248)) - Updating info cache... 2015-11-22 19:29:55,805 DEBUG [Thread-4056] impl.MetricsSystemImpl (MBeanInfoBuilder.java:get(109)) - [javax.management.MBeanAttributeInfo[description=RPC port, name=tag.port, type=java.lang.String, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Metrics context, name=tag.Context, type=java.lang.String, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Local hostname, name=tag.Hostname, type=java.lang.String, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of received bytes, name=ReceivedBytes, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of sent bytes, name=SentBytes, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of ops for queue time, name=RpcQueueTimeNumOps, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Average time for queue time, name=RpcQueueTimeAvgTime, type=java.lang.Double, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of ops for processing time, name=RpcProcessingTimeNumOps, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Average time for processing time, name=RpcProcessingTimeAvgTime, type=java.lang.Double, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of authentication failures, name=RpcAuthenticationFailures, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of authentication successes, name=RpcAuthenticationSuccesses, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of authorization failures, name=RpcAuthorizationFailures, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of authorization successes, name=RpcAuthorizationSuccesses, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of client backoff requests, name=RpcClientBackoff, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of Slow RPC calls, name=RpcSlowCalls, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of open connections, name=NumOpenConnections, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Length of the call queue, name=CallQueueLength, type=java.lang.Integer, read-only, descriptor={}]] 2015-11-22 19:29:55,805 DEBUG [Thread-4056] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateInfoCache(250)) - Done 2015-11-22 19:29:55,806 DEBUG [Thread-4056] util.MBeans (MBeans.java:register(67)) - Registered Hadoop:service=ApplicationHistoryServer,name=RpcActivityForPort40868 2015-11-22 19:29:55,806 DEBUG [Thread-4056] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:startMBeans(227)) - MBean for source RpcActivityForPort40868 registered. 2015-11-22 19:29:55,806 DEBUG [Thread-4056] impl.MetricsSystemImpl (MetricsSystemImpl.java:registerSource(269)) - Registered source RpcActivityForPort40868 2015-11-22 19:29:55,806 DEBUG [Thread-4056] metrics.RpcDetailedMetrics (RpcDetailedMetrics.java:(47)) - MetricsInfoImpl{name=rpcdetailed, description=rpcdetailed} 2015-11-22 19:29:55,806 DEBUG [Thread-4056] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableRates org.apache.hadoop.ipc.metrics.RpcDetailedMetrics.rates with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[]) 2015-11-22 19:29:55,806 DEBUG [Thread-4056] impl.MetricsSystemImpl (MetricsSystemImpl.java:register(232)) - RpcDetailedActivityForPort40868, Per method RPC metrics 2015-11-22 19:29:55,806 DEBUG [Thread-4056] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'PropertiesConfiguration' for key: source.source.start_mbeans 2015-11-22 19:29:55,806 DEBUG [Thread-4056] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'MetricsConfig' for key: source.start_mbeans 2015-11-22 19:29:55,806 DEBUG [Thread-4056] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'PropertiesConfiguration' for key: *.source.start_mbeans 2015-11-22 19:29:55,807 DEBUG [Thread-4056] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateAttrCache(254)) - Updating attr cache... 2015-11-22 19:29:55,807 DEBUG [Thread-4056] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateAttrCache(268)) - Done. # tags & metrics=3 2015-11-22 19:29:55,807 DEBUG [Thread-4056] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateInfoCache(248)) - Updating info cache... 2015-11-22 19:29:55,807 DEBUG [Thread-4056] impl.MetricsSystemImpl (MBeanInfoBuilder.java:get(109)) - [javax.management.MBeanAttributeInfo[description=RPC port, name=tag.port, type=java.lang.String, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Metrics context, name=tag.Context, type=java.lang.String, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Local hostname, name=tag.Hostname, type=java.lang.String, read-only, descriptor={}]] 2015-11-22 19:29:55,807 DEBUG [Thread-4056] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateInfoCache(250)) - Done 2015-11-22 19:29:55,807 DEBUG [Thread-4056] util.MBeans (MBeans.java:register(67)) - Registered Hadoop:service=ApplicationHistoryServer,name=RpcDetailedActivityForPort40868 2015-11-22 19:29:55,807 DEBUG [Thread-4056] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:startMBeans(227)) - MBean for source RpcDetailedActivityForPort40868 registered. 2015-11-22 19:29:55,807 DEBUG [Thread-4056] impl.MetricsSystemImpl (MetricsSystemImpl.java:registerSource(269)) - Registered source RpcDetailedActivityForPort40868 2015-11-22 19:29:55,807 DEBUG [Thread-4056] ipc.Server (RPC.java:registerProtocolAndImpl(876)) - RpcKind = RPC_PROTOCOL_BUFFER Protocol Name = org.apache.hadoop.ipc.ProtocolMetaInfoPB version=1 ProtocolImpl=org.apache.hadoop.ipc.protobuf.ProtocolInfoProtos$ProtocolInfoService$2 protocolClass=org.apache.hadoop.ipc.ProtocolMetaInfoPB 2015-11-22 19:29:55,808 DEBUG [Thread-4056] ipc.Server (RPC.java:registerProtocolAndImpl(876)) - RpcKind = RPC_PROTOCOL_BUFFER Protocol Name = org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB version=1 ProtocolImpl=org.apache.hadoop.yarn.proto.LocalizationProtocol$LocalizationProtocolService$2 protocolClass=org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB 2015-11-22 19:29:55,808 INFO [Thread-4056] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB to the server 2015-11-22 19:29:55,808 DEBUG [Thread-4056] ipc.Server (RPC.java:registerProtocolAndImpl(876)) - RpcKind = RPC_PROTOCOL_BUFFER Protocol Name = org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB version=1 ProtocolImpl=org.apache.hadoop.yarn.proto.LocalizationProtocol$LocalizationProtocolService$2 protocolClass=org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB 2015-11-22 19:29:55,808 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:29:55,808 INFO [IPC Server listener on 40868] ipc.Server (Server.java:run(794)) - IPC Server listener on 40868: starting 2015-11-22 19:29:55,812 DEBUG [IPC Server handler 0 on 40868] ipc.Server (Server.java:run(2176)) - IPC Server handler 0 on 40868: starting 2015-11-22 19:29:55,816 DEBUG [IPC Server handler 3 on 40868] ipc.Server (Server.java:run(2176)) - IPC Server handler 3 on 40868: starting 2015-11-22 19:29:55,812 DEBUG [IPC Server handler 1 on 40868] ipc.Server (Server.java:run(2176)) - IPC Server handler 1 on 40868: starting 2015-11-22 19:29:55,816 DEBUG [IPC Server handler 4 on 40868] ipc.Server (Server.java:run(2176)) - IPC Server handler 4 on 40868: starting 2015-11-22 19:29:55,816 DEBUG [IPC Server handler 2 on 40868] ipc.Server (Server.java:run(2176)) - IPC Server handler 2 on 40868: starting 2015-11-22 19:29:55,816 INFO [Thread-4056] localizer.ResourceLocalizationService (ResourceLocalizationService.java:serviceStart(363)) - Localizer started on port 40868 2015-11-22 19:29:55,820 DEBUG [Thread-4056] service.CompositeService (CompositeService.java:serviceStart(115)) - org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: starting services, size=1 2015-11-22 19:29:55,820 DEBUG [Thread-4056] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerTracker is started 2015-11-22 19:29:55,827 DEBUG [Thread-4056] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService is started 2015-11-22 19:29:55,827 DEBUG [Thread-4056] service.AbstractService (AbstractService.java:start(197)) - Service containers-launcher is started 2015-11-22 19:29:55,827 DEBUG [Thread-4056] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices is started 2015-11-22 19:29:55,828 DEBUG [Thread-4056] service.AbstractService (AbstractService.java:start(197)) - Service containers-monitor is started 2015-11-22 19:29:55,828 DEBUG [Thread-4056] service.AbstractService (AbstractService.java:start(197)) - Service Dispatcher is started 2015-11-22 19:29:55,828 DEBUG [Thread-4056] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.NonAggregatingLogHandler is started 2015-11-22 19:29:55,828 DEBUG [Thread-4056] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.sharedcache.SharedCacheUploadService is started 2015-11-22 19:29:55,828 INFO [Thread-4056] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:serviceStart(509)) - ContainerManager started at /172.31.20.42:35735 2015-11-22 19:29:55,828 INFO [Thread-4056] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:serviceStart(510)) - ContainerManager bound to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:0 2015-11-22 19:29:55,828 DEBUG [Thread-4056] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl is started 2015-11-22 19:29:55,828 INFO [Thread-4056] webapp.WebServer (WebServer.java:serviceStart(71)) - Instantiating NMWebApp at ip-172-31-20-42.ap-northeast-1.compute.internal:0 2015-11-22 19:29:55,829 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap=null 2015-11-22 19:29:55,829 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=null 2015-11-22 19:29:55,829 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:55,829 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/=org.mortbay.jetty.servlet.DefaultServlet-608661613} 2015-11-22 19:29:55,829 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-608661613=org.mortbay.jetty.servlet.DefaultServlet-608661613} 2015-11-22 19:29:55,829 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={NoCacheFilter=NoCacheFilter} 2015-11-22 19:29:55,829 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15)] 2015-11-22 19:29:55,829 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:55,829 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/=org.mortbay.jetty.servlet.DefaultServlet-608661613} 2015-11-22 19:29:55,830 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-608661613=org.mortbay.jetty.servlet.DefaultServlet-608661613} 2015-11-22 19:29:55,830 INFO [Thread-4056] server.AuthenticationFilter (AuthenticationFilter.java:constructSecretProvider(304)) - Unable to initialize FileSignerSecretProvider, falling back to use random secrets. 2015-11-22 19:29:55,830 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container Server@23e7fedb + org.mortbay.thread.QueuedThreadPool@7d421cea as threadpool 2015-11-22 19:29:55,831 INFO [Thread-4056] http.HttpRequestLog (HttpRequestLog.java:getRequestLog(80)) - Http request log for http.requests.nodemanager is not defined 2015-11-22 19:29:55,831 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container Server@23e7fedb + ContextHandlerCollection@3c3c36d2 as handler 2015-11-22 19:29:55,831 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container ContextHandlerCollection@3c3c36d2 + org.mortbay.jetty.webapp.WebAppContext@1f7a38df{/,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes/webapps/node} as handler 2015-11-22 19:29:55,831 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 + NoCacheFilter as filter 2015-11-22 19:29:55,831 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 + (F=NoCacheFilter,[/*],[],15) as filterMapping 2015-11-22 19:29:55,831 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 + org.mortbay.jetty.servlet.DefaultServlet-608661613 as servlet 2015-11-22 19:29:55,831 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 + (S=org.mortbay.jetty.servlet.DefaultServlet-608661613,[/]) as servletMapping 2015-11-22 19:29:55,831 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container SecurityHandler@6ebffb2d + ServletHandler@7232d296 as handler 2015-11-22 19:29:55,832 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container SessionHandler@3368da8e + SecurityHandler@6ebffb2d as handler 2015-11-22 19:29:55,832 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container SessionHandler@3368da8e + org.mortbay.jetty.servlet.HashSessionManager@7d1aa0fc as sessionManager 2015-11-22 19:29:55,832 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container org.mortbay.jetty.webapp.WebAppContext@1f7a38df{/,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes/webapps/node} + SessionHandler@3368da8e as handler 2015-11-22 19:29:55,832 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container org.mortbay.jetty.webapp.WebAppContext@1f7a38df{/,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes/webapps/node} + ErrorPageErrorHandler@19d87bab as error 2015-11-22 19:29:55,832 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container ContextHandlerCollection@3c3c36d2 + org.mortbay.jetty.servlet.Context@82f9d2e{/logs,null} as handler 2015-11-22 19:29:55,832 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container org.mortbay.jetty.servlet.Context@82f9d2e{/logs,null} + ServletHandler@7de68a0f as handler 2015-11-22 19:29:55,832 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7de68a0f + org.apache.hadoop.http.AdminAuthorizedServlet-1439946767 as servlet 2015-11-22 19:29:55,832 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7de68a0f + (S=org.apache.hadoop.http.AdminAuthorizedServlet-1439946767,[/*]) as servletMapping 2015-11-22 19:29:55,832 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap=null 2015-11-22 19:29:55,833 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=null 2015-11-22 19:29:55,833 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:55,833 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/*=org.apache.hadoop.http.AdminAuthorizedServlet-1439946767} 2015-11-22 19:29:55,833 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.apache.hadoop.http.AdminAuthorizedServlet-1439946767=org.apache.hadoop.http.AdminAuthorizedServlet-1439946767} 2015-11-22 19:29:55,833 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 + NoCacheFilter as filter 2015-11-22 19:29:55,833 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 + (F=NoCacheFilter,[/*],[],15) as filterMapping 2015-11-22 19:29:55,833 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={NoCacheFilter=NoCacheFilter} 2015-11-22 19:29:55,833 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15)] 2015-11-22 19:29:55,833 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:55,833 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/=org.mortbay.jetty.servlet.DefaultServlet-608661613} 2015-11-22 19:29:55,834 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-608661613=org.mortbay.jetty.servlet.DefaultServlet-608661613} 2015-11-22 19:29:55,834 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container ContextHandlerCollection@3c3c36d2 + org.mortbay.jetty.servlet.Context@f8cf43d{/static,null} as handler 2015-11-22 19:29:55,834 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container org.mortbay.jetty.servlet.Context@f8cf43d{/static,null} + ServletHandler@7b3ede0f as handler 2015-11-22 19:29:55,834 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7b3ede0f + org.mortbay.jetty.servlet.DefaultServlet-851547541 as servlet 2015-11-22 19:29:55,834 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7b3ede0f + (S=org.mortbay.jetty.servlet.DefaultServlet-851547541,[/*]) as servletMapping 2015-11-22 19:29:55,834 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap=null 2015-11-22 19:29:55,834 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=null 2015-11-22 19:29:55,834 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:55,834 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/*=org.mortbay.jetty.servlet.DefaultServlet-851547541} 2015-11-22 19:29:55,835 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-851547541=org.mortbay.jetty.servlet.DefaultServlet-851547541} 2015-11-22 19:29:55,835 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 + safety as filter 2015-11-22 19:29:55,835 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 + (F=safety,[/*],[],15) as filterMapping 2015-11-22 19:29:55,835 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={safety=safety, NoCacheFilter=NoCacheFilter} 2015-11-22 19:29:55,835 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15)] 2015-11-22 19:29:55,835 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:55,835 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/=org.mortbay.jetty.servlet.DefaultServlet-608661613} 2015-11-22 19:29:55,835 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-608661613=org.mortbay.jetty.servlet.DefaultServlet-608661613} 2015-11-22 19:29:55,835 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7de68a0f + safety as filter 2015-11-22 19:29:55,835 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7de68a0f + (F=safety,[/*],[],15) as filterMapping 2015-11-22 19:29:55,835 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={safety=safety} 2015-11-22 19:29:55,836 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=safety,[/*],[],15)] 2015-11-22 19:29:55,836 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:55,836 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/*=org.apache.hadoop.http.AdminAuthorizedServlet-1439946767} 2015-11-22 19:29:55,836 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.apache.hadoop.http.AdminAuthorizedServlet-1439946767=org.apache.hadoop.http.AdminAuthorizedServlet-1439946767} 2015-11-22 19:29:55,836 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7b3ede0f + safety as filter 2015-11-22 19:29:55,836 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7b3ede0f + (F=safety,[/*],[],15) as filterMapping 2015-11-22 19:29:55,836 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={safety=safety} 2015-11-22 19:29:55,836 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=safety,[/*],[],15)] 2015-11-22 19:29:55,836 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:55,836 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/*=org.mortbay.jetty.servlet.DefaultServlet-851547541} 2015-11-22 19:29:55,837 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-851547541=org.mortbay.jetty.servlet.DefaultServlet-851547541} 2015-11-22 19:29:55,837 INFO [Thread-4056] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter) 2015-11-22 19:29:55,837 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 + RMAuthenticationFilter as filter 2015-11-22 19:29:55,837 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 + (F=RMAuthenticationFilter,[*.html, *.jsp],[],15) as filterMapping 2015-11-22 19:29:55,837 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter} 2015-11-22 19:29:55,837 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15)] 2015-11-22 19:29:55,837 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:55,838 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/=org.mortbay.jetty.servlet.DefaultServlet-608661613} 2015-11-22 19:29:55,838 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-608661613=org.mortbay.jetty.servlet.DefaultServlet-608661613} 2015-11-22 19:29:55,838 INFO [Thread-4056] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context node 2015-11-22 19:29:55,838 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7de68a0f + RMAuthenticationFilter as filter 2015-11-22 19:29:55,838 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7de68a0f + (F=RMAuthenticationFilter,[/*],[],15) as filterMapping 2015-11-22 19:29:55,838 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={safety=safety, RMAuthenticationFilter=RMAuthenticationFilter} 2015-11-22 19:29:55,838 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=safety,[/*],[],15), (F=RMAuthenticationFilter,[/*],[],15)] 2015-11-22 19:29:55,838 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:55,838 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/*=org.apache.hadoop.http.AdminAuthorizedServlet-1439946767} 2015-11-22 19:29:55,838 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.apache.hadoop.http.AdminAuthorizedServlet-1439946767=org.apache.hadoop.http.AdminAuthorizedServlet-1439946767} 2015-11-22 19:29:55,839 INFO [Thread-4056] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context logs 2015-11-22 19:29:55,839 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7b3ede0f + RMAuthenticationFilter as filter 2015-11-22 19:29:55,839 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7b3ede0f + (F=RMAuthenticationFilter,[/*],[],15) as filterMapping 2015-11-22 19:29:55,839 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={safety=safety, RMAuthenticationFilter=RMAuthenticationFilter} 2015-11-22 19:29:55,839 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=safety,[/*],[],15), (F=RMAuthenticationFilter,[/*],[],15)] 2015-11-22 19:29:55,839 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:55,839 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/*=org.mortbay.jetty.servlet.DefaultServlet-851547541} 2015-11-22 19:29:55,839 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-851547541=org.mortbay.jetty.servlet.DefaultServlet-851547541} 2015-11-22 19:29:55,839 INFO [Thread-4056] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context static 2015-11-22 19:29:55,839 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 + static_user_filter as filter 2015-11-22 19:29:55,840 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 + (F=static_user_filter,[*.html, *.jsp],[],15) as filterMapping 2015-11-22 19:29:55,840 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:55,840 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15)] 2015-11-22 19:29:55,840 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:55,840 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/=org.mortbay.jetty.servlet.DefaultServlet-608661613} 2015-11-22 19:29:55,840 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-608661613=org.mortbay.jetty.servlet.DefaultServlet-608661613} 2015-11-22 19:29:55,840 INFO [Thread-4056] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context node 2015-11-22 19:29:55,840 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7de68a0f + static_user_filter as filter 2015-11-22 19:29:55,840 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7de68a0f + (F=static_user_filter,[/*],[],15) as filterMapping 2015-11-22 19:29:55,841 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:55,841 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=safety,[/*],[],15), (F=RMAuthenticationFilter,[/*],[],15), (F=static_user_filter,[/*],[],15)] 2015-11-22 19:29:55,841 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:55,841 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/*=org.apache.hadoop.http.AdminAuthorizedServlet-1439946767} 2015-11-22 19:29:55,841 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.apache.hadoop.http.AdminAuthorizedServlet-1439946767=org.apache.hadoop.http.AdminAuthorizedServlet-1439946767} 2015-11-22 19:29:55,841 INFO [Thread-4056] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2015-11-22 19:29:55,841 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7b3ede0f + static_user_filter as filter 2015-11-22 19:29:55,841 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7b3ede0f + (F=static_user_filter,[/*],[],15) as filterMapping 2015-11-22 19:29:55,841 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:55,842 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=safety,[/*],[],15), (F=RMAuthenticationFilter,[/*],[],15), (F=static_user_filter,[/*],[],15)] 2015-11-22 19:29:55,842 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:55,842 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/*=org.mortbay.jetty.servlet.DefaultServlet-851547541} 2015-11-22 19:29:55,842 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-851547541=org.mortbay.jetty.servlet.DefaultServlet-851547541} 2015-11-22 19:29:55,842 INFO [Thread-4056] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2015-11-22 19:29:55,843 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 + Timeline Authentication Filter as filter 2015-11-22 19:29:55,843 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 + (F=Timeline Authentication Filter,[/*],[],15) as filterMapping 2015-11-22 19:29:55,843 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:55,843 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15)] 2015-11-22 19:29:55,843 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:55,843 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/=org.mortbay.jetty.servlet.DefaultServlet-608661613} 2015-11-22 19:29:55,843 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-608661613=org.mortbay.jetty.servlet.DefaultServlet-608661613} 2015-11-22 19:29:55,843 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7de68a0f + Timeline Authentication Filter as filter 2015-11-22 19:29:55,843 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7de68a0f + (F=Timeline Authentication Filter,[/*],[],15) as filterMapping 2015-11-22 19:29:55,844 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:55,844 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=safety,[/*],[],15), (F=RMAuthenticationFilter,[/*],[],15), (F=static_user_filter,[/*],[],15), (F=Timeline Authentication Filter,[/*],[],15)] 2015-11-22 19:29:55,844 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:55,844 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/*=org.apache.hadoop.http.AdminAuthorizedServlet-1439946767} 2015-11-22 19:29:55,844 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.apache.hadoop.http.AdminAuthorizedServlet-1439946767=org.apache.hadoop.http.AdminAuthorizedServlet-1439946767} 2015-11-22 19:29:55,844 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7b3ede0f + Timeline Authentication Filter as filter 2015-11-22 19:29:55,844 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7b3ede0f + (F=Timeline Authentication Filter,[/*],[],15) as filterMapping 2015-11-22 19:29:55,844 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:55,845 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=safety,[/*],[],15), (F=RMAuthenticationFilter,[/*],[],15), (F=static_user_filter,[/*],[],15), (F=Timeline Authentication Filter,[/*],[],15)] 2015-11-22 19:29:55,845 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:55,845 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/*=org.mortbay.jetty.servlet.DefaultServlet-851547541} 2015-11-22 19:29:55,845 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-851547541=org.mortbay.jetty.servlet.DefaultServlet-851547541} 2015-11-22 19:29:55,845 INFO [Thread-4056] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'Timeline Authentication Filter' (class=org.apache.hadoop.yarn.server.timeline.security.TimelineAuthenticationFilter) 2015-11-22 19:29:55,845 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 + stacks as servlet 2015-11-22 19:29:55,845 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 + (S=stacks,[/stacks]) as servletMapping 2015-11-22 19:29:55,845 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:55,845 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15)] 2015-11-22 19:29:55,846 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:55,846 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/stacks=stacks, /=org.mortbay.jetty.servlet.DefaultServlet-608661613} 2015-11-22 19:29:55,846 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-608661613=org.mortbay.jetty.servlet.DefaultServlet-608661613, stacks=stacks} 2015-11-22 19:29:55,846 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 + (F=RMAuthenticationFilter,[/stacks],[],15) as filterMapping 2015-11-22 19:29:55,846 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:55,846 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15), (F=RMAuthenticationFilter,[/stacks],[],15)] 2015-11-22 19:29:55,846 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:55,846 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/stacks=stacks, /=org.mortbay.jetty.servlet.DefaultServlet-608661613} 2015-11-22 19:29:55,846 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-608661613=org.mortbay.jetty.servlet.DefaultServlet-608661613, stacks=stacks} 2015-11-22 19:29:55,847 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 + (F=static_user_filter,[/stacks],[],15) as filterMapping 2015-11-22 19:29:55,847 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:55,847 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15), (F=RMAuthenticationFilter,[/stacks],[],15), (F=static_user_filter,[/stacks],[],15)] 2015-11-22 19:29:55,847 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:55,847 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/stacks=stacks, /=org.mortbay.jetty.servlet.DefaultServlet-608661613} 2015-11-22 19:29:55,847 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-608661613=org.mortbay.jetty.servlet.DefaultServlet-608661613, stacks=stacks} 2015-11-22 19:29:55,847 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 + logLevel as servlet 2015-11-22 19:29:55,847 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 + (S=logLevel,[/logLevel]) as servletMapping 2015-11-22 19:29:55,848 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:55,848 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15), (F=RMAuthenticationFilter,[/stacks],[],15), (F=static_user_filter,[/stacks],[],15)] 2015-11-22 19:29:55,848 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:55,848 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/stacks=stacks, /logLevel=logLevel, /=org.mortbay.jetty.servlet.DefaultServlet-608661613} 2015-11-22 19:29:55,848 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-608661613=org.mortbay.jetty.servlet.DefaultServlet-608661613, logLevel=logLevel, stacks=stacks} 2015-11-22 19:29:55,848 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 + (F=RMAuthenticationFilter,[/logLevel],[],15) as filterMapping 2015-11-22 19:29:55,848 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:55,848 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15), (F=RMAuthenticationFilter,[/stacks],[],15), (F=static_user_filter,[/stacks],[],15), (F=RMAuthenticationFilter,[/logLevel],[],15)] 2015-11-22 19:29:55,848 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:55,849 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/stacks=stacks, /logLevel=logLevel, /=org.mortbay.jetty.servlet.DefaultServlet-608661613} 2015-11-22 19:29:55,849 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-608661613=org.mortbay.jetty.servlet.DefaultServlet-608661613, logLevel=logLevel, stacks=stacks} 2015-11-22 19:29:55,849 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 + (F=static_user_filter,[/logLevel],[],15) as filterMapping 2015-11-22 19:29:55,850 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:55,850 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15), (F=RMAuthenticationFilter,[/stacks],[],15), (F=static_user_filter,[/stacks],[],15), (F=RMAuthenticationFilter,[/logLevel],[],15), (F=static_user_filter,[/logLevel],[],15)] 2015-11-22 19:29:55,850 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:55,850 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/stacks=stacks, /logLevel=logLevel, /=org.mortbay.jetty.servlet.DefaultServlet-608661613} 2015-11-22 19:29:55,850 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-608661613=org.mortbay.jetty.servlet.DefaultServlet-608661613, logLevel=logLevel, stacks=stacks} 2015-11-22 19:29:55,850 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 + metrics as servlet 2015-11-22 19:29:55,850 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 + (S=metrics,[/metrics]) as servletMapping 2015-11-22 19:29:55,850 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:55,850 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15), (F=RMAuthenticationFilter,[/stacks],[],15), (F=static_user_filter,[/stacks],[],15), (F=RMAuthenticationFilter,[/logLevel],[],15), (F=static_user_filter,[/logLevel],[],15)] 2015-11-22 19:29:55,851 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:55,851 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/stacks=stacks, /logLevel=logLevel, /metrics=metrics, /=org.mortbay.jetty.servlet.DefaultServlet-608661613} 2015-11-22 19:29:55,851 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-608661613=org.mortbay.jetty.servlet.DefaultServlet-608661613, logLevel=logLevel, stacks=stacks, metrics=metrics} 2015-11-22 19:29:55,851 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 + (F=RMAuthenticationFilter,[/metrics],[],15) as filterMapping 2015-11-22 19:29:55,851 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:55,851 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15), (F=RMAuthenticationFilter,[/stacks],[],15), (F=static_user_filter,[/stacks],[],15), (F=RMAuthenticationFilter,[/logLevel],[],15), (F=static_user_filter,[/logLevel],[],15), (F=RMAuthenticationFilter,[/metrics],[],15)] 2015-11-22 19:29:55,851 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:55,851 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/stacks=stacks, /logLevel=logLevel, /metrics=metrics, /=org.mortbay.jetty.servlet.DefaultServlet-608661613} 2015-11-22 19:29:55,852 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-608661613=org.mortbay.jetty.servlet.DefaultServlet-608661613, logLevel=logLevel, stacks=stacks, metrics=metrics} 2015-11-22 19:29:55,852 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 + (F=static_user_filter,[/metrics],[],15) as filterMapping 2015-11-22 19:29:55,852 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:55,852 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15), (F=RMAuthenticationFilter,[/stacks],[],15), (F=static_user_filter,[/stacks],[],15), (F=RMAuthenticationFilter,[/logLevel],[],15), (F=static_user_filter,[/logLevel],[],15), (F=RMAuthenticationFilter,[/metrics],[],15), (F=static_user_filter,[/metrics],[],15)] 2015-11-22 19:29:55,852 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:55,852 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/stacks=stacks, /logLevel=logLevel, /metrics=metrics, /=org.mortbay.jetty.servlet.DefaultServlet-608661613} 2015-11-22 19:29:55,852 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-608661613=org.mortbay.jetty.servlet.DefaultServlet-608661613, logLevel=logLevel, stacks=stacks, metrics=metrics} 2015-11-22 19:29:55,852 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 + jmx as servlet 2015-11-22 19:29:55,853 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 + (S=jmx,[/jmx]) as servletMapping 2015-11-22 19:29:55,853 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:55,853 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15), (F=RMAuthenticationFilter,[/stacks],[],15), (F=static_user_filter,[/stacks],[],15), (F=RMAuthenticationFilter,[/logLevel],[],15), (F=static_user_filter,[/logLevel],[],15), (F=RMAuthenticationFilter,[/metrics],[],15), (F=static_user_filter,[/metrics],[],15)] 2015-11-22 19:29:55,853 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:55,853 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/jmx=jmx, /stacks=stacks, /logLevel=logLevel, /metrics=metrics, /=org.mortbay.jetty.servlet.DefaultServlet-608661613} 2015-11-22 19:29:55,853 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-608661613=org.mortbay.jetty.servlet.DefaultServlet-608661613, logLevel=logLevel, jmx=jmx, stacks=stacks, metrics=metrics} 2015-11-22 19:29:55,853 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 + (F=RMAuthenticationFilter,[/jmx],[],15) as filterMapping 2015-11-22 19:29:55,853 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:55,853 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15), (F=RMAuthenticationFilter,[/stacks],[],15), (F=static_user_filter,[/stacks],[],15), (F=RMAuthenticationFilter,[/logLevel],[],15), (F=static_user_filter,[/logLevel],[],15), (F=RMAuthenticationFilter,[/metrics],[],15), (F=static_user_filter,[/metrics],[],15), (F=RMAuthenticationFilter,[/jmx],[],15)] 2015-11-22 19:29:55,854 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:55,854 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/jmx=jmx, /stacks=stacks, /logLevel=logLevel, /metrics=metrics, /=org.mortbay.jetty.servlet.DefaultServlet-608661613} 2015-11-22 19:29:55,854 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-608661613=org.mortbay.jetty.servlet.DefaultServlet-608661613, logLevel=logLevel, jmx=jmx, stacks=stacks, metrics=metrics} 2015-11-22 19:29:55,854 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 + (F=static_user_filter,[/jmx],[],15) as filterMapping 2015-11-22 19:29:55,854 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:55,854 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15), (F=RMAuthenticationFilter,[/stacks],[],15), (F=static_user_filter,[/stacks],[],15), (F=RMAuthenticationFilter,[/logLevel],[],15), (F=static_user_filter,[/logLevel],[],15), (F=RMAuthenticationFilter,[/metrics],[],15), (F=static_user_filter,[/metrics],[],15), (F=RMAuthenticationFilter,[/jmx],[],15), (F=static_user_filter,[/jmx],[],15)] 2015-11-22 19:29:55,855 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:55,855 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/jmx=jmx, /stacks=stacks, /logLevel=logLevel, /metrics=metrics, /=org.mortbay.jetty.servlet.DefaultServlet-608661613} 2015-11-22 19:29:55,855 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-608661613=org.mortbay.jetty.servlet.DefaultServlet-608661613, logLevel=logLevel, jmx=jmx, stacks=stacks, metrics=metrics} 2015-11-22 19:29:55,855 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 + conf as servlet 2015-11-22 19:29:55,855 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 + (S=conf,[/conf]) as servletMapping 2015-11-22 19:29:55,855 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:55,855 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15), (F=RMAuthenticationFilter,[/stacks],[],15), (F=static_user_filter,[/stacks],[],15), (F=RMAuthenticationFilter,[/logLevel],[],15), (F=static_user_filter,[/logLevel],[],15), (F=RMAuthenticationFilter,[/metrics],[],15), (F=static_user_filter,[/metrics],[],15), (F=RMAuthenticationFilter,[/jmx],[],15), (F=static_user_filter,[/jmx],[],15)] 2015-11-22 19:29:55,855 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:55,856 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/jmx=jmx, /conf=conf, /stacks=stacks, /logLevel=logLevel, /metrics=metrics, /=org.mortbay.jetty.servlet.DefaultServlet-608661613} 2015-11-22 19:29:55,856 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-608661613=org.mortbay.jetty.servlet.DefaultServlet-608661613, logLevel=logLevel, jmx=jmx, stacks=stacks, conf=conf, metrics=metrics} 2015-11-22 19:29:55,856 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 + (F=RMAuthenticationFilter,[/conf],[],15) as filterMapping 2015-11-22 19:29:55,856 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:55,856 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15), (F=RMAuthenticationFilter,[/stacks],[],15), (F=static_user_filter,[/stacks],[],15), (F=RMAuthenticationFilter,[/logLevel],[],15), (F=static_user_filter,[/logLevel],[],15), (F=RMAuthenticationFilter,[/metrics],[],15), (F=static_user_filter,[/metrics],[],15), (F=RMAuthenticationFilter,[/jmx],[],15), (F=static_user_filter,[/jmx],[],15), (F=RMAuthenticationFilter,[/conf],[],15)] 2015-11-22 19:29:55,856 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:55,856 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/jmx=jmx, /conf=conf, /stacks=stacks, /logLevel=logLevel, /metrics=metrics, /=org.mortbay.jetty.servlet.DefaultServlet-608661613} 2015-11-22 19:29:55,856 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-608661613=org.mortbay.jetty.servlet.DefaultServlet-608661613, logLevel=logLevel, jmx=jmx, stacks=stacks, conf=conf, metrics=metrics} 2015-11-22 19:29:55,857 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 + (F=static_user_filter,[/conf],[],15) as filterMapping 2015-11-22 19:29:55,857 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:55,857 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15), (F=RMAuthenticationFilter,[/stacks],[],15), (F=static_user_filter,[/stacks],[],15), (F=RMAuthenticationFilter,[/logLevel],[],15), (F=static_user_filter,[/logLevel],[],15), (F=RMAuthenticationFilter,[/metrics],[],15), (F=static_user_filter,[/metrics],[],15), (F=RMAuthenticationFilter,[/jmx],[],15), (F=static_user_filter,[/jmx],[],15), (F=RMAuthenticationFilter,[/conf],[],15), (F=static_user_filter,[/conf],[],15)] 2015-11-22 19:29:55,857 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:55,857 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/jmx=jmx, /conf=conf, /stacks=stacks, /logLevel=logLevel, /metrics=metrics, /=org.mortbay.jetty.servlet.DefaultServlet-608661613} 2015-11-22 19:29:55,857 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-608661613=org.mortbay.jetty.servlet.DefaultServlet-608661613, logLevel=logLevel, jmx=jmx, stacks=stacks, conf=conf, metrics=metrics} 2015-11-22 19:29:55,857 INFO [Thread-4056] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /node/* 2015-11-22 19:29:55,857 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 + (F=RMAuthenticationFilter,[/node/*],[],15) as filterMapping 2015-11-22 19:29:55,857 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:55,858 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15), (F=RMAuthenticationFilter,[/stacks],[],15), (F=static_user_filter,[/stacks],[],15), (F=RMAuthenticationFilter,[/logLevel],[],15), (F=static_user_filter,[/logLevel],[],15), (F=RMAuthenticationFilter,[/metrics],[],15), (F=static_user_filter,[/metrics],[],15), (F=RMAuthenticationFilter,[/jmx],[],15), (F=static_user_filter,[/jmx],[],15), (F=RMAuthenticationFilter,[/conf],[],15), (F=static_user_filter,[/conf],[],15), (F=RMAuthenticationFilter,[/node/*],[],15)] 2015-11-22 19:29:55,858 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:55,858 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/jmx=jmx, /conf=conf, /stacks=stacks, /logLevel=logLevel, /metrics=metrics, /=org.mortbay.jetty.servlet.DefaultServlet-608661613} 2015-11-22 19:29:55,858 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-608661613=org.mortbay.jetty.servlet.DefaultServlet-608661613, logLevel=logLevel, jmx=jmx, stacks=stacks, conf=conf, metrics=metrics} 2015-11-22 19:29:55,858 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 + (F=static_user_filter,[/node/*],[],15) as filterMapping 2015-11-22 19:29:55,859 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:55,859 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15), (F=RMAuthenticationFilter,[/stacks],[],15), (F=static_user_filter,[/stacks],[],15), (F=RMAuthenticationFilter,[/logLevel],[],15), (F=static_user_filter,[/logLevel],[],15), (F=RMAuthenticationFilter,[/metrics],[],15), (F=static_user_filter,[/metrics],[],15), (F=RMAuthenticationFilter,[/jmx],[],15), (F=static_user_filter,[/jmx],[],15), (F=RMAuthenticationFilter,[/conf],[],15), (F=static_user_filter,[/conf],[],15), (F=RMAuthenticationFilter,[/node/*],[],15), (F=static_user_filter,[/node/*],[],15)] 2015-11-22 19:29:55,859 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:55,859 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/jmx=jmx, /conf=conf, /stacks=stacks, /logLevel=logLevel, /metrics=metrics, /=org.mortbay.jetty.servlet.DefaultServlet-608661613} 2015-11-22 19:29:55,859 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-608661613=org.mortbay.jetty.servlet.DefaultServlet-608661613, logLevel=logLevel, jmx=jmx, stacks=stacks, conf=conf, metrics=metrics} 2015-11-22 19:29:55,859 INFO [Thread-4056] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /ws/* 2015-11-22 19:29:55,859 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 + (F=RMAuthenticationFilter,[/ws/*],[],15) as filterMapping 2015-11-22 19:29:55,859 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:55,859 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15), (F=RMAuthenticationFilter,[/stacks],[],15), (F=static_user_filter,[/stacks],[],15), (F=RMAuthenticationFilter,[/logLevel],[],15), (F=static_user_filter,[/logLevel],[],15), (F=RMAuthenticationFilter,[/metrics],[],15), (F=static_user_filter,[/metrics],[],15), (F=RMAuthenticationFilter,[/jmx],[],15), (F=static_user_filter,[/jmx],[],15), (F=RMAuthenticationFilter,[/conf],[],15), (F=static_user_filter,[/conf],[],15), (F=RMAuthenticationFilter,[/node/*],[],15), (F=static_user_filter,[/node/*],[],15), (F=RMAuthenticationFilter,[/ws/*],[],15)] 2015-11-22 19:29:55,859 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:55,859 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/jmx=jmx, /conf=conf, /stacks=stacks, /logLevel=logLevel, /metrics=metrics, /=org.mortbay.jetty.servlet.DefaultServlet-608661613} 2015-11-22 19:29:55,860 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-608661613=org.mortbay.jetty.servlet.DefaultServlet-608661613, logLevel=logLevel, jmx=jmx, stacks=stacks, conf=conf, metrics=metrics} 2015-11-22 19:29:55,860 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 + (F=static_user_filter,[/ws/*],[],15) as filterMapping 2015-11-22 19:29:55,860 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:55,860 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15), (F=RMAuthenticationFilter,[/stacks],[],15), (F=static_user_filter,[/stacks],[],15), (F=RMAuthenticationFilter,[/logLevel],[],15), (F=static_user_filter,[/logLevel],[],15), (F=RMAuthenticationFilter,[/metrics],[],15), (F=static_user_filter,[/metrics],[],15), (F=RMAuthenticationFilter,[/jmx],[],15), (F=static_user_filter,[/jmx],[],15), (F=RMAuthenticationFilter,[/conf],[],15), (F=static_user_filter,[/conf],[],15), (F=RMAuthenticationFilter,[/node/*],[],15), (F=static_user_filter,[/node/*],[],15), (F=RMAuthenticationFilter,[/ws/*],[],15), (F=static_user_filter,[/ws/*],[],15)] 2015-11-22 19:29:55,860 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:55,860 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/jmx=jmx, /conf=conf, /stacks=stacks, /logLevel=logLevel, /metrics=metrics, /=org.mortbay.jetty.servlet.DefaultServlet-608661613} 2015-11-22 19:29:55,860 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-608661613=org.mortbay.jetty.servlet.DefaultServlet-608661613, logLevel=logLevel, jmx=jmx, stacks=stacks, conf=conf, metrics=metrics} 2015-11-22 19:29:55,860 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container Server@23e7fedb + HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:0 as connector 2015-11-22 19:29:55,860 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 + guice as filter 2015-11-22 19:29:55,860 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 + (F=guice,[/*],[],15) as filterMapping 2015-11-22 19:29:55,861 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter, guice=guice} 2015-11-22 19:29:55,861 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15), (F=RMAuthenticationFilter,[/stacks],[],15), (F=static_user_filter,[/stacks],[],15), (F=RMAuthenticationFilter,[/logLevel],[],15), (F=static_user_filter,[/logLevel],[],15), (F=RMAuthenticationFilter,[/metrics],[],15), (F=static_user_filter,[/metrics],[],15), (F=RMAuthenticationFilter,[/jmx],[],15), (F=static_user_filter,[/jmx],[],15), (F=RMAuthenticationFilter,[/conf],[],15), (F=static_user_filter,[/conf],[],15), (F=RMAuthenticationFilter,[/node/*],[],15), (F=static_user_filter,[/node/*],[],15), (F=RMAuthenticationFilter,[/ws/*],[],15), (F=static_user_filter,[/ws/*],[],15), (F=guice,[/*],[],15)] 2015-11-22 19:29:55,861 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:55,861 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/jmx=jmx, /conf=conf, /stacks=stacks, /logLevel=logLevel, /metrics=metrics, /=org.mortbay.jetty.servlet.DefaultServlet-608661613} 2015-11-22 19:29:55,861 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-608661613=org.mortbay.jetty.servlet.DefaultServlet-608661613, logLevel=logLevel, jmx=jmx, stacks=stacks, conf=conf, metrics=metrics} 2015-11-22 19:29:55,864 DEBUG [Thread-4056] webapp.Router (Router.java:add(85)) - adding /([])->class org.apache.hadoop.yarn.server.nodemanager.webapp.NMController#info 2015-11-22 19:29:55,864 DEBUG [Thread-4056] webapp.Router (Router.java:load(267)) - trying: org.apache.hadoop.yarn.server.nodemanager.webapp.NMView 2015-11-22 19:29:55,864 DEBUG [Thread-4056] webapp.Router (Router.java:load(271)) - found org.apache.hadoop.yarn.server.nodemanager.webapp.NMView 2015-11-22 19:29:55,864 DEBUG [Thread-4056] webapp.Router (Router.java:add(85)) - adding /node([])->class org.apache.hadoop.yarn.server.nodemanager.webapp.NMController#node 2015-11-22 19:29:55,865 DEBUG [Thread-4056] webapp.Router (Router.java:load(267)) - trying: org.apache.hadoop.yarn.server.nodemanager.webapp.NMView 2015-11-22 19:29:55,865 DEBUG [Thread-4056] webapp.Router (Router.java:load(271)) - found org.apache.hadoop.yarn.server.nodemanager.webapp.NMView 2015-11-22 19:29:55,865 DEBUG [Thread-4056] webapp.Router (Router.java:add(85)) - adding /allApplications([])->class org.apache.hadoop.yarn.server.nodemanager.webapp.NMController#allApplications 2015-11-22 19:29:55,865 DEBUG [Thread-4056] webapp.Router (Router.java:load(267)) - trying: org.apache.hadoop.yarn.server.nodemanager.webapp.NMView 2015-11-22 19:29:55,865 DEBUG [Thread-4056] webapp.Router (Router.java:load(271)) - found org.apache.hadoop.yarn.server.nodemanager.webapp.NMView 2015-11-22 19:29:55,866 DEBUG [Thread-4056] webapp.Router (Router.java:add(85)) - adding /allContainers([])->class org.apache.hadoop.yarn.server.nodemanager.webapp.NMController#allContainers 2015-11-22 19:29:55,866 DEBUG [Thread-4056] webapp.Router (Router.java:load(267)) - trying: org.apache.hadoop.yarn.server.nodemanager.webapp.NMView 2015-11-22 19:29:55,866 DEBUG [Thread-4056] webapp.Router (Router.java:load(271)) - found org.apache.hadoop.yarn.server.nodemanager.webapp.NMView 2015-11-22 19:29:55,866 DEBUG [Thread-4056] webapp.Router (Router.java:add(85)) - adding /application([:app.id])->class org.apache.hadoop.yarn.server.nodemanager.webapp.NMController#application 2015-11-22 19:29:55,866 DEBUG [Thread-4056] webapp.Router (Router.java:load(267)) - trying: org.apache.hadoop.yarn.server.nodemanager.webapp.NMView 2015-11-22 19:29:55,866 DEBUG [Thread-4056] webapp.Router (Router.java:load(271)) - found org.apache.hadoop.yarn.server.nodemanager.webapp.NMView 2015-11-22 19:29:55,867 DEBUG [Thread-4056] webapp.Router (Router.java:add(85)) - adding /container([:container.id])->class org.apache.hadoop.yarn.server.nodemanager.webapp.NMController#container 2015-11-22 19:29:55,868 DEBUG [Thread-4056] webapp.Router (Router.java:load(267)) - trying: org.apache.hadoop.yarn.server.nodemanager.webapp.NMView 2015-11-22 19:29:55,868 DEBUG [Thread-4056] webapp.Router (Router.java:load(271)) - found org.apache.hadoop.yarn.server.nodemanager.webapp.NMView 2015-11-22 19:29:55,868 DEBUG [Thread-4056] webapp.Router (Router.java:add(85)) - adding /containerlogs([:container.id, :app.owner, :log.type])->class org.apache.hadoop.yarn.server.nodemanager.webapp.NMController#logs 2015-11-22 19:29:55,868 DEBUG [Thread-4056] webapp.Router (Router.java:load(267)) - trying: org.apache.hadoop.yarn.server.nodemanager.webapp.NMView 2015-11-22 19:29:55,868 DEBUG [Thread-4056] webapp.Router (Router.java:load(271)) - found org.apache.hadoop.yarn.server.nodemanager.webapp.NMView 2015-11-22 19:29:55,868 DEBUG [Thread-4056] webapp.Router (Router.java:add(85)) - adding /errors-and-warnings([])->class org.apache.hadoop.yarn.server.nodemanager.webapp.NMController#errorsAndWarnings 2015-11-22 19:29:55,868 DEBUG [Thread-4056] webapp.Router (Router.java:load(267)) - trying: org.apache.hadoop.yarn.server.nodemanager.webapp.NMView 2015-11-22 19:29:55,869 DEBUG [Thread-4056] webapp.Router (Router.java:load(271)) - found org.apache.hadoop.yarn.server.nodemanager.webapp.NMView Nov 22, 2015 7:29:55 PM com.google.inject.servlet.GuiceFilter setPipeline WARNING: Multiple Servlet injectors detected. This is a warning indicating that you have more than one GuiceFilter running in your web application. If this is deliberate, you may safely ignore this message. If this is NOT deliberate however, your application may not work as expected. 2015-11-22 19:29:55,872 INFO [Thread-4056] webapp.WebApps (WebApps.java:build(288)) - Registered webapp guice modules 2015-11-22 19:29:55,872 INFO [Thread-4056] http.HttpServer2 (HttpServer2.java:openListeners(917)) - Jetty bound to port 43914 2015-11-22 19:29:55,873 INFO [Thread-4056] mortbay.log (Slf4jLog.java:info(67)) - jetty-6.1.26 2015-11-22 19:29:55,876 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - started org.mortbay.thread.QueuedThreadPool@7d421cea 2015-11-22 19:29:55,878 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Thread Context class loader is: ContextLoader@node([]) / sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,878 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Parent class loader is: sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,878 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Parent class loader is: sun.misc.Launcher$ExtClassLoader@2de8284b 2015-11-22 19:29:55,878 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Try webapp=file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes/webapps/node/, exists=true, directory=true 2015-11-22 19:29:55,878 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - webapp=file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes/webapps/node/ 2015-11-22 19:29:55,879 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Created temp dir /tmp/Jetty_ip.172.31.20.42_ap.northeast.1_compute_internal_43914_node____.adf677 for org.mortbay.jetty.webapp.WebAppContext@1f7a38df{/,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes/webapps/node} 2015-11-22 19:29:55,879 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Configuring web-jetty.xml 2015-11-22 19:29:55,879 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - TLD search of file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/surefire/surefirebooter2438562254715847001.jar 2015-11-22 19:29:55,879 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - TLD search of file:/usr/lib/jvm/java-8-oracle/jre/lib/ext/zipfs.jar 2015-11-22 19:29:55,879 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - TLD search of file:/usr/lib/jvm/java-8-oracle/jre/lib/ext/sunjce_provider.jar 2015-11-22 19:29:55,880 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - TLD search of file:/usr/lib/jvm/java-8-oracle/jre/lib/ext/sunpkcs11.jar 2015-11-22 19:29:55,880 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - TLD search of file:/usr/lib/jvm/java-8-oracle/jre/lib/ext/sunec.jar 2015-11-22 19:29:55,880 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - TLD search of file:/usr/lib/jvm/java-8-oracle/jre/lib/ext/dnsns.jar 2015-11-22 19:29:55,880 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - TLD search of file:/usr/lib/jvm/java-8-oracle/jre/lib/ext/localedata.jar 2015-11-22 19:29:55,880 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - TLD search of file:/usr/lib/jvm/java-8-oracle/jre/lib/ext/jfxrt.jar 2015-11-22 19:29:55,885 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - TLD search of file:/usr/lib/jvm/java-8-oracle/jre/lib/ext/cldrdata.jar 2015-11-22 19:29:55,886 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - TLD search of file:/usr/lib/jvm/java-8-oracle/jre/lib/ext/nashorn.jar 2015-11-22 19:29:55,887 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.org.apache.xerces.internal.jaxp.SAXParserFactoryImpl from null 2015-11-22 19:29:55,887 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.org.apache.xerces.internal.impl.dv.dtd.DTDDVFactoryImpl from null 2015-11-22 19:29:55,888 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - getResource(javax/servlet/jsp/resources/web-jsptaglibrary_1_1.dtd)=jar:file:/home/ubuntu/.m2/repository/javax/servlet/jsp/jsp-api/2.1/jsp-api-2.1.jar!/javax/servlet/jsp/resources/web-jsptaglibrary_1_1.dtd 2015-11-22 19:29:55,888 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - getResource(javax/servlet/jsp/resources/web-jsptaglibrary_1_2.dtd)=jar:file:/home/ubuntu/.m2/repository/javax/servlet/jsp/jsp-api/2.1/jsp-api-2.1.jar!/javax/servlet/jsp/resources/web-jsptaglibrary_1_2.dtd 2015-11-22 19:29:55,888 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - getResource(javax/servlet/jsp/resources/web-jsptaglibrary_2_0.xsd)=jar:file:/home/ubuntu/.m2/repository/javax/servlet/jsp/jsp-api/2.1/jsp-api-2.1.jar!/javax/servlet/jsp/resources/web-jsptaglibrary_2_0.xsd 2015-11-22 19:29:55,888 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - getResource(javax/servlet/jsp/resources/web-jsptaglibrary_1_1.dtd)=jar:file:/home/ubuntu/.m2/repository/javax/servlet/jsp/jsp-api/2.1/jsp-api-2.1.jar!/javax/servlet/jsp/resources/web-jsptaglibrary_1_1.dtd 2015-11-22 19:29:55,888 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - getResource(javax/servlet/jsp/resources/web-jsptaglibrary_1_2.dtd)=jar:file:/home/ubuntu/.m2/repository/javax/servlet/jsp/jsp-api/2.1/jsp-api-2.1.jar!/javax/servlet/jsp/resources/web-jsptaglibrary_1_2.dtd 2015-11-22 19:29:55,888 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - getResource(javax/servlet/jsp/resources/web-jsptaglibrary_2_0.xsd)=jar:file:/home/ubuntu/.m2/repository/javax/servlet/jsp/jsp-api/2.1/jsp-api-2.1.jar!/javax/servlet/jsp/resources/web-jsptaglibrary_2_0.xsd 2015-11-22 19:29:55,888 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container Server@23e7fedb + org.mortbay.jetty.servlet.HashSessionIdManager@4f1268d0 as sessionIdManager 2015-11-22 19:29:55,889 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Init SecureRandom. 2015-11-22 19:29:55,889 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - started org.mortbay.jetty.servlet.HashSessionIdManager@4f1268d0 2015-11-22 19:29:55,889 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - started org.mortbay.jetty.servlet.HashSessionManager@7d1aa0fc 2015-11-22 19:29:55,889 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter, guice=guice} 2015-11-22 19:29:55,889 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15), (F=RMAuthenticationFilter,[/stacks],[],15), (F=static_user_filter,[/stacks],[],15), (F=RMAuthenticationFilter,[/logLevel],[],15), (F=static_user_filter,[/logLevel],[],15), (F=RMAuthenticationFilter,[/metrics],[],15), (F=static_user_filter,[/metrics],[],15), (F=RMAuthenticationFilter,[/jmx],[],15), (F=static_user_filter,[/jmx],[],15), (F=RMAuthenticationFilter,[/conf],[],15), (F=static_user_filter,[/conf],[],15), (F=RMAuthenticationFilter,[/node/*],[],15), (F=static_user_filter,[/node/*],[],15), (F=RMAuthenticationFilter,[/ws/*],[],15), (F=static_user_filter,[/ws/*],[],15), (F=guice,[/*],[],15)] 2015-11-22 19:29:55,889 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:55,889 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/jmx=jmx, /conf=conf, /stacks=stacks, /logLevel=logLevel, /metrics=metrics, /=org.mortbay.jetty.servlet.DefaultServlet-608661613} 2015-11-22 19:29:55,889 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-608661613=org.mortbay.jetty.servlet.DefaultServlet-608661613, logLevel=logLevel, jmx=jmx, stacks=stacks, conf=conf, metrics=metrics} 2015-11-22 19:29:55,890 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - starting ServletHandler@7232d296 2015-11-22 19:29:55,890 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - started ServletHandler@7232d296 2015-11-22 19:29:55,890 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - starting SecurityHandler@6ebffb2d 2015-11-22 19:29:55,890 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - started SecurityHandler@6ebffb2d 2015-11-22 19:29:55,890 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - starting SessionHandler@3368da8e 2015-11-22 19:29:55,890 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - started SessionHandler@3368da8e 2015-11-22 19:29:55,890 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - starting org.mortbay.jetty.webapp.WebAppContext@1f7a38df{/,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes/webapps/node} 2015-11-22 19:29:55,890 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - starting ErrorPageErrorHandler@19d87bab 2015-11-22 19:29:55,890 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - started ErrorPageErrorHandler@19d87bab 2015-11-22 19:29:55,890 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class org.apache.hadoop.http.NoCacheFilter from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,891 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Holding class org.apache.hadoop.http.NoCacheFilter 2015-11-22 19:29:55,891 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - started NoCacheFilter 2015-11-22 19:29:55,891 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class org.apache.hadoop.http.NoCacheFilter from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,891 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Holding class org.apache.hadoop.http.NoCacheFilter 2015-11-22 19:29:55,891 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - started NoCacheFilter 2015-11-22 19:29:55,891 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class org.apache.hadoop.http.HttpServer2$QuotingInputFilter from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,891 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Holding class org.apache.hadoop.http.HttpServer2$QuotingInputFilter 2015-11-22 19:29:55,891 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - started safety 2015-11-22 19:29:55,891 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,891 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Holding class org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter 2015-11-22 19:29:55,892 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class org.apache.hadoop.security.token.delegation.web.PseudoDelegationTokenAuthenticationHandler from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,892 INFO [Thread-4056] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:29:55,892 INFO [Thread[Thread-4097,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2015-11-22 19:29:55,893 INFO [Thread[Thread-4097,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:29:55,896 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - started RMAuthenticationFilter 2015-11-22 19:29:55,896 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,896 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Holding class org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter 2015-11-22 19:29:55,896 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - started static_user_filter 2015-11-22 19:29:55,897 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class org.apache.hadoop.yarn.server.timeline.security.TimelineAuthenticationFilter from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,897 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Holding class org.apache.hadoop.yarn.server.timeline.security.TimelineAuthenticationFilter 2015-11-22 19:29:55,897 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class org.apache.hadoop.security.token.delegation.web.PseudoDelegationTokenAuthenticationHandler from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,897 INFO [Thread-4056] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:29:55,897 DEBUG [Thread-4056] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:stopThreads(619)) - Stopping expired delegation token remover thread 2015-11-22 19:29:55,897 INFO [Thread[Thread-4099,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2015-11-22 19:29:55,898 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - started Timeline Authentication Filter 2015-11-22 19:29:55,898 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.google.inject.servlet.GuiceFilter from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,898 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Holding class com.google.inject.servlet.GuiceFilter 2015-11-22 19:29:55,900 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.server.impl.container.WebApplicationProviderImpl from sun.misc.Launcher$AppClassLoader@42a57993 Nov 22, 2015 7:29:55 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.nodemanager.webapp.NMWebServices as a root resource class Nov 22, 2015 7:29:55 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a provider class Nov 22, 2015 7:29:55 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.nodemanager.webapp.JAXBContextResolver as a provider class Nov 22, 2015 7:29:55 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate INFO: Initiating Jersey application, version 'Jersey: 1.9 09/02/2011 11:17 AM' 2015-11-22 19:29:55,905 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.server.impl.model.method.dispatch.VoidVoidDispatchProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,905 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.server.impl.model.method.dispatch.HttpReqResDispatchProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,905 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.server.impl.model.method.dispatch.MultipartFormDispatchProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,905 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.server.impl.model.method.dispatch.FormDispatchProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,905 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.server.impl.model.method.dispatch.EntityParamDispatchProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,906 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded interface javax.annotation.PostConstruct 2015-11-22 19:29:55,906 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded interface javax.annotation.PostConstruct from null 2015-11-22 19:29:55,906 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded interface javax.annotation.PreDestroy 2015-11-22 19:29:55,906 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded interface javax.annotation.PreDestroy from null 2015-11-22 19:29:55,908 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.xml.SAXParserContextProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,908 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.xml.XMLStreamReaderContextProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,908 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.xml.DocumentBuilderFactoryProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,908 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.xml.TransformerFactoryProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,911 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.api.container.filter.GZIPContentEncodingFilter from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,911 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.server.impl.container.filter.NormalizeFilter from sun.misc.Launcher$AppClassLoader@42a57993 Nov 22, 2015 7:29:55 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.nodemanager.webapp.JAXBContextResolver to GuiceManagedComponentProvider with the scope "Singleton" 2015-11-22 19:29:55,913 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - getResource(META-INF/services/javax.xml.bind.JAXBContext)=jar:file:/home/ubuntu/.m2/repository/com/sun/xml/bind/jaxb-impl/2.2.3-1/jaxb-impl-2.2.3-1.jar!/META-INF/services/javax.xml.bind.JAXBContext 2015-11-22 19:29:55,914 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.xml.bind.v2.ContextFactory from sun.misc.Launcher$AppClassLoader@42a57993 Nov 22, 2015 7:29:55 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.webapp.GenericExceptionHandler to GuiceManagedComponentProvider with the scope "Singleton" 2015-11-22 19:29:55,923 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.StringProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,923 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.ByteArrayProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,923 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.FileProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,923 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.InputStreamProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,924 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.DataSourceProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,924 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.RenderedImageProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,924 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.MimeMultipartProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,924 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.FormProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,924 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.FormMultivaluedMapProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,924 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.XMLRootElementProvider$App from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,925 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.XMLRootElementProvider$Text from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,925 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.XMLRootElementProvider$General from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,925 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.XMLJAXBElementProvider$App from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,925 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.XMLJAXBElementProvider$Text from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,925 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.XMLJAXBElementProvider$General from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,925 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.XMLListElementProvider$App from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,926 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.XMLListElementProvider$Text from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,926 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.XMLListElementProvider$General from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,926 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.ReaderProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,926 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.DocumentProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,926 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.SourceProvider$StreamSourceReader from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,927 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.SourceProvider$SAXSourceReader from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,927 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.SourceProvider$DOMSourceReader from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,927 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.XMLRootObjectProvider$App from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,927 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.XMLRootObjectProvider$Text from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,927 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.XMLRootObjectProvider$General from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,927 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.EntityHolderReader from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,928 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.json.impl.provider.entity.JSONRootElementProvider$App from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,928 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.json.impl.provider.entity.JSONRootElementProvider$General from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,928 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.json.impl.provider.entity.JSONJAXBElementProvider$App from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,928 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.json.impl.provider.entity.JSONJAXBElementProvider$General from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,928 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.json.impl.provider.entity.JSONListElementProvider$App from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,929 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.json.impl.provider.entity.JSONListElementProvider$General from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,929 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.json.impl.provider.entity.JSONArrayProvider$App from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,929 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.json.impl.provider.entity.JSONArrayProvider$General from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,929 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.json.impl.provider.entity.JSONObjectProvider$App from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,929 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.json.impl.provider.entity.JSONObjectProvider$General from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,929 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.json.impl.provider.entity.JacksonProviderProxy from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,958 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.StreamingOutputProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,958 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.SourceProvider$SourceWriter from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,958 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.json.impl.provider.entity.JSONWithPaddingProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,958 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.server.impl.template.ViewableMessageBodyWriter from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,961 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.server.impl.model.parameter.multivalued.StringReaderProviders$TypeFromStringEnum from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,962 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.server.impl.model.parameter.multivalued.StringReaderProviders$TypeValueOf from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,962 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.server.impl.model.parameter.multivalued.StringReaderProviders$TypeFromString from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,962 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.server.impl.model.parameter.multivalued.StringReaderProviders$StringConstructor from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,962 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.server.impl.model.parameter.multivalued.StringReaderProviders$DateProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,963 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.server.impl.model.parameter.multivalued.JAXBStringReaderProviders$RootElementProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,974 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - getResource(META-INF/services/javax.xml.bind.JAXBContext)=jar:file:/home/ubuntu/.m2/repository/com/sun/xml/bind/jaxb-impl/2.2.3-1/jaxb-impl-2.2.3-1.jar!/META-INF/services/javax.xml.bind.JAXBContext 2015-11-22 19:29:55,975 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.xml.bind.v2.ContextFactory from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:55,975 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.research.ws.wadl.ObjectFactory from sun.misc.Launcher$AppClassLoader@42a57993 Nov 22, 2015 7:29:55 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.nodemanager.webapp.NMWebServices to GuiceManagedComponentProvider with the scope "Singleton" 2015-11-22 19:29:56,001 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - started guice 2015-11-22 19:29:56,002 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - started conf 2015-11-22 19:29:56,002 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - started stacks 2015-11-22 19:29:56,002 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - started jmx 2015-11-22 19:29:56,002 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - started logLevel 2015-11-22 19:29:56,002 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - started metrics 2015-11-22 19:29:56,002 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - started org.mortbay.jetty.servlet.DefaultServlet$NIOResourceCache@740a7a1d 2015-11-22 19:29:56,002 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - resource base = file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes/webapps/node/ 2015-11-22 19:29:56,003 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - started org.mortbay.jetty.servlet.DefaultServlet-608661613 2015-11-22 19:29:56,003 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - started org.mortbay.jetty.webapp.WebAppContext@1f7a38df{/,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes/webapps/node} 2015-11-22 19:29:56,003 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container org.mortbay.jetty.servlet.Context@82f9d2e{/logs,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/log} + ErrorHandler@101e5de1 as errorHandler 2015-11-22 19:29:56,003 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:56,003 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=safety,[/*],[],15), (F=RMAuthenticationFilter,[/*],[],15), (F=static_user_filter,[/*],[],15), (F=Timeline Authentication Filter,[/*],[],15)] 2015-11-22 19:29:56,003 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:56,003 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/*=org.apache.hadoop.http.AdminAuthorizedServlet-1439946767} 2015-11-22 19:29:56,003 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.apache.hadoop.http.AdminAuthorizedServlet-1439946767=org.apache.hadoop.http.AdminAuthorizedServlet-1439946767} 2015-11-22 19:29:56,004 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - starting ServletHandler@7de68a0f 2015-11-22 19:29:56,004 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - started ServletHandler@7de68a0f 2015-11-22 19:29:56,004 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - starting org.mortbay.jetty.servlet.Context@82f9d2e{/logs,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/log} 2015-11-22 19:29:56,004 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - starting ErrorHandler@101e5de1 2015-11-22 19:29:56,004 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - started ErrorHandler@101e5de1 2015-11-22 19:29:56,004 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Holding class org.apache.hadoop.http.AdminAuthorizedServlet 2015-11-22 19:29:56,004 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - started org.apache.hadoop.http.AdminAuthorizedServlet-1439946767 2015-11-22 19:29:56,004 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - started org.mortbay.jetty.servlet.Context@82f9d2e{/logs,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/log} 2015-11-22 19:29:56,004 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Container org.mortbay.jetty.servlet.Context@f8cf43d{/static,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes/webapps/static/} + ErrorHandler@48efb6c1 as errorHandler 2015-11-22 19:29:56,005 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:56,005 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=safety,[/*],[],15), (F=RMAuthenticationFilter,[/*],[],15), (F=static_user_filter,[/*],[],15), (F=Timeline Authentication Filter,[/*],[],15)] 2015-11-22 19:29:56,005 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:56,005 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/*=org.mortbay.jetty.servlet.DefaultServlet-851547541} 2015-11-22 19:29:56,005 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-851547541=org.mortbay.jetty.servlet.DefaultServlet-851547541} 2015-11-22 19:29:56,005 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - starting ServletHandler@7b3ede0f 2015-11-22 19:29:56,005 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - started ServletHandler@7b3ede0f 2015-11-22 19:29:56,005 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - starting org.mortbay.jetty.servlet.Context@f8cf43d{/static,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes/webapps/static/} 2015-11-22 19:29:56,005 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - starting ErrorHandler@48efb6c1 2015-11-22 19:29:56,006 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - started ErrorHandler@48efb6c1 2015-11-22 19:29:56,006 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - Holding class org.mortbay.jetty.servlet.DefaultServlet 2015-11-22 19:29:56,006 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - started org.mortbay.jetty.servlet.DefaultServlet-851547541 2015-11-22 19:29:56,006 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - started org.mortbay.jetty.servlet.Context@f8cf43d{/static,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes/webapps/static/} 2015-11-22 19:29:56,006 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - starting ContextHandlerCollection@3c3c36d2 2015-11-22 19:29:56,006 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - started ContextHandlerCollection@3c3c36d2 2015-11-22 19:29:56,006 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - starting Server@23e7fedb 2015-11-22 19:29:56,006 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - started org.mortbay.jetty.nio.SelectChannelConnector$1@26525736 2015-11-22 19:29:56,006 INFO [Thread-4056] mortbay.log (Slf4jLog.java:info(67)) - Started HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:43914 2015-11-22 19:29:56,007 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - started HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:43914 2015-11-22 19:29:56,007 DEBUG [Thread-4056] mortbay.log (Slf4jLog.java:debug(40)) - started Server@23e7fedb 2015-11-22 19:29:56,007 INFO [Thread-4056] webapp.WebApps (WebApps.java:start(307)) - Web app node started at 43914 2015-11-22 19:29:56,007 DEBUG [Thread-4056] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.nodemanager.webapp.WebServer is started 2015-11-22 19:29:56,008 DEBUG [Thread-4056] service.AbstractService (AbstractService.java:start(197)) - Service Dispatcher is started 2015-11-22 19:29:56,008 INFO [Thread-4056] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:serviceStart(222)) - Node ID assigned is : ip-172-31-20-42.ap-northeast-1.compute.internal:35735 2015-11-22 19:29:56,008 INFO [Thread-4056] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNMContainerStatuses(550)) - Sending out 0 NM container statuses: [] 2015-11-22 19:29:56,008 INFO [Thread-4056] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:registerWithRM(330)) - Registering with RM using containers :[] 2015-11-22 19:29:56,008 DEBUG [Thread-4056] util.RackResolver (RackResolver.java:coreResolve(112)) - Resolved ip-172-31-20-42.ap-northeast-1.compute.internal to /default-rack 2015-11-22 19:29:56,009 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStartedEvent.EventType: STARTED 2015-11-22 19:29:56,009 INFO [Thread-4056] resourcemanager.ResourceTrackerService (ResourceTrackerService.java:registerNodeManager(375)) - NodeManager from node ip-172-31-20-42.ap-northeast-1.compute.internal(cmPort: 35735 httpPort: 43914) registered with capability: , assigned nodeId ip-172-31-20-42.ap-northeast-1.compute.internal:35735 2015-11-22 19:29:56,009 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:35735 of type STARTED 2015-11-22 19:29:56,009 INFO [Thread-4056] security.NMContainerTokenSecretManager (NMContainerTokenSecretManager.java:setMasterKey(138)) - Rolling master-key for container-tokens, got key with id -368252818 2015-11-22 19:29:56,009 INFO [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(566)) - ip-172-31-20-42.ap-northeast-1.compute.internal:35735 Node Transitioned from NEW to RUNNING 2015-11-22 19:29:56,009 INFO [Thread-4056] security.NMTokenSecretManagerInNM (NMTokenSecretManagerInNM.java:setMasterKey(135)) - Rolling master-key for container-tokens, got key with id -1658271729 2015-11-22 19:29:56,009 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeAddedSchedulerEvent.EventType: NODE_ADDED 2015-11-22 19:29:56,009 INFO [Thread-4056] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:registerWithRM(391)) - Registered with ResourceManager as ip-172-31-20-42.ap-northeast-1.compute.internal:35735 with total resource of 2015-11-22 19:29:56,009 INFO [Thread-4056] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:registerWithRM(392)) - Notifying ContainerManager to unblock new container-requests 2015-11-22 19:29:56,009 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.NodesListManagerEvent.EventType: NODE_USABLE 2015-11-22 19:29:56,010 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeLabelsUpdateSchedulerEvent.EventType: NODE_LABELS_UPDATE 2015-11-22 19:29:56,010 DEBUG [Thread-4056] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl is started 2015-11-22 19:29:56,013 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:29:56,013 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 0 container statuses: [] 2015-11-22 19:29:56,013 DEBUG [Thread-4056] service.AbstractService (AbstractService.java:start(197)) - Service NodeManager is started 2015-11-22 19:29:56,013 INFO [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:addNode(1427)) - Added node ip-172-31-20-42.ap-northeast-1.compute.internal:35735 clusterResource: 2015-11-22 19:29:56,014 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:56,014 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:35735 of type STATUS_UPDATE 2015-11-22 19:29:56,014 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:29:56,014 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:35735 clusterResources: 2015-11-22 19:29:56,014 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:35735 availableResource: 2015-11-22 19:29:56,014 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:29:56,014 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:29:56,781 DEBUG [main] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.MiniYARNCluster$NodeManagerWrapper_0 is started 2015-11-22 19:29:56,781 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(728)) - Waiting for Timeline Server to start... 2015-11-22 19:29:56,781 DEBUG [Thread-4102] service.CompositeService (CompositeService.java:serviceStart(115)) - org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer: starting services, size=5 2015-11-22 19:29:56,781 DEBUG [Thread-4102] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.timeline.MemoryTimelineStore is started 2015-11-22 19:29:56,782 INFO [Thread-4102] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:29:56,782 DEBUG [Thread-4102] security.TimelineDelegationTokenSecretManagerService$TimelineDelegationTokenSecretManager (TimelineDelegationTokenSecretManagerService.java:storeNewMasterKey(157)) - Storing master key 1 2015-11-22 19:29:56,782 DEBUG [Thread-4102] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.timeline.security.TimelineDelegationTokenSecretManagerService is started 2015-11-22 19:29:56,782 INFO [Thread[Thread-4104,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2015-11-22 19:29:56,782 DEBUG [Thread-4102] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.timeline.TimelineDataManager is started 2015-11-22 19:29:56,782 INFO [Thread[Thread-4104,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:29:56,782 DEBUG [Thread-4102] ipc.YarnRPC (YarnRPC.java:create(59)) - Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC 2015-11-22 19:29:56,783 DEBUG [Thread[Thread-4104,5,main]] security.TimelineDelegationTokenSecretManagerService$TimelineDelegationTokenSecretManager (TimelineDelegationTokenSecretManagerService.java:storeNewMasterKey(157)) - Storing master key 2 2015-11-22 19:29:56,783 DEBUG [Thread-4102] ipc.HadoopYarnProtoRPC (HadoopYarnProtoRPC.java:getServer(62)) - Creating a HadoopYarnProtoRpc server for protocol interface org.apache.hadoop.yarn.api.ApplicationHistoryProtocol with 10 handlers 2015-11-22 19:29:56,783 INFO [Thread-4102] ipc.CallQueueManager (CallQueueManager.java:(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-11-22 19:29:56,783 DEBUG [Thread-4102] ipc.Server (Server.java:getAuthMethods(2447)) - Server accepts auth methods:[SIMPLE] 2015-11-22 19:29:56,784 INFO [Socket Reader #1 for port 56823] ipc.Server (Server.java:run(724)) - Starting Socket Reader #1 for port 56823 2015-11-22 19:29:56,784 DEBUG [Thread-4102] metrics.RpcMetrics (RpcMetrics.java:(74)) - Initialized MetricsRegistry{info=MetricsInfoImpl{name=rpc, description=rpc}, tags=[MetricsTag{info=MetricsInfoImpl{name=port, description=RPC port}, value=56823}], metrics=[]} 2015-11-22 19:29:56,785 DEBUG [Thread-4102] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.ipc.metrics.RpcMetrics.receivedBytes with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Number of received bytes]) 2015-11-22 19:29:56,785 DEBUG [Thread-4102] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.ipc.metrics.RpcMetrics.sentBytes with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Number of sent bytes]) 2015-11-22 19:29:56,785 DEBUG [Thread-4102] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.ipc.metrics.RpcMetrics.rpcQueueTime with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Queue time]) 2015-11-22 19:29:56,785 DEBUG [Thread-4102] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.ipc.metrics.RpcMetrics.rpcProcessingTime with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Processing time]) 2015-11-22 19:29:56,785 DEBUG [Thread-4102] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.ipc.metrics.RpcMetrics.rpcAuthenticationFailures with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Number of authentication failures]) 2015-11-22 19:29:56,785 DEBUG [Thread-4102] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.ipc.metrics.RpcMetrics.rpcAuthenticationSuccesses with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Number of authentication successes]) 2015-11-22 19:29:56,785 DEBUG [Thread-4102] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.ipc.metrics.RpcMetrics.rpcAuthorizationFailures with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Number of authorization failures]) 2015-11-22 19:29:56,785 DEBUG [Thread-4102] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.ipc.metrics.RpcMetrics.rpcAuthorizationSuccesses with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Number of authorization successes]) 2015-11-22 19:29:56,785 DEBUG [Thread-4102] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.ipc.metrics.RpcMetrics.rpcClientBackoff with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Number of client backoff requests]) 2015-11-22 19:29:56,785 DEBUG [Thread-4102] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableCounterLong org.apache.hadoop.ipc.metrics.RpcMetrics.rpcSlowCalls with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Number of Slow RPC calls]) 2015-11-22 19:29:56,786 DEBUG [Thread-4102] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForMethod(82)) - method public int org.apache.hadoop.ipc.metrics.RpcMetrics.numOpenConnections() with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Number of open connections]) 2015-11-22 19:29:56,786 DEBUG [Thread-4102] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForMethod(82)) - method public int org.apache.hadoop.ipc.metrics.RpcMetrics.callQueueLength() with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[Length of the call queue]) 2015-11-22 19:29:56,786 DEBUG [Thread-4102] impl.MetricsSystemImpl (MetricsSystemImpl.java:register(232)) - RpcActivityForPort56823, Aggregate RPC metrics 2015-11-22 19:29:56,786 DEBUG [Thread-4102] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'PropertiesConfiguration' for key: source.source.start_mbeans 2015-11-22 19:29:56,786 DEBUG [Thread-4102] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'MetricsConfig' for key: source.start_mbeans 2015-11-22 19:29:56,786 DEBUG [Thread-4102] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'PropertiesConfiguration' for key: *.source.start_mbeans 2015-11-22 19:29:56,786 DEBUG [Thread-4102] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateAttrCache(254)) - Updating attr cache... 2015-11-22 19:29:56,786 DEBUG [Thread-4102] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateAttrCache(268)) - Done. # tags & metrics=17 2015-11-22 19:29:56,786 DEBUG [Thread-4102] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateInfoCache(248)) - Updating info cache... 2015-11-22 19:29:56,786 DEBUG [Thread-4102] impl.MetricsSystemImpl (MBeanInfoBuilder.java:get(109)) - [javax.management.MBeanAttributeInfo[description=RPC port, name=tag.port, type=java.lang.String, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Metrics context, name=tag.Context, type=java.lang.String, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Local hostname, name=tag.Hostname, type=java.lang.String, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of received bytes, name=ReceivedBytes, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of sent bytes, name=SentBytes, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of ops for queue time, name=RpcQueueTimeNumOps, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Average time for queue time, name=RpcQueueTimeAvgTime, type=java.lang.Double, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of ops for processing time, name=RpcProcessingTimeNumOps, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Average time for processing time, name=RpcProcessingTimeAvgTime, type=java.lang.Double, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of authentication failures, name=RpcAuthenticationFailures, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of authentication successes, name=RpcAuthenticationSuccesses, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of authorization failures, name=RpcAuthorizationFailures, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of authorization successes, name=RpcAuthorizationSuccesses, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of client backoff requests, name=RpcClientBackoff, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of Slow RPC calls, name=RpcSlowCalls, type=java.lang.Long, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Number of open connections, name=NumOpenConnections, type=java.lang.Integer, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Length of the call queue, name=CallQueueLength, type=java.lang.Integer, read-only, descriptor={}]] 2015-11-22 19:29:56,787 DEBUG [Thread-4102] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateInfoCache(250)) - Done 2015-11-22 19:29:56,787 DEBUG [Thread-4102] util.MBeans (MBeans.java:register(67)) - Registered Hadoop:service=ApplicationHistoryServer,name=RpcActivityForPort56823 2015-11-22 19:29:56,787 DEBUG [Thread-4102] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:startMBeans(227)) - MBean for source RpcActivityForPort56823 registered. 2015-11-22 19:29:56,787 DEBUG [Thread-4102] impl.MetricsSystemImpl (MetricsSystemImpl.java:registerSource(269)) - Registered source RpcActivityForPort56823 2015-11-22 19:29:56,787 DEBUG [Thread-4102] metrics.RpcDetailedMetrics (RpcDetailedMetrics.java:(47)) - MetricsInfoImpl{name=rpcdetailed, description=rpcdetailed} 2015-11-22 19:29:56,787 DEBUG [Thread-4102] lib.MutableMetricsFactory (MutableMetricsFactory.java:newForField(42)) - field org.apache.hadoop.metrics2.lib.MutableRates org.apache.hadoop.ipc.metrics.RpcDetailedMetrics.rates with annotation @org.apache.hadoop.metrics2.annotation.Metric(sampleName=Ops, always=false, about=, type=DEFAULT, valueName=Time, value=[]) 2015-11-22 19:29:56,787 DEBUG [Thread-4102] impl.MetricsSystemImpl (MetricsSystemImpl.java:register(232)) - RpcDetailedActivityForPort56823, Per method RPC metrics 2015-11-22 19:29:56,787 DEBUG [Thread-4102] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'PropertiesConfiguration' for key: source.source.start_mbeans 2015-11-22 19:29:56,787 DEBUG [Thread-4102] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'MetricsConfig' for key: source.start_mbeans 2015-11-22 19:29:56,787 DEBUG [Thread-4102] impl.MetricsConfig (MetricsConfig.java:getProperty(183)) - poking parent 'PropertiesConfiguration' for key: *.source.start_mbeans 2015-11-22 19:29:56,787 DEBUG [Thread-4102] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateAttrCache(254)) - Updating attr cache... 2015-11-22 19:29:56,787 DEBUG [Thread-4102] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateAttrCache(268)) - Done. # tags & metrics=3 2015-11-22 19:29:56,788 DEBUG [Thread-4102] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateInfoCache(248)) - Updating info cache... 2015-11-22 19:29:56,788 DEBUG [Thread-4102] impl.MetricsSystemImpl (MBeanInfoBuilder.java:get(109)) - [javax.management.MBeanAttributeInfo[description=RPC port, name=tag.port, type=java.lang.String, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Metrics context, name=tag.Context, type=java.lang.String, read-only, descriptor={}], javax.management.MBeanAttributeInfo[description=Local hostname, name=tag.Hostname, type=java.lang.String, read-only, descriptor={}]] 2015-11-22 19:29:56,788 DEBUG [Thread-4102] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:updateInfoCache(250)) - Done 2015-11-22 19:29:56,788 DEBUG [Thread-4102] util.MBeans (MBeans.java:register(67)) - Registered Hadoop:service=ApplicationHistoryServer,name=RpcDetailedActivityForPort56823 2015-11-22 19:29:56,788 DEBUG [Thread-4102] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:startMBeans(227)) - MBean for source RpcDetailedActivityForPort56823 registered. 2015-11-22 19:29:56,788 DEBUG [Thread-4102] impl.MetricsSystemImpl (MetricsSystemImpl.java:registerSource(269)) - Registered source RpcDetailedActivityForPort56823 2015-11-22 19:29:56,788 DEBUG [Thread-4102] ipc.Server (RPC.java:registerProtocolAndImpl(876)) - RpcKind = RPC_PROTOCOL_BUFFER Protocol Name = org.apache.hadoop.ipc.ProtocolMetaInfoPB version=1 ProtocolImpl=org.apache.hadoop.ipc.protobuf.ProtocolInfoProtos$ProtocolInfoService$2 protocolClass=org.apache.hadoop.ipc.ProtocolMetaInfoPB 2015-11-22 19:29:56,788 DEBUG [Thread-4102] ipc.Server (RPC.java:registerProtocolAndImpl(876)) - RpcKind = RPC_PROTOCOL_BUFFER Protocol Name = org.apache.hadoop.yarn.api.ApplicationHistoryProtocolPB version=1 ProtocolImpl=org.apache.hadoop.yarn.proto.ApplicationHistoryProtocol$ApplicationHistoryProtocolService$2 protocolClass=org.apache.hadoop.yarn.api.ApplicationHistoryProtocolPB 2015-11-22 19:29:56,788 INFO [Thread-4102] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.api.ApplicationHistoryProtocolPB to the server 2015-11-22 19:29:56,788 DEBUG [Thread-4102] ipc.Server (RPC.java:registerProtocolAndImpl(876)) - RpcKind = RPC_PROTOCOL_BUFFER Protocol Name = org.apache.hadoop.yarn.api.ApplicationHistoryProtocolPB version=1 ProtocolImpl=org.apache.hadoop.yarn.proto.ApplicationHistoryProtocol$ApplicationHistoryProtocolService$2 protocolClass=org.apache.hadoop.yarn.api.ApplicationHistoryProtocolPB 2015-11-22 19:29:56,789 INFO [IPC Server Responder] ipc.Server (Server.java:run(954)) - IPC Server Responder: starting 2015-11-22 19:29:56,789 INFO [IPC Server listener on 56823] ipc.Server (Server.java:run(794)) - IPC Server listener on 56823: starting 2015-11-22 19:29:56,789 DEBUG [IPC Server handler 1 on 56823] ipc.Server (Server.java:run(2176)) - IPC Server handler 1 on 56823: starting 2015-11-22 19:29:56,789 DEBUG [IPC Server handler 0 on 56823] ipc.Server (Server.java:run(2176)) - IPC Server handler 0 on 56823: starting 2015-11-22 19:29:56,790 DEBUG [IPC Server handler 4 on 56823] ipc.Server (Server.java:run(2176)) - IPC Server handler 4 on 56823: starting 2015-11-22 19:29:56,790 DEBUG [IPC Server handler 2 on 56823] ipc.Server (Server.java:run(2176)) - IPC Server handler 2 on 56823: starting 2015-11-22 19:29:56,791 DEBUG [IPC Server handler 7 on 56823] ipc.Server (Server.java:run(2176)) - IPC Server handler 7 on 56823: starting 2015-11-22 19:29:56,790 DEBUG [IPC Server handler 3 on 56823] ipc.Server (Server.java:run(2176)) - IPC Server handler 3 on 56823: starting 2015-11-22 19:29:56,791 DEBUG [IPC Server handler 9 on 56823] ipc.Server (Server.java:run(2176)) - IPC Server handler 9 on 56823: starting 2015-11-22 19:29:56,791 DEBUG [IPC Server handler 8 on 56823] ipc.Server (Server.java:run(2176)) - IPC Server handler 8 on 56823: starting 2015-11-22 19:29:56,791 DEBUG [IPC Server handler 6 on 56823] ipc.Server (Server.java:run(2176)) - IPC Server handler 6 on 56823: starting 2015-11-22 19:29:56,790 DEBUG [IPC Server handler 5 on 56823] ipc.Server (Server.java:run(2176)) - IPC Server handler 5 on 56823: starting 2015-11-22 19:29:56,791 INFO [Thread-4102] applicationhistoryservice.ApplicationHistoryClientService (ApplicationHistoryClientService.java:serviceStart(111)) - Instantiated ApplicationHistoryClientService at ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:56823 2015-11-22 19:29:56,791 DEBUG [Thread-4102] service.AbstractService (AbstractService.java:start(197)) - Service ApplicationHistoryClientService is started 2015-11-22 19:29:56,792 INFO [Thread-4102] applicationhistoryservice.ApplicationHistoryManagerImpl (ApplicationHistoryManagerImpl.java:serviceStart(71)) - Starting ApplicationHistory 2015-11-22 19:29:56,792 DEBUG [Thread-4102] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.applicationhistoryservice.MemoryApplicationHistoryStore is started 2015-11-22 19:29:56,792 DEBUG [Thread-4102] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryManagerImpl is started 2015-11-22 19:29:56,792 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap=null 2015-11-22 19:29:56,792 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=null 2015-11-22 19:29:56,792 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:56,792 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/=org.mortbay.jetty.servlet.DefaultServlet-1061138277} 2015-11-22 19:29:56,793 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-1061138277=org.mortbay.jetty.servlet.DefaultServlet-1061138277} 2015-11-22 19:29:56,793 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={NoCacheFilter=NoCacheFilter} 2015-11-22 19:29:56,793 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15)] 2015-11-22 19:29:56,793 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:56,793 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/=org.mortbay.jetty.servlet.DefaultServlet-1061138277} 2015-11-22 19:29:56,793 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-1061138277=org.mortbay.jetty.servlet.DefaultServlet-1061138277} 2015-11-22 19:29:56,793 INFO [Thread-4102] server.AuthenticationFilter (AuthenticationFilter.java:constructSecretProvider(304)) - Unable to initialize FileSignerSecretProvider, falling back to use random secrets. 2015-11-22 19:29:56,794 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container Server@1d16906a + org.mortbay.thread.QueuedThreadPool@eca50fb as threadpool 2015-11-22 19:29:56,794 INFO [Thread-4102] http.HttpRequestLog (HttpRequestLog.java:getRequestLog(80)) - Http request log for http.requests.applicationhistory is not defined 2015-11-22 19:29:56,794 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container Server@1d16906a + ContextHandlerCollection@5f10502a as handler 2015-11-22 19:29:56,794 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container ContextHandlerCollection@5f10502a + org.mortbay.jetty.webapp.WebAppContext@733f6120{/,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes/webapps/applicationhistory} as handler 2015-11-22 19:29:56,794 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 + NoCacheFilter as filter 2015-11-22 19:29:56,794 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 + (F=NoCacheFilter,[/*],[],15) as filterMapping 2015-11-22 19:29:56,794 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 + org.mortbay.jetty.servlet.DefaultServlet-1061138277 as servlet 2015-11-22 19:29:56,795 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 + (S=org.mortbay.jetty.servlet.DefaultServlet-1061138277,[/]) as servletMapping 2015-11-22 19:29:56,795 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container SecurityHandler@612fa7b + ServletHandler@2f2eba34 as handler 2015-11-22 19:29:56,795 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container SessionHandler@5b5e6b93 + SecurityHandler@612fa7b as handler 2015-11-22 19:29:56,795 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container SessionHandler@5b5e6b93 + org.mortbay.jetty.servlet.HashSessionManager@536027b9 as sessionManager 2015-11-22 19:29:56,795 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container org.mortbay.jetty.webapp.WebAppContext@733f6120{/,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes/webapps/applicationhistory} + SessionHandler@5b5e6b93 as handler 2015-11-22 19:29:56,795 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container org.mortbay.jetty.webapp.WebAppContext@733f6120{/,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes/webapps/applicationhistory} + ErrorPageErrorHandler@945d8fb as error 2015-11-22 19:29:56,795 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container ContextHandlerCollection@5f10502a + org.mortbay.jetty.servlet.Context@18153a8a{/logs,null} as handler 2015-11-22 19:29:56,795 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container org.mortbay.jetty.servlet.Context@18153a8a{/logs,null} + ServletHandler@303fd8e8 as handler 2015-11-22 19:29:56,795 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@303fd8e8 + org.apache.hadoop.http.AdminAuthorizedServlet-1655733433 as servlet 2015-11-22 19:29:56,795 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@303fd8e8 + (S=org.apache.hadoop.http.AdminAuthorizedServlet-1655733433,[/*]) as servletMapping 2015-11-22 19:29:56,796 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap=null 2015-11-22 19:29:56,796 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=null 2015-11-22 19:29:56,796 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:56,796 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/*=org.apache.hadoop.http.AdminAuthorizedServlet-1655733433} 2015-11-22 19:29:56,796 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.apache.hadoop.http.AdminAuthorizedServlet-1655733433=org.apache.hadoop.http.AdminAuthorizedServlet-1655733433} 2015-11-22 19:29:56,796 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 + NoCacheFilter as filter 2015-11-22 19:29:56,796 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 + (F=NoCacheFilter,[/*],[],15) as filterMapping 2015-11-22 19:29:56,796 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={NoCacheFilter=NoCacheFilter} 2015-11-22 19:29:56,796 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15)] 2015-11-22 19:29:56,796 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:56,796 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/=org.mortbay.jetty.servlet.DefaultServlet-1061138277} 2015-11-22 19:29:56,797 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-1061138277=org.mortbay.jetty.servlet.DefaultServlet-1061138277} 2015-11-22 19:29:56,797 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container ContextHandlerCollection@5f10502a + org.mortbay.jetty.servlet.Context@4515f214{/static,null} as handler 2015-11-22 19:29:56,797 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container org.mortbay.jetty.servlet.Context@4515f214{/static,null} + ServletHandler@74e55156 as handler 2015-11-22 19:29:56,797 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@74e55156 + org.mortbay.jetty.servlet.DefaultServlet-978534330 as servlet 2015-11-22 19:29:56,797 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@74e55156 + (S=org.mortbay.jetty.servlet.DefaultServlet-978534330,[/*]) as servletMapping 2015-11-22 19:29:56,797 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap=null 2015-11-22 19:29:56,797 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=null 2015-11-22 19:29:56,797 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:56,797 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/*=org.mortbay.jetty.servlet.DefaultServlet-978534330} 2015-11-22 19:29:56,797 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-978534330=org.mortbay.jetty.servlet.DefaultServlet-978534330} 2015-11-22 19:29:56,798 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 + safety as filter 2015-11-22 19:29:56,798 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 + (F=safety,[/*],[],15) as filterMapping 2015-11-22 19:29:56,798 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={safety=safety, NoCacheFilter=NoCacheFilter} 2015-11-22 19:29:56,798 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15)] 2015-11-22 19:29:56,798 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:56,798 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/=org.mortbay.jetty.servlet.DefaultServlet-1061138277} 2015-11-22 19:29:56,798 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-1061138277=org.mortbay.jetty.servlet.DefaultServlet-1061138277} 2015-11-22 19:29:56,798 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@74e55156 + safety as filter 2015-11-22 19:29:56,798 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@74e55156 + (F=safety,[/*],[],15) as filterMapping 2015-11-22 19:29:56,798 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={safety=safety} 2015-11-22 19:29:56,798 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=safety,[/*],[],15)] 2015-11-22 19:29:56,798 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:56,799 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/*=org.mortbay.jetty.servlet.DefaultServlet-978534330} 2015-11-22 19:29:56,799 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-978534330=org.mortbay.jetty.servlet.DefaultServlet-978534330} 2015-11-22 19:29:56,799 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@303fd8e8 + safety as filter 2015-11-22 19:29:56,799 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@303fd8e8 + (F=safety,[/*],[],15) as filterMapping 2015-11-22 19:29:56,799 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={safety=safety} 2015-11-22 19:29:56,799 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=safety,[/*],[],15)] 2015-11-22 19:29:56,799 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:56,799 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/*=org.apache.hadoop.http.AdminAuthorizedServlet-1655733433} 2015-11-22 19:29:56,799 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.apache.hadoop.http.AdminAuthorizedServlet-1655733433=org.apache.hadoop.http.AdminAuthorizedServlet-1655733433} 2015-11-22 19:29:56,799 INFO [Thread-4102] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter) 2015-11-22 19:29:56,800 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 + RMAuthenticationFilter as filter 2015-11-22 19:29:56,800 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 + (F=RMAuthenticationFilter,[*.html, *.jsp],[],15) as filterMapping 2015-11-22 19:29:56,800 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter} 2015-11-22 19:29:56,800 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15)] 2015-11-22 19:29:56,800 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:56,800 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/=org.mortbay.jetty.servlet.DefaultServlet-1061138277} 2015-11-22 19:29:56,800 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-1061138277=org.mortbay.jetty.servlet.DefaultServlet-1061138277} 2015-11-22 19:29:56,800 INFO [Thread-4102] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context applicationhistory 2015-11-22 19:29:56,800 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@74e55156 + RMAuthenticationFilter as filter 2015-11-22 19:29:56,801 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@74e55156 + (F=RMAuthenticationFilter,[/*],[],15) as filterMapping 2015-11-22 19:29:56,801 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={safety=safety, RMAuthenticationFilter=RMAuthenticationFilter} 2015-11-22 19:29:56,801 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=safety,[/*],[],15), (F=RMAuthenticationFilter,[/*],[],15)] 2015-11-22 19:29:56,801 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:56,801 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/*=org.mortbay.jetty.servlet.DefaultServlet-978534330} 2015-11-22 19:29:56,801 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-978534330=org.mortbay.jetty.servlet.DefaultServlet-978534330} 2015-11-22 19:29:56,801 INFO [Thread-4102] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context static 2015-11-22 19:29:56,801 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@303fd8e8 + RMAuthenticationFilter as filter 2015-11-22 19:29:56,801 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@303fd8e8 + (F=RMAuthenticationFilter,[/*],[],15) as filterMapping 2015-11-22 19:29:56,801 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={safety=safety, RMAuthenticationFilter=RMAuthenticationFilter} 2015-11-22 19:29:56,801 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=safety,[/*],[],15), (F=RMAuthenticationFilter,[/*],[],15)] 2015-11-22 19:29:56,801 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:56,802 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/*=org.apache.hadoop.http.AdminAuthorizedServlet-1655733433} 2015-11-22 19:29:56,802 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.apache.hadoop.http.AdminAuthorizedServlet-1655733433=org.apache.hadoop.http.AdminAuthorizedServlet-1655733433} 2015-11-22 19:29:56,802 INFO [Thread-4102] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context logs 2015-11-22 19:29:56,802 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 + static_user_filter as filter 2015-11-22 19:29:56,802 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 + (F=static_user_filter,[*.html, *.jsp],[],15) as filterMapping 2015-11-22 19:29:56,802 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:56,802 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15)] 2015-11-22 19:29:56,802 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:56,802 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/=org.mortbay.jetty.servlet.DefaultServlet-1061138277} 2015-11-22 19:29:56,802 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-1061138277=org.mortbay.jetty.servlet.DefaultServlet-1061138277} 2015-11-22 19:29:56,802 INFO [Thread-4102] http.HttpServer2 (HttpServer2.java:addFilter(687)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context applicationhistory 2015-11-22 19:29:56,802 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@74e55156 + static_user_filter as filter 2015-11-22 19:29:56,803 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@74e55156 + (F=static_user_filter,[/*],[],15) as filterMapping 2015-11-22 19:29:56,803 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:56,803 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=safety,[/*],[],15), (F=RMAuthenticationFilter,[/*],[],15), (F=static_user_filter,[/*],[],15)] 2015-11-22 19:29:56,803 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:56,803 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/*=org.mortbay.jetty.servlet.DefaultServlet-978534330} 2015-11-22 19:29:56,803 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-978534330=org.mortbay.jetty.servlet.DefaultServlet-978534330} 2015-11-22 19:29:56,803 INFO [Thread-4102] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2015-11-22 19:29:56,803 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@303fd8e8 + static_user_filter as filter 2015-11-22 19:29:56,803 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@303fd8e8 + (F=static_user_filter,[/*],[],15) as filterMapping 2015-11-22 19:29:56,803 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:56,803 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=safety,[/*],[],15), (F=RMAuthenticationFilter,[/*],[],15), (F=static_user_filter,[/*],[],15)] 2015-11-22 19:29:56,803 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:56,804 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/*=org.apache.hadoop.http.AdminAuthorizedServlet-1655733433} 2015-11-22 19:29:56,804 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.apache.hadoop.http.AdminAuthorizedServlet-1655733433=org.apache.hadoop.http.AdminAuthorizedServlet-1655733433} 2015-11-22 19:29:56,804 INFO [Thread-4102] http.HttpServer2 (HttpServer2.java:addFilter(695)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2015-11-22 19:29:56,804 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 + Timeline Authentication Filter as filter 2015-11-22 19:29:56,804 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 + (F=Timeline Authentication Filter,[/*],[],15) as filterMapping 2015-11-22 19:29:56,804 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:56,804 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15)] 2015-11-22 19:29:56,804 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:56,805 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/=org.mortbay.jetty.servlet.DefaultServlet-1061138277} 2015-11-22 19:29:56,805 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-1061138277=org.mortbay.jetty.servlet.DefaultServlet-1061138277} 2015-11-22 19:29:56,805 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@74e55156 + Timeline Authentication Filter as filter 2015-11-22 19:29:56,805 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@74e55156 + (F=Timeline Authentication Filter,[/*],[],15) as filterMapping 2015-11-22 19:29:56,805 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:56,805 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=safety,[/*],[],15), (F=RMAuthenticationFilter,[/*],[],15), (F=static_user_filter,[/*],[],15), (F=Timeline Authentication Filter,[/*],[],15)] 2015-11-22 19:29:56,805 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:56,805 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/*=org.mortbay.jetty.servlet.DefaultServlet-978534330} 2015-11-22 19:29:56,805 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-978534330=org.mortbay.jetty.servlet.DefaultServlet-978534330} 2015-11-22 19:29:56,805 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@303fd8e8 + Timeline Authentication Filter as filter 2015-11-22 19:29:56,805 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@303fd8e8 + (F=Timeline Authentication Filter,[/*],[],15) as filterMapping 2015-11-22 19:29:56,806 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:56,806 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=safety,[/*],[],15), (F=RMAuthenticationFilter,[/*],[],15), (F=static_user_filter,[/*],[],15), (F=Timeline Authentication Filter,[/*],[],15)] 2015-11-22 19:29:56,806 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:56,806 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/*=org.apache.hadoop.http.AdminAuthorizedServlet-1655733433} 2015-11-22 19:29:56,806 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.apache.hadoop.http.AdminAuthorizedServlet-1655733433=org.apache.hadoop.http.AdminAuthorizedServlet-1655733433} 2015-11-22 19:29:56,806 INFO [Thread-4102] http.HttpServer2 (HttpServer2.java:addGlobalFilter(712)) - Added global filter 'Timeline Authentication Filter' (class=org.apache.hadoop.yarn.server.timeline.security.TimelineAuthenticationFilter) 2015-11-22 19:29:56,806 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 + stacks as servlet 2015-11-22 19:29:56,806 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 + (S=stacks,[/stacks]) as servletMapping 2015-11-22 19:29:56,806 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:56,806 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15)] 2015-11-22 19:29:56,806 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:56,806 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/stacks=stacks, /=org.mortbay.jetty.servlet.DefaultServlet-1061138277} 2015-11-22 19:29:56,807 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={stacks=stacks, org.mortbay.jetty.servlet.DefaultServlet-1061138277=org.mortbay.jetty.servlet.DefaultServlet-1061138277} 2015-11-22 19:29:56,807 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 + (F=RMAuthenticationFilter,[/stacks],[],15) as filterMapping 2015-11-22 19:29:56,807 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:56,807 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15), (F=RMAuthenticationFilter,[/stacks],[],15)] 2015-11-22 19:29:56,807 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:56,807 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/stacks=stacks, /=org.mortbay.jetty.servlet.DefaultServlet-1061138277} 2015-11-22 19:29:56,807 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={stacks=stacks, org.mortbay.jetty.servlet.DefaultServlet-1061138277=org.mortbay.jetty.servlet.DefaultServlet-1061138277} 2015-11-22 19:29:56,807 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 + (F=static_user_filter,[/stacks],[],15) as filterMapping 2015-11-22 19:29:56,807 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:56,807 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15), (F=RMAuthenticationFilter,[/stacks],[],15), (F=static_user_filter,[/stacks],[],15)] 2015-11-22 19:29:56,807 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:56,807 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/stacks=stacks, /=org.mortbay.jetty.servlet.DefaultServlet-1061138277} 2015-11-22 19:29:56,807 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={stacks=stacks, org.mortbay.jetty.servlet.DefaultServlet-1061138277=org.mortbay.jetty.servlet.DefaultServlet-1061138277} 2015-11-22 19:29:56,807 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 + logLevel as servlet 2015-11-22 19:29:56,807 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 + (S=logLevel,[/logLevel]) as servletMapping 2015-11-22 19:29:56,808 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:56,808 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15), (F=RMAuthenticationFilter,[/stacks],[],15), (F=static_user_filter,[/stacks],[],15)] 2015-11-22 19:29:56,808 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:56,808 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/stacks=stacks, /logLevel=logLevel, /=org.mortbay.jetty.servlet.DefaultServlet-1061138277} 2015-11-22 19:29:56,808 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={logLevel=logLevel, stacks=stacks, org.mortbay.jetty.servlet.DefaultServlet-1061138277=org.mortbay.jetty.servlet.DefaultServlet-1061138277} 2015-11-22 19:29:56,808 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 + (F=RMAuthenticationFilter,[/logLevel],[],15) as filterMapping 2015-11-22 19:29:56,808 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:56,808 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15), (F=RMAuthenticationFilter,[/stacks],[],15), (F=static_user_filter,[/stacks],[],15), (F=RMAuthenticationFilter,[/logLevel],[],15)] 2015-11-22 19:29:56,808 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:56,808 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/stacks=stacks, /logLevel=logLevel, /=org.mortbay.jetty.servlet.DefaultServlet-1061138277} 2015-11-22 19:29:56,808 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={logLevel=logLevel, stacks=stacks, org.mortbay.jetty.servlet.DefaultServlet-1061138277=org.mortbay.jetty.servlet.DefaultServlet-1061138277} 2015-11-22 19:29:56,808 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 + (F=static_user_filter,[/logLevel],[],15) as filterMapping 2015-11-22 19:29:56,808 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:56,808 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15), (F=RMAuthenticationFilter,[/stacks],[],15), (F=static_user_filter,[/stacks],[],15), (F=RMAuthenticationFilter,[/logLevel],[],15), (F=static_user_filter,[/logLevel],[],15)] 2015-11-22 19:29:56,808 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:56,809 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/stacks=stacks, /logLevel=logLevel, /=org.mortbay.jetty.servlet.DefaultServlet-1061138277} 2015-11-22 19:29:56,809 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={logLevel=logLevel, stacks=stacks, org.mortbay.jetty.servlet.DefaultServlet-1061138277=org.mortbay.jetty.servlet.DefaultServlet-1061138277} 2015-11-22 19:29:56,809 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 + metrics as servlet 2015-11-22 19:29:56,809 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 + (S=metrics,[/metrics]) as servletMapping 2015-11-22 19:29:56,809 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:56,809 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15), (F=RMAuthenticationFilter,[/stacks],[],15), (F=static_user_filter,[/stacks],[],15), (F=RMAuthenticationFilter,[/logLevel],[],15), (F=static_user_filter,[/logLevel],[],15)] 2015-11-22 19:29:56,809 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:56,809 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/stacks=stacks, /logLevel=logLevel, /metrics=metrics, /=org.mortbay.jetty.servlet.DefaultServlet-1061138277} 2015-11-22 19:29:56,809 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={logLevel=logLevel, stacks=stacks, metrics=metrics, org.mortbay.jetty.servlet.DefaultServlet-1061138277=org.mortbay.jetty.servlet.DefaultServlet-1061138277} 2015-11-22 19:29:56,809 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 + (F=RMAuthenticationFilter,[/metrics],[],15) as filterMapping 2015-11-22 19:29:56,809 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:56,809 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15), (F=RMAuthenticationFilter,[/stacks],[],15), (F=static_user_filter,[/stacks],[],15), (F=RMAuthenticationFilter,[/logLevel],[],15), (F=static_user_filter,[/logLevel],[],15), (F=RMAuthenticationFilter,[/metrics],[],15)] 2015-11-22 19:29:56,809 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:56,809 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/stacks=stacks, /logLevel=logLevel, /metrics=metrics, /=org.mortbay.jetty.servlet.DefaultServlet-1061138277} 2015-11-22 19:29:56,809 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={logLevel=logLevel, stacks=stacks, metrics=metrics, org.mortbay.jetty.servlet.DefaultServlet-1061138277=org.mortbay.jetty.servlet.DefaultServlet-1061138277} 2015-11-22 19:29:56,810 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 + (F=static_user_filter,[/metrics],[],15) as filterMapping 2015-11-22 19:29:56,810 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:56,810 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15), (F=RMAuthenticationFilter,[/stacks],[],15), (F=static_user_filter,[/stacks],[],15), (F=RMAuthenticationFilter,[/logLevel],[],15), (F=static_user_filter,[/logLevel],[],15), (F=RMAuthenticationFilter,[/metrics],[],15), (F=static_user_filter,[/metrics],[],15)] 2015-11-22 19:29:56,810 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:56,810 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/stacks=stacks, /logLevel=logLevel, /metrics=metrics, /=org.mortbay.jetty.servlet.DefaultServlet-1061138277} 2015-11-22 19:29:56,810 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={logLevel=logLevel, stacks=stacks, metrics=metrics, org.mortbay.jetty.servlet.DefaultServlet-1061138277=org.mortbay.jetty.servlet.DefaultServlet-1061138277} 2015-11-22 19:29:56,810 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 + jmx as servlet 2015-11-22 19:29:56,810 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 + (S=jmx,[/jmx]) as servletMapping 2015-11-22 19:29:56,810 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:56,810 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15), (F=RMAuthenticationFilter,[/stacks],[],15), (F=static_user_filter,[/stacks],[],15), (F=RMAuthenticationFilter,[/logLevel],[],15), (F=static_user_filter,[/logLevel],[],15), (F=RMAuthenticationFilter,[/metrics],[],15), (F=static_user_filter,[/metrics],[],15)] 2015-11-22 19:29:56,810 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:56,810 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/jmx=jmx, /stacks=stacks, /logLevel=logLevel, /metrics=metrics, /=org.mortbay.jetty.servlet.DefaultServlet-1061138277} 2015-11-22 19:29:56,810 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={logLevel=logLevel, jmx=jmx, stacks=stacks, metrics=metrics, org.mortbay.jetty.servlet.DefaultServlet-1061138277=org.mortbay.jetty.servlet.DefaultServlet-1061138277} 2015-11-22 19:29:56,810 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 + (F=RMAuthenticationFilter,[/jmx],[],15) as filterMapping 2015-11-22 19:29:56,810 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:56,811 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15), (F=RMAuthenticationFilter,[/stacks],[],15), (F=static_user_filter,[/stacks],[],15), (F=RMAuthenticationFilter,[/logLevel],[],15), (F=static_user_filter,[/logLevel],[],15), (F=RMAuthenticationFilter,[/metrics],[],15), (F=static_user_filter,[/metrics],[],15), (F=RMAuthenticationFilter,[/jmx],[],15)] 2015-11-22 19:29:56,811 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:56,811 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/jmx=jmx, /stacks=stacks, /logLevel=logLevel, /metrics=metrics, /=org.mortbay.jetty.servlet.DefaultServlet-1061138277} 2015-11-22 19:29:56,811 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={logLevel=logLevel, jmx=jmx, stacks=stacks, metrics=metrics, org.mortbay.jetty.servlet.DefaultServlet-1061138277=org.mortbay.jetty.servlet.DefaultServlet-1061138277} 2015-11-22 19:29:56,811 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 + (F=static_user_filter,[/jmx],[],15) as filterMapping 2015-11-22 19:29:56,811 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:56,811 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15), (F=RMAuthenticationFilter,[/stacks],[],15), (F=static_user_filter,[/stacks],[],15), (F=RMAuthenticationFilter,[/logLevel],[],15), (F=static_user_filter,[/logLevel],[],15), (F=RMAuthenticationFilter,[/metrics],[],15), (F=static_user_filter,[/metrics],[],15), (F=RMAuthenticationFilter,[/jmx],[],15), (F=static_user_filter,[/jmx],[],15)] 2015-11-22 19:29:56,811 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:56,811 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/jmx=jmx, /stacks=stacks, /logLevel=logLevel, /metrics=metrics, /=org.mortbay.jetty.servlet.DefaultServlet-1061138277} 2015-11-22 19:29:56,811 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={logLevel=logLevel, jmx=jmx, stacks=stacks, metrics=metrics, org.mortbay.jetty.servlet.DefaultServlet-1061138277=org.mortbay.jetty.servlet.DefaultServlet-1061138277} 2015-11-22 19:29:56,811 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 + conf as servlet 2015-11-22 19:29:56,811 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 + (S=conf,[/conf]) as servletMapping 2015-11-22 19:29:56,811 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:56,811 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15), (F=RMAuthenticationFilter,[/stacks],[],15), (F=static_user_filter,[/stacks],[],15), (F=RMAuthenticationFilter,[/logLevel],[],15), (F=static_user_filter,[/logLevel],[],15), (F=RMAuthenticationFilter,[/metrics],[],15), (F=static_user_filter,[/metrics],[],15), (F=RMAuthenticationFilter,[/jmx],[],15), (F=static_user_filter,[/jmx],[],15)] 2015-11-22 19:29:56,812 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:56,812 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/jmx=jmx, /conf=conf, /stacks=stacks, /logLevel=logLevel, /metrics=metrics, /=org.mortbay.jetty.servlet.DefaultServlet-1061138277} 2015-11-22 19:29:56,812 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={logLevel=logLevel, jmx=jmx, stacks=stacks, conf=conf, metrics=metrics, org.mortbay.jetty.servlet.DefaultServlet-1061138277=org.mortbay.jetty.servlet.DefaultServlet-1061138277} 2015-11-22 19:29:56,812 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 + (F=RMAuthenticationFilter,[/conf],[],15) as filterMapping 2015-11-22 19:29:56,812 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:56,812 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15), (F=RMAuthenticationFilter,[/stacks],[],15), (F=static_user_filter,[/stacks],[],15), (F=RMAuthenticationFilter,[/logLevel],[],15), (F=static_user_filter,[/logLevel],[],15), (F=RMAuthenticationFilter,[/metrics],[],15), (F=static_user_filter,[/metrics],[],15), (F=RMAuthenticationFilter,[/jmx],[],15), (F=static_user_filter,[/jmx],[],15), (F=RMAuthenticationFilter,[/conf],[],15)] 2015-11-22 19:29:56,812 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:56,812 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/jmx=jmx, /conf=conf, /stacks=stacks, /logLevel=logLevel, /metrics=metrics, /=org.mortbay.jetty.servlet.DefaultServlet-1061138277} 2015-11-22 19:29:56,812 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={logLevel=logLevel, jmx=jmx, stacks=stacks, conf=conf, metrics=metrics, org.mortbay.jetty.servlet.DefaultServlet-1061138277=org.mortbay.jetty.servlet.DefaultServlet-1061138277} 2015-11-22 19:29:56,812 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 + (F=static_user_filter,[/conf],[],15) as filterMapping 2015-11-22 19:29:56,812 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:56,812 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15), (F=RMAuthenticationFilter,[/stacks],[],15), (F=static_user_filter,[/stacks],[],15), (F=RMAuthenticationFilter,[/logLevel],[],15), (F=static_user_filter,[/logLevel],[],15), (F=RMAuthenticationFilter,[/metrics],[],15), (F=static_user_filter,[/metrics],[],15), (F=RMAuthenticationFilter,[/jmx],[],15), (F=static_user_filter,[/jmx],[],15), (F=RMAuthenticationFilter,[/conf],[],15), (F=static_user_filter,[/conf],[],15)] 2015-11-22 19:29:56,812 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:56,812 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/jmx=jmx, /conf=conf, /stacks=stacks, /logLevel=logLevel, /metrics=metrics, /=org.mortbay.jetty.servlet.DefaultServlet-1061138277} 2015-11-22 19:29:56,812 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={logLevel=logLevel, jmx=jmx, stacks=stacks, conf=conf, metrics=metrics, org.mortbay.jetty.servlet.DefaultServlet-1061138277=org.mortbay.jetty.servlet.DefaultServlet-1061138277} 2015-11-22 19:29:56,813 INFO [Thread-4102] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /applicationhistory/* 2015-11-22 19:29:56,813 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 + (F=RMAuthenticationFilter,[/applicationhistory/*],[],15) as filterMapping 2015-11-22 19:29:56,813 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:56,813 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15), (F=RMAuthenticationFilter,[/stacks],[],15), (F=static_user_filter,[/stacks],[],15), (F=RMAuthenticationFilter,[/logLevel],[],15), (F=static_user_filter,[/logLevel],[],15), (F=RMAuthenticationFilter,[/metrics],[],15), (F=static_user_filter,[/metrics],[],15), (F=RMAuthenticationFilter,[/jmx],[],15), (F=static_user_filter,[/jmx],[],15), (F=RMAuthenticationFilter,[/conf],[],15), (F=static_user_filter,[/conf],[],15), (F=RMAuthenticationFilter,[/applicationhistory/*],[],15)] 2015-11-22 19:29:56,813 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:56,813 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/jmx=jmx, /conf=conf, /stacks=stacks, /logLevel=logLevel, /metrics=metrics, /=org.mortbay.jetty.servlet.DefaultServlet-1061138277} 2015-11-22 19:29:56,813 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={logLevel=logLevel, jmx=jmx, stacks=stacks, conf=conf, metrics=metrics, org.mortbay.jetty.servlet.DefaultServlet-1061138277=org.mortbay.jetty.servlet.DefaultServlet-1061138277} 2015-11-22 19:29:56,813 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 + (F=static_user_filter,[/applicationhistory/*],[],15) as filterMapping 2015-11-22 19:29:56,813 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:56,813 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15), (F=RMAuthenticationFilter,[/stacks],[],15), (F=static_user_filter,[/stacks],[],15), (F=RMAuthenticationFilter,[/logLevel],[],15), (F=static_user_filter,[/logLevel],[],15), (F=RMAuthenticationFilter,[/metrics],[],15), (F=static_user_filter,[/metrics],[],15), (F=RMAuthenticationFilter,[/jmx],[],15), (F=static_user_filter,[/jmx],[],15), (F=RMAuthenticationFilter,[/conf],[],15), (F=static_user_filter,[/conf],[],15), (F=RMAuthenticationFilter,[/applicationhistory/*],[],15), (F=static_user_filter,[/applicationhistory/*],[],15)] 2015-11-22 19:29:56,813 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:56,813 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/jmx=jmx, /conf=conf, /stacks=stacks, /logLevel=logLevel, /metrics=metrics, /=org.mortbay.jetty.servlet.DefaultServlet-1061138277} 2015-11-22 19:29:56,813 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={logLevel=logLevel, jmx=jmx, stacks=stacks, conf=conf, metrics=metrics, org.mortbay.jetty.servlet.DefaultServlet-1061138277=org.mortbay.jetty.servlet.DefaultServlet-1061138277} 2015-11-22 19:29:56,813 INFO [Thread-4102] http.HttpServer2 (HttpServer2.java:initializeWebServer(408)) - adding path spec: /ws/* 2015-11-22 19:29:56,813 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 + (F=RMAuthenticationFilter,[/ws/*],[],15) as filterMapping 2015-11-22 19:29:56,814 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:56,814 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15), (F=RMAuthenticationFilter,[/stacks],[],15), (F=static_user_filter,[/stacks],[],15), (F=RMAuthenticationFilter,[/logLevel],[],15), (F=static_user_filter,[/logLevel],[],15), (F=RMAuthenticationFilter,[/metrics],[],15), (F=static_user_filter,[/metrics],[],15), (F=RMAuthenticationFilter,[/jmx],[],15), (F=static_user_filter,[/jmx],[],15), (F=RMAuthenticationFilter,[/conf],[],15), (F=static_user_filter,[/conf],[],15), (F=RMAuthenticationFilter,[/applicationhistory/*],[],15), (F=static_user_filter,[/applicationhistory/*],[],15), (F=RMAuthenticationFilter,[/ws/*],[],15)] 2015-11-22 19:29:56,814 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:56,814 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/jmx=jmx, /conf=conf, /stacks=stacks, /logLevel=logLevel, /metrics=metrics, /=org.mortbay.jetty.servlet.DefaultServlet-1061138277} 2015-11-22 19:29:56,814 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={logLevel=logLevel, jmx=jmx, stacks=stacks, conf=conf, metrics=metrics, org.mortbay.jetty.servlet.DefaultServlet-1061138277=org.mortbay.jetty.servlet.DefaultServlet-1061138277} 2015-11-22 19:29:56,814 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 + (F=static_user_filter,[/ws/*],[],15) as filterMapping 2015-11-22 19:29:56,814 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:56,814 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15), (F=RMAuthenticationFilter,[/stacks],[],15), (F=static_user_filter,[/stacks],[],15), (F=RMAuthenticationFilter,[/logLevel],[],15), (F=static_user_filter,[/logLevel],[],15), (F=RMAuthenticationFilter,[/metrics],[],15), (F=static_user_filter,[/metrics],[],15), (F=RMAuthenticationFilter,[/jmx],[],15), (F=static_user_filter,[/jmx],[],15), (F=RMAuthenticationFilter,[/conf],[],15), (F=static_user_filter,[/conf],[],15), (F=RMAuthenticationFilter,[/applicationhistory/*],[],15), (F=static_user_filter,[/applicationhistory/*],[],15), (F=RMAuthenticationFilter,[/ws/*],[],15), (F=static_user_filter,[/ws/*],[],15)] 2015-11-22 19:29:56,814 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:56,814 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/jmx=jmx, /conf=conf, /stacks=stacks, /logLevel=logLevel, /metrics=metrics, /=org.mortbay.jetty.servlet.DefaultServlet-1061138277} 2015-11-22 19:29:56,814 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={logLevel=logLevel, jmx=jmx, stacks=stacks, conf=conf, metrics=metrics, org.mortbay.jetty.servlet.DefaultServlet-1061138277=org.mortbay.jetty.servlet.DefaultServlet-1061138277} 2015-11-22 19:29:56,814 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container Server@1d16906a + HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:0 as connector 2015-11-22 19:29:56,814 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 + guice as filter 2015-11-22 19:29:56,814 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 + (F=guice,[/*],[],15) as filterMapping 2015-11-22 19:29:56,814 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter, guice=guice} 2015-11-22 19:29:56,815 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15), (F=RMAuthenticationFilter,[/stacks],[],15), (F=static_user_filter,[/stacks],[],15), (F=RMAuthenticationFilter,[/logLevel],[],15), (F=static_user_filter,[/logLevel],[],15), (F=RMAuthenticationFilter,[/metrics],[],15), (F=static_user_filter,[/metrics],[],15), (F=RMAuthenticationFilter,[/jmx],[],15), (F=static_user_filter,[/jmx],[],15), (F=RMAuthenticationFilter,[/conf],[],15), (F=static_user_filter,[/conf],[],15), (F=RMAuthenticationFilter,[/applicationhistory/*],[],15), (F=static_user_filter,[/applicationhistory/*],[],15), (F=RMAuthenticationFilter,[/ws/*],[],15), (F=static_user_filter,[/ws/*],[],15), (F=guice,[/*],[],15)] 2015-11-22 19:29:56,815 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:56,815 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/jmx=jmx, /conf=conf, /stacks=stacks, /logLevel=logLevel, /metrics=metrics, /=org.mortbay.jetty.servlet.DefaultServlet-1061138277} 2015-11-22 19:29:56,815 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={logLevel=logLevel, jmx=jmx, stacks=stacks, conf=conf, metrics=metrics, org.mortbay.jetty.servlet.DefaultServlet-1061138277=org.mortbay.jetty.servlet.DefaultServlet-1061138277} 2015-11-22 19:29:56,816 DEBUG [Thread-4102] webapp.Router (Router.java:add(85)) - adding /([])->class org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.AHSController#index 2015-11-22 19:29:56,816 DEBUG [Thread-4102] webapp.Router (Router.java:load(267)) - trying: org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.AHSView 2015-11-22 19:29:56,816 DEBUG [Thread-4102] webapp.Router (Router.java:load(271)) - found org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.AHSView 2015-11-22 19:29:56,816 DEBUG [Thread-4102] webapp.Router (Router.java:add(85)) - adding /about([])->class org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.AHSController#about 2015-11-22 19:29:56,816 DEBUG [Thread-4102] webapp.Router (Router.java:load(267)) - trying: org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.AHSView 2015-11-22 19:29:56,816 DEBUG [Thread-4102] webapp.Router (Router.java:load(271)) - found org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.AHSView 2015-11-22 19:29:56,816 DEBUG [Thread-4102] webapp.Router (Router.java:add(85)) - adding /apps([:app.state])->class org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.AHSController#index 2015-11-22 19:29:56,816 DEBUG [Thread-4102] webapp.Router (Router.java:load(267)) - trying: org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.AHSView 2015-11-22 19:29:56,817 DEBUG [Thread-4102] webapp.Router (Router.java:load(271)) - found org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.AHSView 2015-11-22 19:29:56,817 DEBUG [Thread-4102] webapp.Router (Router.java:add(85)) - adding /app([:app.id])->class org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.AHSController#app 2015-11-22 19:29:56,817 DEBUG [Thread-4102] webapp.Router (Router.java:load(267)) - trying: org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.AHSView 2015-11-22 19:29:56,817 DEBUG [Thread-4102] webapp.Router (Router.java:load(271)) - found org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.AHSView 2015-11-22 19:29:56,817 DEBUG [Thread-4102] webapp.Router (Router.java:add(85)) - adding /appattempt([:appattempt.id])->class org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.AHSController#appattempt 2015-11-22 19:29:56,817 DEBUG [Thread-4102] webapp.Router (Router.java:load(267)) - trying: org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.AHSView 2015-11-22 19:29:56,817 DEBUG [Thread-4102] webapp.Router (Router.java:load(271)) - found org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.AHSView 2015-11-22 19:29:56,817 DEBUG [Thread-4102] webapp.Router (Router.java:add(85)) - adding /container([:container.id])->class org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.AHSController#container 2015-11-22 19:29:56,817 DEBUG [Thread-4102] webapp.Router (Router.java:load(267)) - trying: org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.AHSView 2015-11-22 19:29:56,818 DEBUG [Thread-4102] webapp.Router (Router.java:load(271)) - found org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.AHSView 2015-11-22 19:29:56,818 DEBUG [Thread-4102] webapp.Router (Router.java:add(85)) - adding /logs([:nm.id, :container.id, :entity.string, :app.owner, :log.type])->class org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.AHSController#logs 2015-11-22 19:29:56,818 DEBUG [Thread-4102] webapp.Router (Router.java:load(267)) - trying: org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.AHSView 2015-11-22 19:29:56,818 DEBUG [Thread-4102] webapp.Router (Router.java:load(271)) - found org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.AHSView 2015-11-22 19:29:56,818 DEBUG [Thread-4102] webapp.Router (Router.java:add(85)) - adding /errors-and-warnings([])->class org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.AHSController#errorsAndWarnings 2015-11-22 19:29:56,818 DEBUG [Thread-4102] webapp.Router (Router.java:load(267)) - trying: org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.AHSView 2015-11-22 19:29:56,818 DEBUG [Thread-4102] webapp.Router (Router.java:load(271)) - found org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.AHSView Nov 22, 2015 7:29:56 PM com.google.inject.servlet.GuiceFilter setPipeline WARNING: Multiple Servlet injectors detected. This is a warning indicating that you have more than one GuiceFilter running in your web application. If this is deliberate, you may safely ignore this message. If this is NOT deliberate however, your application may not work as expected. 2015-11-22 19:29:56,821 INFO [Thread-4102] webapp.WebApps (WebApps.java:build(288)) - Registered webapp guice modules 2015-11-22 19:29:56,823 INFO [Thread-4102] http.HttpServer2 (HttpServer2.java:openListeners(917)) - Jetty bound to port 49959 2015-11-22 19:29:56,823 INFO [Thread-4102] mortbay.log (Slf4jLog.java:info(67)) - jetty-6.1.26 2015-11-22 19:29:56,823 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - started org.mortbay.thread.QueuedThreadPool@eca50fb 2015-11-22 19:29:56,825 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Thread Context class loader is: ContextLoader@applicationhistory([]) / sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,825 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Parent class loader is: sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,825 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Parent class loader is: sun.misc.Launcher$ExtClassLoader@2de8284b 2015-11-22 19:29:56,825 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Try webapp=file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes/webapps/applicationhistory/, exists=true, directory=true 2015-11-22 19:29:56,825 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - webapp=file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes/webapps/applicationhistory/ 2015-11-22 19:29:56,826 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Created temp dir /tmp/Jetty_ip.172.31.20.42_ap.northeast.1_compute_internal_49959_applicationhistory____.msuaei for org.mortbay.jetty.webapp.WebAppContext@733f6120{/,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes/webapps/applicationhistory} 2015-11-22 19:29:56,826 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Configuring web-jetty.xml 2015-11-22 19:29:56,826 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - TLD search of file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/surefire/surefirebooter2438562254715847001.jar 2015-11-22 19:29:56,826 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - TLD search of file:/usr/lib/jvm/java-8-oracle/jre/lib/ext/zipfs.jar 2015-11-22 19:29:56,827 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - TLD search of file:/usr/lib/jvm/java-8-oracle/jre/lib/ext/sunjce_provider.jar 2015-11-22 19:29:56,827 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - TLD search of file:/usr/lib/jvm/java-8-oracle/jre/lib/ext/sunpkcs11.jar 2015-11-22 19:29:56,827 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - TLD search of file:/usr/lib/jvm/java-8-oracle/jre/lib/ext/sunec.jar 2015-11-22 19:29:56,827 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - TLD search of file:/usr/lib/jvm/java-8-oracle/jre/lib/ext/dnsns.jar 2015-11-22 19:29:56,828 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - TLD search of file:/usr/lib/jvm/java-8-oracle/jre/lib/ext/localedata.jar 2015-11-22 19:29:56,828 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - TLD search of file:/usr/lib/jvm/java-8-oracle/jre/lib/ext/jfxrt.jar 2015-11-22 19:29:56,837 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - TLD search of file:/usr/lib/jvm/java-8-oracle/jre/lib/ext/cldrdata.jar 2015-11-22 19:29:56,838 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - TLD search of file:/usr/lib/jvm/java-8-oracle/jre/lib/ext/nashorn.jar 2015-11-22 19:29:56,839 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.org.apache.xerces.internal.jaxp.SAXParserFactoryImpl from null 2015-11-22 19:29:56,840 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.org.apache.xerces.internal.impl.dv.dtd.DTDDVFactoryImpl from null 2015-11-22 19:29:56,840 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - getResource(javax/servlet/jsp/resources/web-jsptaglibrary_1_1.dtd)=jar:file:/home/ubuntu/.m2/repository/javax/servlet/jsp/jsp-api/2.1/jsp-api-2.1.jar!/javax/servlet/jsp/resources/web-jsptaglibrary_1_1.dtd 2015-11-22 19:29:56,840 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - getResource(javax/servlet/jsp/resources/web-jsptaglibrary_1_2.dtd)=jar:file:/home/ubuntu/.m2/repository/javax/servlet/jsp/jsp-api/2.1/jsp-api-2.1.jar!/javax/servlet/jsp/resources/web-jsptaglibrary_1_2.dtd 2015-11-22 19:29:56,840 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - getResource(javax/servlet/jsp/resources/web-jsptaglibrary_2_0.xsd)=jar:file:/home/ubuntu/.m2/repository/javax/servlet/jsp/jsp-api/2.1/jsp-api-2.1.jar!/javax/servlet/jsp/resources/web-jsptaglibrary_2_0.xsd 2015-11-22 19:29:56,840 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - getResource(javax/servlet/jsp/resources/web-jsptaglibrary_1_1.dtd)=jar:file:/home/ubuntu/.m2/repository/javax/servlet/jsp/jsp-api/2.1/jsp-api-2.1.jar!/javax/servlet/jsp/resources/web-jsptaglibrary_1_1.dtd 2015-11-22 19:29:56,840 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - getResource(javax/servlet/jsp/resources/web-jsptaglibrary_1_2.dtd)=jar:file:/home/ubuntu/.m2/repository/javax/servlet/jsp/jsp-api/2.1/jsp-api-2.1.jar!/javax/servlet/jsp/resources/web-jsptaglibrary_1_2.dtd 2015-11-22 19:29:56,841 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - getResource(javax/servlet/jsp/resources/web-jsptaglibrary_2_0.xsd)=jar:file:/home/ubuntu/.m2/repository/javax/servlet/jsp/jsp-api/2.1/jsp-api-2.1.jar!/javax/servlet/jsp/resources/web-jsptaglibrary_2_0.xsd 2015-11-22 19:29:56,841 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container Server@1d16906a + org.mortbay.jetty.servlet.HashSessionIdManager@1fe1b8b1 as sessionIdManager 2015-11-22 19:29:56,841 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Init SecureRandom. 2015-11-22 19:29:56,841 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - started org.mortbay.jetty.servlet.HashSessionIdManager@1fe1b8b1 2015-11-22 19:29:56,841 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - started org.mortbay.jetty.servlet.HashSessionManager@536027b9 2015-11-22 19:29:56,841 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, NoCacheFilter=NoCacheFilter, static_user_filter=static_user_filter, guice=guice} 2015-11-22 19:29:56,841 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=NoCacheFilter,[/*],[],15), (F=NoCacheFilter,[/*],[],15), (F=safety,[/*],[],15), (F=RMAuthenticationFilter,[*.html, *.jsp],[],15), (F=static_user_filter,[*.html, *.jsp],[],15), (F=Timeline Authentication Filter,[/*],[],15), (F=RMAuthenticationFilter,[/stacks],[],15), (F=static_user_filter,[/stacks],[],15), (F=RMAuthenticationFilter,[/logLevel],[],15), (F=static_user_filter,[/logLevel],[],15), (F=RMAuthenticationFilter,[/metrics],[],15), (F=static_user_filter,[/metrics],[],15), (F=RMAuthenticationFilter,[/jmx],[],15), (F=static_user_filter,[/jmx],[],15), (F=RMAuthenticationFilter,[/conf],[],15), (F=static_user_filter,[/conf],[],15), (F=RMAuthenticationFilter,[/applicationhistory/*],[],15), (F=static_user_filter,[/applicationhistory/*],[],15), (F=RMAuthenticationFilter,[/ws/*],[],15), (F=static_user_filter,[/ws/*],[],15), (F=guice,[/*],[],15)] 2015-11-22 19:29:56,841 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:56,841 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/jmx=jmx, /conf=conf, /stacks=stacks, /logLevel=logLevel, /metrics=metrics, /=org.mortbay.jetty.servlet.DefaultServlet-1061138277} 2015-11-22 19:29:56,841 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={logLevel=logLevel, jmx=jmx, stacks=stacks, conf=conf, metrics=metrics, org.mortbay.jetty.servlet.DefaultServlet-1061138277=org.mortbay.jetty.servlet.DefaultServlet-1061138277} 2015-11-22 19:29:56,842 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - starting ServletHandler@2f2eba34 2015-11-22 19:29:56,842 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - started ServletHandler@2f2eba34 2015-11-22 19:29:56,842 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - starting SecurityHandler@612fa7b 2015-11-22 19:29:56,842 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - started SecurityHandler@612fa7b 2015-11-22 19:29:56,842 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - starting SessionHandler@5b5e6b93 2015-11-22 19:29:56,842 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - started SessionHandler@5b5e6b93 2015-11-22 19:29:56,842 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - starting org.mortbay.jetty.webapp.WebAppContext@733f6120{/,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes/webapps/applicationhistory} 2015-11-22 19:29:56,842 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - starting ErrorPageErrorHandler@945d8fb 2015-11-22 19:29:56,842 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - started ErrorPageErrorHandler@945d8fb 2015-11-22 19:29:56,842 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class org.apache.hadoop.http.NoCacheFilter from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,842 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Holding class org.apache.hadoop.http.NoCacheFilter 2015-11-22 19:29:56,842 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - started NoCacheFilter 2015-11-22 19:29:56,842 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class org.apache.hadoop.http.NoCacheFilter from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,842 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Holding class org.apache.hadoop.http.NoCacheFilter 2015-11-22 19:29:56,843 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - started NoCacheFilter 2015-11-22 19:29:56,843 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class org.apache.hadoop.http.HttpServer2$QuotingInputFilter from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,843 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Holding class org.apache.hadoop.http.HttpServer2$QuotingInputFilter 2015-11-22 19:29:56,843 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - started safety 2015-11-22 19:29:56,843 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,843 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Holding class org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter 2015-11-22 19:29:56,843 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class org.apache.hadoop.security.token.delegation.web.PseudoDelegationTokenAuthenticationHandler from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,843 INFO [Thread-4102] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:29:56,844 INFO [Thread[Thread-4122,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2015-11-22 19:29:56,844 INFO [Thread[Thread-4122,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:29:56,845 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - started RMAuthenticationFilter 2015-11-22 19:29:56,845 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,845 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Holding class org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter 2015-11-22 19:29:56,845 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - started static_user_filter 2015-11-22 19:29:56,845 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class org.apache.hadoop.yarn.server.timeline.security.TimelineAuthenticationFilter from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,845 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Holding class org.apache.hadoop.yarn.server.timeline.security.TimelineAuthenticationFilter 2015-11-22 19:29:56,845 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class org.apache.hadoop.security.token.delegation.web.PseudoDelegationTokenAuthenticationHandler from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,846 INFO [Thread-4102] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(336)) - Updating the current master key for generating delegation tokens 2015-11-22 19:29:56,846 DEBUG [Thread-4102] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:stopThreads(619)) - Stopping expired delegation token remover thread 2015-11-22 19:29:56,846 INFO [Thread[Thread-4124,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(649)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2015-11-22 19:29:56,846 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - started Timeline Authentication Filter 2015-11-22 19:29:56,846 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.google.inject.servlet.GuiceFilter from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,846 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Holding class com.google.inject.servlet.GuiceFilter 2015-11-22 19:29:56,847 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.server.impl.container.WebApplicationProviderImpl from sun.misc.Launcher$AppClassLoader@42a57993 Nov 22, 2015 7:29:56 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider as a provider class Nov 22, 2015 7:29:56 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.AHSWebServices as a root resource class Nov 22, 2015 7:29:56 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.timeline.webapp.TimelineWebServices as a root resource class Nov 22, 2015 7:29:56 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a provider class Nov 22, 2015 7:29:56 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate INFO: Initiating Jersey application, version 'Jersey: 1.9 09/02/2011 11:17 AM' 2015-11-22 19:29:56,850 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.server.impl.model.method.dispatch.VoidVoidDispatchProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,850 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.server.impl.model.method.dispatch.HttpReqResDispatchProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,850 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.server.impl.model.method.dispatch.MultipartFormDispatchProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,850 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.server.impl.model.method.dispatch.FormDispatchProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,850 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.server.impl.model.method.dispatch.EntityParamDispatchProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,850 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded interface javax.annotation.PostConstruct 2015-11-22 19:29:56,851 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded interface javax.annotation.PostConstruct from null 2015-11-22 19:29:56,851 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded interface javax.annotation.PreDestroy 2015-11-22 19:29:56,851 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded interface javax.annotation.PreDestroy from null 2015-11-22 19:29:56,852 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.xml.SAXParserContextProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,853 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.xml.XMLStreamReaderContextProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,853 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.xml.DocumentBuilderFactoryProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,853 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.xml.TransformerFactoryProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,855 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.api.container.filter.GZIPContentEncodingFilter from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,856 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.server.impl.container.filter.NormalizeFilter from sun.misc.Launcher$AppClassLoader@42a57993 Nov 22, 2015 7:29:56 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.webapp.GenericExceptionHandler to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:29:56 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider to GuiceManagedComponentProvider with the scope "Singleton" 2015-11-22 19:29:56,859 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.StringProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,859 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.ByteArrayProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,859 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.FileProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,859 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.InputStreamProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,859 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.DataSourceProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,860 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.RenderedImageProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,860 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.MimeMultipartProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,860 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.FormProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,860 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.FormMultivaluedMapProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,860 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.XMLRootElementProvider$App from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,860 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.XMLRootElementProvider$Text from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,860 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.XMLRootElementProvider$General from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,861 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.XMLJAXBElementProvider$App from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,861 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.XMLJAXBElementProvider$Text from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,861 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.XMLJAXBElementProvider$General from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,861 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.XMLListElementProvider$App from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,861 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.XMLListElementProvider$Text from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,861 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.XMLListElementProvider$General from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,861 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.ReaderProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,862 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.DocumentProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,862 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.SourceProvider$StreamSourceReader from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,862 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.SourceProvider$SAXSourceReader from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,862 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.SourceProvider$DOMSourceReader from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,862 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.XMLRootObjectProvider$App from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,862 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.XMLRootObjectProvider$Text from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,862 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.XMLRootObjectProvider$General from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,863 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.EntityHolderReader from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,863 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.json.impl.provider.entity.JSONRootElementProvider$App from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,863 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.json.impl.provider.entity.JSONRootElementProvider$General from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,863 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.json.impl.provider.entity.JSONJAXBElementProvider$App from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,863 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.json.impl.provider.entity.JSONJAXBElementProvider$General from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,863 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.json.impl.provider.entity.JSONListElementProvider$App from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,863 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.json.impl.provider.entity.JSONListElementProvider$General from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,864 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.json.impl.provider.entity.JSONArrayProvider$App from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,864 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.json.impl.provider.entity.JSONArrayProvider$General from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,864 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.json.impl.provider.entity.JSONObjectProvider$App from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,864 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.json.impl.provider.entity.JSONObjectProvider$General from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,864 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.json.impl.provider.entity.JacksonProviderProxy from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,891 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.StreamingOutputProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,891 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.core.impl.provider.entity.SourceProvider$SourceWriter from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,891 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.json.impl.provider.entity.JSONWithPaddingProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,891 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.server.impl.template.ViewableMessageBodyWriter from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,894 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.server.impl.model.parameter.multivalued.StringReaderProviders$TypeFromStringEnum from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,894 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.server.impl.model.parameter.multivalued.StringReaderProviders$TypeValueOf from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,894 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.server.impl.model.parameter.multivalued.StringReaderProviders$TypeFromString from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,894 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.server.impl.model.parameter.multivalued.StringReaderProviders$StringConstructor from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,895 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.server.impl.model.parameter.multivalued.StringReaderProviders$DateProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,895 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.jersey.server.impl.model.parameter.multivalued.JAXBStringReaderProviders$RootElementProvider from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,905 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - getResource(META-INF/services/javax.xml.bind.JAXBContext)=jar:file:/home/ubuntu/.m2/repository/com/sun/xml/bind/jaxb-impl/2.2.3-1/jaxb-impl-2.2.3-1.jar!/META-INF/services/javax.xml.bind.JAXBContext 2015-11-22 19:29:56,905 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.xml.bind.v2.ContextFactory from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:29:56,905 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.research.ws.wadl.ObjectFactory from sun.misc.Launcher$AppClassLoader@42a57993 Nov 22, 2015 7:29:56 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.AHSWebServices to GuiceManagedComponentProvider with the scope "Singleton" Nov 22, 2015 7:29:56 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.timeline.webapp.TimelineWebServices to GuiceManagedComponentProvider with the scope "Singleton" 2015-11-22 19:29:56,926 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - started guice 2015-11-22 19:29:56,926 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - started conf 2015-11-22 19:29:56,926 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - started stacks 2015-11-22 19:29:56,926 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - started jmx 2015-11-22 19:29:56,926 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - started logLevel 2015-11-22 19:29:56,926 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - started metrics 2015-11-22 19:29:56,926 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - started org.mortbay.jetty.servlet.DefaultServlet$NIOResourceCache@79e2192 2015-11-22 19:29:56,926 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - resource base = file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes/webapps/applicationhistory/ 2015-11-22 19:29:56,927 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - started org.mortbay.jetty.servlet.DefaultServlet-1061138277 2015-11-22 19:29:56,927 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - started org.mortbay.jetty.webapp.WebAppContext@733f6120{/,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes/webapps/applicationhistory} 2015-11-22 19:29:56,927 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container org.mortbay.jetty.servlet.Context@18153a8a{/logs,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/log} + ErrorHandler@7dd52434 as errorHandler 2015-11-22 19:29:56,927 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:56,927 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=safety,[/*],[],15), (F=RMAuthenticationFilter,[/*],[],15), (F=static_user_filter,[/*],[],15), (F=Timeline Authentication Filter,[/*],[],15)] 2015-11-22 19:29:56,927 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:56,927 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/*=org.apache.hadoop.http.AdminAuthorizedServlet-1655733433} 2015-11-22 19:29:56,927 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.apache.hadoop.http.AdminAuthorizedServlet-1655733433=org.apache.hadoop.http.AdminAuthorizedServlet-1655733433} 2015-11-22 19:29:56,927 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - starting ServletHandler@303fd8e8 2015-11-22 19:29:56,927 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - started ServletHandler@303fd8e8 2015-11-22 19:29:56,927 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - starting org.mortbay.jetty.servlet.Context@18153a8a{/logs,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/log} 2015-11-22 19:29:56,927 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - starting ErrorHandler@7dd52434 2015-11-22 19:29:56,927 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - started ErrorHandler@7dd52434 2015-11-22 19:29:56,927 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Holding class org.apache.hadoop.http.AdminAuthorizedServlet 2015-11-22 19:29:56,928 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - started org.apache.hadoop.http.AdminAuthorizedServlet-1655733433 2015-11-22 19:29:56,928 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - started org.mortbay.jetty.servlet.Context@18153a8a{/logs,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/log} 2015-11-22 19:29:56,928 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Container org.mortbay.jetty.servlet.Context@4515f214{/static,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes/webapps/static/} + ErrorHandler@f815e6b as errorHandler 2015-11-22 19:29:56,928 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap={Timeline Authentication Filter=Timeline Authentication Filter, safety=safety, RMAuthenticationFilter=RMAuthenticationFilter, static_user_filter=static_user_filter} 2015-11-22 19:29:56,928 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=[(F=safety,[/*],[],15), (F=RMAuthenticationFilter,[/*],[],15), (F=static_user_filter,[/*],[],15), (F=Timeline Authentication Filter,[/*],[],15)] 2015-11-22 19:29:56,928 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:29:56,928 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/*=org.mortbay.jetty.servlet.DefaultServlet-978534330} 2015-11-22 19:29:56,928 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-978534330=org.mortbay.jetty.servlet.DefaultServlet-978534330} 2015-11-22 19:29:56,928 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - starting ServletHandler@74e55156 2015-11-22 19:29:56,928 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - started ServletHandler@74e55156 2015-11-22 19:29:56,928 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - starting org.mortbay.jetty.servlet.Context@4515f214{/static,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes/webapps/static/} 2015-11-22 19:29:56,928 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - starting ErrorHandler@f815e6b 2015-11-22 19:29:56,928 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - started ErrorHandler@f815e6b 2015-11-22 19:29:56,928 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - Holding class org.mortbay.jetty.servlet.DefaultServlet 2015-11-22 19:29:56,929 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - started org.mortbay.jetty.servlet.DefaultServlet-978534330 2015-11-22 19:29:56,929 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - started org.mortbay.jetty.servlet.Context@4515f214{/static,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes/webapps/static/} 2015-11-22 19:29:56,929 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - starting ContextHandlerCollection@5f10502a 2015-11-22 19:29:56,929 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - started ContextHandlerCollection@5f10502a 2015-11-22 19:29:56,929 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - starting Server@1d16906a 2015-11-22 19:29:56,929 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - started org.mortbay.jetty.nio.SelectChannelConnector$1@4ba8cb34 2015-11-22 19:29:56,929 INFO [Thread-4102] mortbay.log (Slf4jLog.java:info(67)) - Started HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:49959 2015-11-22 19:29:56,930 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - started HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:49959 2015-11-22 19:29:56,930 DEBUG [Thread-4102] mortbay.log (Slf4jLog.java:debug(40)) - started Server@1d16906a 2015-11-22 19:29:56,930 INFO [Thread-4102] applicationhistoryservice.ApplicationHistoryServer (ApplicationHistoryServer.java:startWebApp(342)) - Instantiating AHSWebApp at 49959 2015-11-22 19:29:56,930 DEBUG [Thread-4102] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer is started 2015-11-22 19:29:57,014 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:29:57,014 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 0 container statuses: [] 2015-11-22 19:29:57,014 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:57,014 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:35735 of type STATUS_UPDATE 2015-11-22 19:29:57,014 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:29:57,014 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:35735 clusterResources: 2015-11-22 19:29:57,014 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:35735 availableResource: 2015-11-22 19:29:57,014 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:29:57,015 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:29:58,014 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:29:58,014 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 0 container statuses: [] 2015-11-22 19:29:58,015 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:58,015 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:35735 of type STATUS_UPDATE 2015-11-22 19:29:58,015 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:29:58,015 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:35735 clusterResources: 2015-11-22 19:29:58,015 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:35735 availableResource: 2015-11-22 19:29:58,015 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:29:58,015 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:29:58,281 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(739)) - MiniYARN ApplicationHistoryServer address: ip-172-31-20-42.ap-northeast-1.compute.internal:56823 2015-11-22 19:29:58,281 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(741)) - MiniYARN ApplicationHistoryServer web address: ip-172-31-20-42.ap-northeast-1.compute.internal:49959 2015-11-22 19:29:58,282 DEBUG [main] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.server.MiniYARNCluster$ApplicationHistoryServerWrapper is started 2015-11-22 19:29:58,282 DEBUG [main] service.AbstractService (AbstractService.java:start(197)) - Service TestDistributedShell is started 2015-11-22 19:29:58,282 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(647)) - Handling deprecation for all properties in config... 2015-11-22 19:29:58,283 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.scheduler.monitor.policies 2015-11-22 19:29:58,283 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.security.group.mapping.ldap.posix.attr.uid.name 2015-11-22 19:29:58,283 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.application.classpath 2015-11-22 19:29:58,283 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.admin.acl 2015-11-22 19:29:58,283 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for fs.ftp.host.port 2015-11-22 19:29:58,283 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.timeline-service.leveldb-timeline-store.ttl-interval-ms 2015-11-22 19:29:58,283 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.leveldb-state-store.path 2015-11-22 19:29:58,283 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.ipc.rpc.class 2015-11-22 19:29:58,283 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for ipc.client.connection.maxidletime 2015-11-22 19:29:58,283 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.process-kill-wait.ms 2015-11-22 19:29:58,283 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.minicluster.use-rpc 2015-11-22 19:29:58,283 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for io.map.index.interval 2015-11-22 19:29:58,283 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for io.seqfile.sorter.recordlimit 2015-11-22 19:29:58,284 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for fs.s3n.multipart.uploads.enabled 2015-11-22 19:29:58,284 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.util.hash.type 2015-11-22 19:29:58,284 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.container-manager.thread-count 2015-11-22 19:29:58,284 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for fs.AbstractFileSystem.file.impl 2015-11-22 19:29:58,284 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for net.topology.script.number.args 2015-11-22 19:29:58,284 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.container-tokens.master-key-rolling-interval-secs 2015-11-22 19:29:58,284 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for s3native.bytes-per-checksum 2015-11-22 19:29:58,284 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.windows-container.memory-limit.enabled 2015-11-22 19:29:58,284 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.security.group.mapping 2015-11-22 19:29:58,284 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.localizer.cache.cleanup.interval-ms 2015-11-22 19:29:58,284 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for fs.s3a.threads.core 2015-11-22 19:29:58,284 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.node-labels.resync-interval-ms 2015-11-22 19:29:58,284 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.timeline-service.leveldb-timeline-store.start-time-read-cache-size 2015-11-22 19:29:58,284 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.log.dir 2015-11-22 19:29:58,284 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.registry.zk.retry.interval.ms 2015-11-22 19:29:58,285 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.keytab 2015-11-22 19:29:58,285 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.admin.address 2015-11-22 19:29:58,285 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.am.blacklisting.enabled 2015-11-22 19:29:58,285 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.fs.state-store.retry-policy-spec 2015-11-22 19:29:58,285 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.timeline-service.ttl-enable 2015-11-22 19:29:58,285 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.delete.debug-delay-sec 2015-11-22 19:29:58,285 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for fs.trash.interval 2015-11-22 19:29:58,285 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.amrmproxy.enable 2015-11-22 19:29:58,285 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.zk-num-retries 2015-11-22 19:29:58,285 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for s3.blocksize 2015-11-22 19:29:58,285 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.resource-monitor.interval-ms 2015-11-22 19:29:58,285 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for fs.s3.buffer.dir 2015-11-22 19:29:58,285 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.security.instrumentation.requires.admin 2015-11-22 19:29:58,285 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.registry.zk.retry.ceiling.ms 2015-11-22 19:29:58,285 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.env-whitelist 2015-11-22 19:29:58,286 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.container.liveness-monitor.interval-ms 2015-11-22 19:29:58,286 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.linux-container-executor.cgroups.hierarchy 2015-11-22 19:29:58,286 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for ha.failover-controller.graceful-fence.connection.retries 2015-11-22 19:29:58,286 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.recovery.enabled 2015-11-22 19:29:58,286 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.disk-health-checker.enable 2015-11-22 19:29:58,286 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.disk-health-checker.interval-ms 2015-11-22 19:29:58,286 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.security.java.secure.random.algorithm 2015-11-22 19:29:58,286 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.node-labels.provider.fetch-interval-ms 2015-11-22 19:29:58,286 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for fs.s3a.max.total.tasks 2015-11-22 19:29:58,286 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.fail-fast 2015-11-22 19:29:58,286 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.resource-tracker.client.thread-count 2015-11-22 19:29:58,286 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.webapp.cross-origin.enabled 2015-11-22 19:29:58,286 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.delete.thread-count 2015-11-22 19:29:58,286 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.admin-env 2015-11-22 19:29:58,286 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.timeline-service.hostname 2015-11-22 19:29:58,287 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for io.skip.checksum.errors 2015-11-22 19:29:58,287 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.proxy-user-privileges.enabled 2015-11-22 19:29:58,287 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.acl.enable 2015-11-22 19:29:58,287 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for fs.s3a.fast.upload 2015-11-22 19:29:58,287 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for file.blocksize 2015-11-22 19:29:58,287 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for ftp.replication 2015-11-22 19:29:58,287 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.sharedcache.cleaner.initial-delay-mins 2015-11-22 19:29:58,287 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for s3native.client-write-packet-size 2015-11-22 19:29:58,287 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.rpc.socket.factory.class.default 2015-11-22 19:29:58,287 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for file.bytes-per-checksum 2015-11-22 19:29:58,287 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.delegation-token-renewer.thread-count 2015-11-22 19:29:58,287 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for io.seqfile.lazydecompress 2015-11-22 19:29:58,287 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for fs.AbstractFileSystem.swebhdfs.impl 2015-11-22 19:29:58,287 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.common.configuration.version 2015-11-22 19:29:58,287 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.client.thread-count 2015-11-22 19:29:58,288 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.sharedcache.admin.address 2015-11-22 19:29:58,288 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.linux-container-executor.nonsecure-mode.user-pattern 2015-11-22 19:29:58,288 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.zk-timeout-ms 2015-11-22 19:29:58,288 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.max-completed-applications 2015-11-22 19:29:58,288 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.sharedcache.cleaner.period-mins 2015-11-22 19:29:58,288 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.linux-container-executor.cgroups.mount 2015-11-22 19:29:58,288 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.sharedcache.checksum.algo.impl 2015-11-22 19:29:58,288 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.node-labels.fs-store.retry-policy-spec 2015-11-22 19:29:58,288 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.log-aggregation-enable 2015-11-22 19:29:58,288 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.nodemanager.minimum.version 2015-11-22 19:29:58,288 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.security.kms.client.encrypted.key.cache.size 2015-11-22 19:29:58,288 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.security.group.mapping.ldap.search.attr.member 2015-11-22 19:29:58,288 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.log.retain-seconds 2015-11-22 19:29:58,288 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.local-cache.max-files-per-directory 2015-11-22 19:29:58,289 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.ssl.client.conf 2015-11-22 19:29:58,289 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.sharedcache.root-dir 2015-11-22 19:29:58,289 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for ha.failover-controller.new-active.rpc-timeout.ms 2015-11-22 19:29:58,289 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.ssl.hostname.verifier 2015-11-22 19:29:58,289 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.timeline-service.webapp.address 2015-11-22 19:29:58,289 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.connect.retry-interval.ms 2015-11-22 19:29:58,289 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for s3native.blocksize 2015-11-22 19:29:58,289 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.scheduler.minimum-allocation-mb 2015-11-22 19:29:58,289 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.sharedcache.cleaner.resource-sleep-ms 2015-11-22 19:29:58,289 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for net.topology.impl 2015-11-22 19:29:58,289 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for io.seqfile.compress.blocksize 2015-11-22 19:29:58,289 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.registry.zk.retry.times 2015-11-22 19:29:58,289 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for fs.AbstractFileSystem.ftp.impl 2015-11-22 19:29:58,289 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.container-executor.os.sched.priority.adjustment 2015-11-22 19:29:58,289 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.scheduler.maximum-allocation-mb 2015-11-22 19:29:58,289 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.vmem-check-enabled 2015-11-22 19:29:58,290 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.rpc.protection 2015-11-22 19:29:58,290 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for fs.permissions.umask-mode 2015-11-22 19:29:58,290 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for fs.s3.sleepTimeSeconds 2015-11-22 19:29:58,290 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.nm-tokens.master-key-rolling-interval-secs 2015-11-22 19:29:58,290 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.log.deletion-threads-count 2015-11-22 19:29:58,290 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for ha.health-monitor.rpc-timeout.ms 2015-11-22 19:29:58,290 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.http.staticuser.user 2015-11-22 19:29:58,290 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for fs.s3a.connection.maximum 2015-11-22 19:29:58,290 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.linux-container-executor.nonsecure-mode.limit-users 2015-11-22 19:29:58,290 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for fs.s3a.paging.maximum 2015-11-22 19:29:58,290 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for fs.AbstractFileSystem.viewfs.impl 2015-11-22 19:29:58,290 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.delegation.token.renew-interval 2015-11-22 19:29:58,290 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for fs.ftp.host 2015-11-22 19:29:58,290 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.shell.missing.defaultFs.warning 2015-11-22 19:29:58,290 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.linux-container-executor.nonsecure-mode.local-user 2015-11-22 19:29:58,291 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.http.authentication.kerberos.keytab 2015-11-22 19:29:58,291 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for fs.s3a.impl 2015-11-22 19:29:58,291 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.delegation.key.update-interval 2015-11-22 19:29:58,291 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.am-rm-tokens.master-key-rolling-interval-secs 2015-11-22 19:29:58,291 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.webapp.https.address 2015-11-22 19:29:58,291 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.container-localizer.java.opts 2015-11-22 19:29:58,291 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for io.compression.codec.bzip2.library 2015-11-22 19:29:58,291 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for net.topology.node.switch.mapping.impl 2015-11-22 19:29:58,291 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.client.application-client-protocol.poll-interval-ms 2015-11-22 19:29:58,291 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.localizer.address 2015-11-22 19:29:58,291 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.resource.count-logical-processors-as-cores 2015-11-22 19:29:58,291 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.resource.system-reserved-memory-mb 2015-11-22 19:29:58,291 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.timeline-service.client.best-effort 2015-11-22 19:29:58,291 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for ha.zookeeper.parent-znode 2015-11-22 19:29:58,291 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.sharedcache.admin.thread-count 2015-11-22 19:29:58,292 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.resource.cpu-vcores 2015-11-22 19:29:58,292 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for fs.s3a.attempts.maximum 2015-11-22 19:29:58,292 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.webapp.delegation-token-auth-filter.enabled 2015-11-22 19:29:58,292 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for s3native.stream-buffer-size 2015-11-22 19:29:58,292 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.log-aggregation.retain-check-interval-seconds 2015-11-22 19:29:58,292 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for io.seqfile.local.dir 2015-11-22 19:29:58,292 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for fs.s3n.multipart.copy.block.size 2015-11-22 19:29:58,292 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.node-ip-cache.expiry-interval-secs 2015-11-22 19:29:58,292 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.zk-acl 2015-11-22 19:29:58,292 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.ssl.keystores.factory.class 2015-11-22 19:29:58,292 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.security.random.device.file.path 2015-11-22 19:29:58,292 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.client.nodemanager-connect.max-wait-ms 2015-11-22 19:29:58,292 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for fs.s3.maxRetries 2015-11-22 19:29:58,292 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.zk-state-store.parent-path 2015-11-22 19:29:58,292 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for fs.client.resolve.remote.symlinks 2015-11-22 19:29:58,293 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.linux-container-executor.cgroups.strict-resource-usage 2015-11-22 19:29:58,293 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.ssl.enabled.protocols 2015-11-22 19:29:58,293 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for mapreduce.jobhistory.address 2015-11-22 19:29:58,293 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.client.failover-retries 2015-11-22 19:29:58,293 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.ssl.enabled 2015-11-22 19:29:58,293 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for ipc.client.kill.max 2015-11-22 19:29:58,293 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.configuration.file-system-based-store 2015-11-22 19:29:58,293 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.localizer.cache.target-size-mb 2015-11-22 19:29:58,293 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.admin.client.thread-count 2015-11-22 19:29:58,293 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.log-aggregation.policy.class 2015-11-22 19:29:58,293 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for ipc.client.connect.max.retries 2015-11-22 19:29:58,293 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.timeline-service.store-class 2015-11-22 19:29:58,293 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.tmp.dir 2015-11-22 19:29:58,293 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.reservation-system.planfollower.time-step 2015-11-22 19:29:58,294 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.ha.automatic-failover.embedded 2015-11-22 19:29:58,294 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.node-labels.configuration-type 2015-11-22 19:29:58,294 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.timeline-service.ttl-ms 2015-11-22 19:29:58,294 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.nodemanagers.heartbeat-interval-ms 2015-11-22 19:29:58,294 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.timeline-service.recovery.enabled 2015-11-22 19:29:58,294 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.linux-container-executor.cgroups.delete-timeout-ms 2015-11-22 19:29:58,294 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.recovery.dir 2015-11-22 19:29:58,294 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.keytab 2015-11-22 19:29:58,294 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.linux-container-executor.cgroups.delete-delay-ms 2015-11-22 19:29:58,294 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.cluster.max-application-priority 2015-11-22 19:29:58,294 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.log-aggregation-status.time-out.ms 2015-11-22 19:29:58,294 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.client.max-cached-nodemanagers-proxies 2015-11-22 19:29:58,294 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for fs.trash.checkpoint.interval 2015-11-22 19:29:58,295 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.sharedcache.app-checker.class 2015-11-22 19:29:58,295 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nm.liveness-monitor.expiry-interval-ms 2015-11-22 19:29:58,295 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for ha.health-monitor.check-interval.ms 2015-11-22 19:29:58,295 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for rpc.engine.org.apache.hadoop.yarn.server.api.ResourceTrackerPB 2015-11-22 19:29:58,295 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.docker-container-executor.exec-name 2015-11-22 19:29:58,295 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for ipc.client.connect.timeout 2015-11-22 19:29:58,295 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.fs.state-store.retry-interval-ms 2015-11-22 19:29:58,295 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.local-dirs 2015-11-22 19:29:58,295 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.recovery.enabled 2015-11-22 19:29:58,295 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for io.native.lib.available 2015-11-22 19:29:58,295 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.sharedcache.store.in-memory.staleness-period-mins 2015-11-22 19:29:58,295 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for s3.replication 2015-11-22 19:29:58,296 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.am.max-attempts 2015-11-22 19:29:58,296 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for fs.AbstractFileSystem.har.impl 2015-11-22 19:29:58,296 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.resource.pcores-vcores-multiplier 2015-11-22 19:29:58,296 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.am.blacklisting.disable-failure-threshold 2015-11-22 19:29:58,296 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.webapp.address 2015-11-22 19:29:58,296 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for nfs3.mountd.port 2015-11-22 19:29:58,296 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.security.group.mapping.ldap.ssl 2015-11-22 19:29:58,296 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for fs.defaultFS 2015-11-22 19:29:58,296 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for fs.s3a.multipart.size 2015-11-22 19:29:58,296 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.security.group.mapping.ldap.search.attr.group.name 2015-11-22 19:29:58,296 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.disk-health-checker.max-disk-utilization-per-disk-percentage 2015-11-22 19:29:58,296 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.container-monitor.procfs-tree.smaps-based-rss.enabled 2015-11-22 19:29:58,297 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.security.crypto.codec.classes.aes.ctr.nopadding 2015-11-22 19:29:58,297 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.security.groups.negative-cache.secs 2015-11-22 19:29:58,297 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.ssl.server.conf 2015-11-22 19:29:58,297 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.client.nodemanager-client-async.thread-pool-max-size 2015-11-22 19:29:58,297 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for s3native.replication 2015-11-22 19:29:58,297 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.minicluster.yarn.nodemanager.resource.memory-mb 2015-11-22 19:29:58,297 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.health-checker.interval-ms 2015-11-22 19:29:58,297 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for s3.stream-buffer-size 2015-11-22 19:29:58,297 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for ftp.client-write-packet-size 2015-11-22 19:29:58,297 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.timeline-service.keytab 2015-11-22 19:29:58,297 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.sharedcache.store.in-memory.initial-delay-mins 2015-11-22 19:29:58,298 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for ha.zookeeper.session-timeout.ms 2015-11-22 19:29:58,298 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for fs.s3a.connection.ssl.enabled 2015-11-22 19:29:58,298 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.sharedcache.webapp.address 2015-11-22 19:29:58,298 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.http.authentication.signature.secret.file 2015-11-22 19:29:58,298 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.log-aggregation.compression-type 2015-11-22 19:29:58,298 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.rm.container-allocation.expiry-interval-ms 2015-11-22 19:29:58,298 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for ipc.server.max.connections 2015-11-22 19:29:58,298 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.work-preserving-recovery.scheduling-wait-ms 2015-11-22 19:29:58,298 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for rpc.engine.org.apache.hadoop.yarn.api.ApplicationClientProtocolPB 2015-11-22 19:29:58,298 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.log-dirs 2015-11-22 19:29:58,298 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.security.groups.cache.secs 2015-11-22 19:29:58,299 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.recovery.supervised 2015-11-22 19:29:58,299 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for s3.client-write-packet-size 2015-11-22 19:29:58,299 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.amrmproxy.interceptor-class.pipeline 2015-11-22 19:29:58,299 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.security.group.mapping.ldap.directory.search.timeout 2015-11-22 19:29:58,299 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.work.around.non.threadsafe.getpwuid 2015-11-22 19:29:58,299 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for ipc.client.low-latency 2015-11-22 19:29:58,299 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.address 2015-11-22 19:29:58,299 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.scheduler.maximum-allocation-vcores 2015-11-22 19:29:58,299 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.sleep-delay-before-sigkill.ms 2015-11-22 19:29:58,299 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.security.kms.client.encrypted.key.cache.expiry 2015-11-22 19:29:58,299 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.timeline-service.address 2015-11-22 19:29:58,299 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.configuration.provider-class 2015-11-22 19:29:58,300 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.sharedcache.enabled 2015-11-22 19:29:58,300 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.registry.zk.session.timeout.ms 2015-11-22 19:29:58,300 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for tfile.io.chunk.size 2015-11-22 19:29:58,300 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for fs.automatic.close 2015-11-22 19:29:58,300 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for ha.health-monitor.sleep-after-disconnect.ms 2015-11-22 19:29:58,300 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.security.group.mapping.ldap.posix.attr.gid.name 2015-11-22 19:29:58,300 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.http.authentication.token.validity 2015-11-22 19:29:58,300 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for fs.s3.block.size 2015-11-22 19:29:58,300 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for ha.failover-controller.graceful-fence.rpc-timeout.ms 2015-11-22 19:29:58,300 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.http.policy 2015-11-22 19:29:58,300 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.reservation-system.enable 2015-11-22 19:29:58,300 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.security.groups.cache.warn.after.ms 2015-11-22 19:29:58,300 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for io.serializations 2015-11-22 19:29:58,300 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.security.crypto.buffer.size 2015-11-22 19:29:58,301 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.http.cross-origin.allowed-methods 2015-11-22 19:29:58,301 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.registry.secure 2015-11-22 19:29:58,301 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.kerberos.kinit.command 2015-11-22 19:29:58,301 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.timeline-service.http-authentication.type 2015-11-22 19:29:58,301 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for test.fs.s3.name 2015-11-22 19:29:58,301 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.dispatcher.drain-events.timeout 2015-11-22 19:29:58,301 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.log-aggregation.retain-seconds 2015-11-22 19:29:58,301 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.fail-fast 2015-11-22 19:29:58,301 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.scheduler.minimum-allocation-vcores 2015-11-22 19:29:58,301 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for ipc.client.connect.max.retries.on.timeouts 2015-11-22 19:29:58,301 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.runtime.linux.docker.privileged-containers.allowed 2015-11-22 19:29:58,301 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.node-labels.provider.fetch-interval-ms 2015-11-22 19:29:58,301 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.container-metrics.enable 2015-11-22 19:29:58,301 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.timeline-service.client.retry-interval-ms 2015-11-22 19:29:58,302 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.timeline-service.client.max-retries 2015-11-22 19:29:58,302 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for fs.s3n.block.size 2015-11-22 19:29:58,302 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for nfs.exports.allowed.hosts 2015-11-22 19:29:58,302 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for ha.health-monitor.connect-retry-interval.ms 2015-11-22 19:29:58,302 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for io.file.buffer.size 2015-11-22 19:29:58,302 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.container-metrics.unregister-delay-ms 2015-11-22 19:29:58,302 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for io.mapfile.bloom.size 2015-11-22 19:29:58,302 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for rpc.engine.org.apache.hadoop.ipc.ProtocolMetaInfoPB 2015-11-22 19:29:58,302 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for ipc.client.connect.retry.interval 2015-11-22 19:29:58,302 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.security.kms.client.authentication.retry-count 2015-11-22 19:29:58,302 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.container-metrics.period-ms 2015-11-22 19:29:58,302 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.fs.state-store.uri 2015-11-22 19:29:58,302 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for fs.swift.impl 2015-11-22 19:29:58,302 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.registry.zk.connection.timeout.ms 2015-11-22 19:29:58,302 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.resourcemanager.minimum.version 2015-11-22 19:29:58,303 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for ftp.blocksize 2015-11-22 19:29:58,303 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.address 2015-11-22 19:29:58,303 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for file.stream-buffer-size 2015-11-22 19:29:58,303 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.scheduler.monitor.enable 2015-11-22 19:29:58,303 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.nodemanager-connect-retries 2015-11-22 19:29:58,303 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.sharedcache.nm.uploader.thread-count 2015-11-22 19:29:58,303 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for ipc.client.idlethreshold 2015-11-22 19:29:58,303 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.sharedcache.client-server.address 2015-11-22 19:29:58,303 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for ftp.stream-buffer-size 2015-11-22 19:29:58,303 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.security.authorization 2015-11-22 19:29:58,303 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.http.authentication.simple.anonymous.allowed 2015-11-22 19:29:58,303 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.client.nodemanager-connect.retry-interval-ms 2015-11-22 19:29:58,303 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for fs.har.impl.disable.cache 2015-11-22 19:29:58,303 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.am.liveness-monitor.expiry-interval-ms 2015-11-22 19:29:58,304 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.linux-container-executor.resources-handler.class 2015-11-22 19:29:58,304 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.timeline-service.leveldb-timeline-store.read-cache-size 2015-11-22 19:29:58,304 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.security.authentication 2015-11-22 19:29:58,304 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for test.fs.s3n.name 2015-11-22 19:29:58,304 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.timeline-service.leveldb-timeline-store.path 2015-11-22 19:29:58,304 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.delegation.token.max-lifetime 2015-11-22 19:29:58,304 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for file.replication 2015-11-22 19:29:58,304 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.ha.automatic-failover.enabled 2015-11-22 19:29:58,304 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.amrmproxy.client.thread-count 2015-11-22 19:29:58,304 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.work-preserving-recovery.enabled 2015-11-22 19:29:58,304 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.store.class 2015-11-22 19:29:58,304 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for io.mapfile.bloom.error.rate 2015-11-22 19:29:58,304 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.timeline-service.leveldb-state-store.path 2015-11-22 19:29:58,304 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for fs.s3a.multipart.purge 2015-11-22 19:29:58,304 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.webapp.ui-actions.enabled 2015-11-22 19:29:58,304 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for fs.s3a.connection.establish.timeout 2015-11-22 19:29:58,305 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for rpc.engine.org.apache.hadoop.yarn.api.ApplicationHistoryProtocolPB 2015-11-22 19:29:58,305 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.container-executor.class 2015-11-22 19:29:58,305 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for fs.s3a.multipart.purge.age 2015-11-22 19:29:58,305 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.scheduler.client.thread-count 2015-11-22 19:29:58,305 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.security.kms.client.encrypted.key.cache.low-watermark 2015-11-22 19:29:58,305 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.user.group.static.mapping.overrides 2015-11-22 19:29:58,305 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for fs.s3a.fast.buffer.size 2015-11-22 19:29:58,305 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.amrmproxy.address 2015-11-22 19:29:58,305 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for tfile.fs.input.buffer.size 2015-11-22 19:29:58,305 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.http.authentication.type 2015-11-22 19:29:58,305 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for fs.ftp.user.localhost 2015-11-22 19:29:58,305 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for fs.AbstractFileSystem.webhdfs.impl 2015-11-22 19:29:58,305 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.zk-retry-interval-ms 2015-11-22 19:29:58,305 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for ftp.bytes-per-checksum 2015-11-22 19:29:58,305 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.runtime.linux.docker.capabilities 2015-11-22 19:29:58,306 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.minicluster.fixed.ports 2015-11-22 19:29:58,306 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for ipc.client.fallback-to-simple-auth-allowed 2015-11-22 19:29:58,306 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.pmem-check-enabled 2015-11-22 19:29:58,306 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.remote-app-log-dir 2015-11-22 19:29:58,306 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.timeline-service.webapp.https.address 2015-11-22 19:29:58,306 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.resource.percentage-physical-cpu-limit 2015-11-22 19:29:58,306 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.amlauncher.thread-count 2015-11-22 19:29:58,306 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.sharedcache.nm.uploader.replication.factor 2015-11-22 19:29:58,306 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.registry.zk.root 2015-11-22 19:29:58,306 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.jetty.logs.serve.aliases 2015-11-22 19:29:58,306 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.client.failover-proxy-provider 2015-11-22 19:29:58,306 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.http.cross-origin.max-age 2015-11-22 19:29:58,306 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.remote-app-log-dir-suffix 2015-11-22 19:29:58,306 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.webapp.address 2015-11-22 19:29:58,306 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.sharedcache.store.in-memory.check-period-mins 2015-11-22 19:29:58,306 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for fs.df.interval 2015-11-22 19:29:58,307 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.timeline-service.enabled 2015-11-22 19:29:58,307 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.http.cross-origin.allowed-headers 2015-11-22 19:29:58,307 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.hostname 2015-11-22 19:29:58,307 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.localizer.client.thread-count 2015-11-22 19:29:58,307 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.sharedcache.uploader.server.thread-count 2015-11-22 19:29:58,307 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for fs.AbstractFileSystem.hdfs.impl 2015-11-22 19:29:58,307 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.timeline-service.http-authentication.simple.anonymous.allowed 2015-11-22 19:29:58,307 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.http.filter.initializers 2015-11-22 19:29:58,307 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.sharedcache.client-server.thread-count 2015-11-22 19:29:58,307 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.resource-tracker.address 2015-11-22 19:29:58,307 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for rpc.metrics.quantile.enable 2015-11-22 19:29:58,307 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for io.bytes.per.checksum 2015-11-22 19:29:58,307 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.resource.memory-mb 2015-11-22 19:29:58,307 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.disk-health-checker.min-healthy-disks 2015-11-22 19:29:58,307 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.node-labels.enabled 2015-11-22 19:29:58,308 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.timeline-service.handler-thread-count 2015-11-22 19:29:58,308 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for fs.s3a.threads.max 2015-11-22 19:29:58,308 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for ipc.server.listen.queue.size 2015-11-22 19:29:58,308 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.connect.max-wait.ms 2015-11-22 19:29:58,308 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.resource.detect-hardware-capabilities 2015-11-22 19:29:58,308 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.history-writer.multi-threaded-dispatcher.pool-size 2015-11-22 19:29:58,308 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.scheduler.class 2015-11-22 19:29:58,308 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.system-metrics-publisher.enabled 2015-11-22 19:29:58,308 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.is.minicluster 2015-11-22 19:29:58,308 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.sharedcache.nested-level 2015-11-22 19:29:58,308 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for fs.s3a.connection.timeout 2015-11-22 19:29:58,308 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for fs.s3a.threads.keepalivetime 2015-11-22 19:29:58,308 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for file.client-write-packet-size 2015-11-22 19:29:58,308 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for ha.failover-controller.cli-check.rpc-timeout.ms 2015-11-22 19:29:58,309 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for ha.zookeeper.acl 2015-11-22 19:29:58,309 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.timeline-service.generic-application-history.store-class 2015-11-22 19:29:58,309 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.delayed.delegation-token.removal-interval-ms 2015-11-22 19:29:58,309 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.localizer.fetch.thread-count 2015-11-22 19:29:58,309 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.webapp.cross-origin.enabled 2015-11-22 19:29:58,309 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.scheduler.address 2015-11-22 19:29:58,309 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.minicluster.control-resource-monitoring 2015-11-22 19:29:58,309 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.timeline-service.leveldb-timeline-store.start-time-write-cache-size 2015-11-22 19:29:58,309 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.health-checker.script.timeout-ms 2015-11-22 19:29:58,309 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.fs.state-store.num-retries 2015-11-22 19:29:58,309 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.ssl.require.client.cert 2015-11-22 19:29:58,309 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.security.uid.cache.secs 2015-11-22 19:29:58,309 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.ha.automatic-failover.zk-base-path 2015-11-22 19:29:58,309 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.registry.rm.enabled 2015-11-22 19:29:58,309 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.intermediate-data-encryption.enable 2015-11-22 19:29:58,310 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.system-metrics-publisher.dispatcher.pool-size 2015-11-22 19:29:58,310 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.ha.enabled 2015-11-22 19:29:58,310 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for rpc.engine.org.apache.hadoop.yarn.server.api.ResourceManagerAdministrationProtocolPB 2015-11-22 19:29:58,310 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for ipc.client.tcpnodelay 2015-11-22 19:29:58,310 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.dispatcher.exit-on-error 2015-11-22 19:29:58,310 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.metrics.runtime.buckets 2015-11-22 19:29:58,310 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.client.application-client-protocol.poll-timeout-ms 2015-11-22 19:29:58,310 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for fs.s3a.multipart.threshold 2015-11-22 19:29:58,310 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.http.cross-origin.enabled 2015-11-22 19:29:58,310 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for io.map.index.skip 2015-11-22 19:29:58,310 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for mapreduce.job.hdfs-servers 2015-11-22 19:29:58,310 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.scheduler.include-port-in-node-name 2015-11-22 19:29:58,310 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for fs.ftp.password.localhost 2015-11-22 19:29:58,310 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.state-store.max-completed-applications 2015-11-22 19:29:58,311 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.security.kms.client.encrypted.key.cache.num.refill.threads 2015-11-22 19:29:58,311 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for fs.s3n.multipart.uploads.block.size 2015-11-22 19:29:58,311 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.aux-services.mapreduce_shuffle.class 2015-11-22 19:29:58,311 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for tfile.fs.output.buffer.size 2015-11-22 19:29:58,311 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for fs.du.interval 2015-11-22 19:29:58,311 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for fs.s3a.buffer.dir 2015-11-22 19:29:58,311 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.sharedcache.uploader.server.address 2015-11-22 19:29:58,311 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for dfs.ha.fencing.ssh.connect-timeout 2015-11-22 19:29:58,311 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.log-aggregation.roll-monitoring-interval-seconds 2015-11-22 19:29:58,311 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.registry.zk.quorum 2015-11-22 19:29:58,311 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.http.cross-origin.allowed-origins 2015-11-22 19:29:58,311 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.timeline-service.state-store-class 2015-11-22 19:29:58,312 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.registry.system.acls 2015-11-22 19:29:58,312 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.security.crypto.cipher.suite 2015-11-22 19:29:58,312 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.timeline-service.generic-application-history.max-applications 2015-11-22 19:29:58,312 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.registry.jaas.context 2015-11-22 19:29:58,312 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.hostname 2015-11-22 19:29:58,312 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.security.group.mapping.ldap.search.filter.group 2015-11-22 19:29:58,312 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.http.authentication.kerberos.principal 2015-11-22 19:29:58,312 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.shell.safely.delete.limit.num.files 2015-11-22 19:29:58,312 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for hadoop.security.group.mapping.ldap.search.filter.user 2015-11-22 19:29:58,312 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.disk-health-checker.min-free-space-per-disk-mb 2015-11-22 19:29:58,312 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for nfs3.server.port 2015-11-22 19:29:58,312 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.client.failover-retries-on-socket-timeouts 2015-11-22 19:29:58,312 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for ipc.server.log.slow.rpc 2015-11-22 19:29:58,313 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.sharedcache.store.class 2015-11-22 19:29:58,313 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for s3.bytes-per-checksum 2015-11-22 19:29:58,313 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.max-log-aggregation-diagnostics-in-memory 2015-11-22 19:29:58,313 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.windows-container.cpu-limit.enabled 2015-11-22 19:29:58,313 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.resourcemanager.webapp.https.address 2015-11-22 19:29:58,313 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.vmem-pmem-ratio 2015-11-22 19:29:58,313 DEBUG [main] conf.Configuration (Configuration.java:handleDeprecation(652)) - Handling deprecation for yarn.nodemanager.node-labels.provider.fetch-timeout-ms 2015-11-22 19:29:58,325 DEBUG [main] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.fs.FileContext.getAbstractFileSystem(FileContext.java:339) 2015-11-22 19:29:59,015 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:29:59,015 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 0 container statuses: [] 2015-11-22 19:29:59,015 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:29:59,015 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:35735 of type STATUS_UPDATE 2015-11-22 19:29:59,015 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:29:59,015 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:35735 clusterResources: 2015-11-22 19:29:59,016 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:35735 availableResource: 2015-11-22 19:29:59,016 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:29:59,016 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:30:00,015 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:30:00,016 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 0 container statuses: [] 2015-11-22 19:30:00,016 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:30:00,016 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:35735 of type STATUS_UPDATE 2015-11-22 19:30:00,016 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:30:00,016 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:35735 clusterResources: 2015-11-22 19:30:00,016 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:35735 availableResource: 2015-11-22 19:30:00,016 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:30:00,016 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:30:00,340 INFO [Thread-4126] distributedshell.TestDistributedShell (TestDistributedShell.java:testContainerLaunchFailureHandling(803)) - Initializing DS Client 2015-11-22 19:30:00,342 DEBUG [Thread-4126] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.client.api.impl.YarnClientImpl entered state INITED 2015-11-22 19:30:00,342 DEBUG [Thread-4126] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl entered state INITED 2015-11-22 19:30:00,343 DEBUG [Thread-4126] ssl.FileBasedKeyStoresFactory (FileBasedKeyStoresFactory.java:init(226)) - The property 'ssl.client.truststore.location' has not been set, no TrustStore will be loaded 2015-11-22 19:30:00,405 INFO [Thread-4126] impl.TimelineClientImpl (TimelineClientImpl.java:serviceInit(296)) - Timeline service address: http://ip-172-31-20-42.ap-northeast-1.compute.internal:49959/ws/v1/timeline/ 2015-11-22 19:30:00,406 INFO [Thread-4126] distributedshell.TestDistributedShell (TestDistributedShell.java:testContainerLaunchFailureHandling(808)) - Running DS Client 2015-11-22 19:30:00,406 INFO [Thread-4126] distributedshell.Client (Client.java:run(443)) - Running Client 2015-11-22 19:30:00,423 INFO [Thread-4126] client.RMProxy (RMProxy.java:createRMProxy(98)) - Connecting to ResourceManager at ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:38037 2015-11-22 19:30:00,423 DEBUG [Thread-4126] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.yarn.client.RMProxy.getProxy(RMProxy.java:136) 2015-11-22 19:30:00,423 DEBUG [Thread-4126] ipc.YarnRPC (YarnRPC.java:create(59)) - Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC 2015-11-22 19:30:00,423 DEBUG [Thread-4126] ipc.HadoopYarnProtoRPC (HadoopYarnProtoRPC.java:getProxy(47)) - Creating a HadoopYarnProtoRpc proxy for protocol interface org.apache.hadoop.yarn.api.ApplicationClientProtocol 2015-11-22 19:30:00,423 DEBUG [Thread-4126] ipc.Client (ClientCache.java:getClient(63)) - getting client out of cache: org.apache.hadoop.ipc.Client@67cf996b 2015-11-22 19:30:00,423 DEBUG [Thread-4126] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl is started 2015-11-22 19:30:00,423 DEBUG [Thread-4126] service.AbstractService (AbstractService.java:start(197)) - Service org.apache.hadoop.yarn.client.api.impl.YarnClientImpl is started 2015-11-22 19:30:00,424 DEBUG [Thread-4126] ipc.Client (Client.java:(439)) - The ping interval is 60000 ms. 2015-11-22 19:30:00,424 DEBUG [Thread-4126] ipc.Client (Client.java:setupIOstreams(723)) - Connecting to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:38037 2015-11-22 19:30:00,424 DEBUG [IPC Server listener on 38037] ipc.Server (Server.java:register(2883)) - Server connection from 172.31.20.42:35310; # active connections: 1; # queued calls: 0 2015-11-22 19:30:00,425 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:38037 from ubuntu] ipc.Client (Client.java:run(1000)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:38037 from ubuntu: starting, having connections 1 2015-11-22 19:30:00,425 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:38037 from ubuntu sending #936 2015-11-22 19:30:00,426 DEBUG [Socket Reader #1 for port 38037] ipc.Server (Server.java:processOneRpc(1897)) - got #-3 2015-11-22 19:30:00,426 DEBUG [Socket Reader #1 for port 38037] ipc.Server (Server.java:authorizeConnection(2097)) - Successfully authorized userInfo { effectiveUser: "ubuntu" } protocol: "org.apache.hadoop.yarn.api.ApplicationClientProtocolPB" 2015-11-22 19:30:00,426 DEBUG [Socket Reader #1 for port 38037] ipc.Server (Server.java:processOneRpc(1897)) - got #936 2015-11-22 19:30:00,426 DEBUG [IPC Server handler 0 on 38037] ipc.Server (Server.java:run(2185)) - IPC Server handler 0 on 38037: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getClusterMetrics from 172.31.20.42:35310 Call#936 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:30:00,426 DEBUG [IPC Server handler 0 on 38037] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:30:00,427 DEBUG [IPC Server handler 0 on 38037] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getClusterMetrics queueTime= 1 procesingTime= 0 2015-11-22 19:30:00,428 DEBUG [IPC Server handler 0 on 38037] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 0 on 38037: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getClusterMetrics from 172.31.20.42:35310 Call#936 Retry#0 2015-11-22 19:30:00,428 DEBUG [IPC Server handler 0 on 38037] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 0 on 38037: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getClusterMetrics from 172.31.20.42:35310 Call#936 Retry#0 Wrote 47 bytes. 2015-11-22 19:30:00,428 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:38037 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:38037 from ubuntu got value #936 2015-11-22 19:30:00,428 DEBUG [Thread-4126] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getClusterMetrics took 4ms 2015-11-22 19:30:00,428 INFO [Thread-4126] distributedshell.Client (Client.java:run(447)) - Got Cluster metric info from ASM, numNodeManagers=1 2015-11-22 19:30:00,429 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:38037 from ubuntu sending #937 2015-11-22 19:30:00,429 DEBUG [Socket Reader #1 for port 38037] ipc.Server (Server.java:processOneRpc(1897)) - got #937 2015-11-22 19:30:00,429 DEBUG [IPC Server handler 2 on 38037] ipc.Server (Server.java:run(2185)) - IPC Server handler 2 on 38037: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getClusterNodes from 172.31.20.42:35310 Call#937 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:30:00,429 DEBUG [IPC Server handler 2 on 38037] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:30:00,430 DEBUG [IPC Server handler 2 on 38037] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getClusterNodes queueTime= 0 procesingTime= 1 2015-11-22 19:30:00,430 DEBUG [IPC Server handler 2 on 38037] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 2 on 38037: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getClusterNodes from 172.31.20.42:35310 Call#937 Retry#0 2015-11-22 19:30:00,430 DEBUG [IPC Server handler 2 on 38037] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 2 on 38037: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getClusterNodes from 172.31.20.42:35310 Call#937 Retry#0 Wrote 186 bytes. 2015-11-22 19:30:00,430 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:38037 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:38037 from ubuntu got value #937 2015-11-22 19:30:00,431 DEBUG [Thread-4126] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getClusterNodes took 2ms 2015-11-22 19:30:00,431 INFO [Thread-4126] distributedshell.Client (Client.java:run(452)) - Got Cluster node info from ASM 2015-11-22 19:30:00,431 INFO [Thread-4126] distributedshell.Client (Client.java:run(454)) - Got node report from ASM for, nodeId=ip-172-31-20-42.ap-northeast-1.compute.internal:35735, nodeAddress=ip-172-31-20-42.ap-northeast-1.compute.internal:43914, nodeRackName=/default-rack, nodeNumContainers=0 2015-11-22 19:30:00,431 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:38037 from ubuntu sending #938 2015-11-22 19:30:00,431 DEBUG [Socket Reader #1 for port 38037] ipc.Server (Server.java:processOneRpc(1897)) - got #938 2015-11-22 19:30:00,431 DEBUG [IPC Server handler 3 on 38037] ipc.Server (Server.java:run(2185)) - IPC Server handler 3 on 38037: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getQueueInfo from 172.31.20.42:35310 Call#938 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:30:00,431 DEBUG [IPC Server handler 3 on 38037] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:30:00,432 DEBUG [IPC Server handler 3 on 38037] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getQueueInfo queueTime= 1 procesingTime= 0 2015-11-22 19:30:00,432 DEBUG [IPC Server handler 3 on 38037] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 3 on 38037: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getQueueInfo from 172.31.20.42:35310 Call#938 Retry#0 2015-11-22 19:30:00,432 DEBUG [IPC Server handler 3 on 38037] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 3 on 38037: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getQueueInfo from 172.31.20.42:35310 Call#938 Retry#0 Wrote 106 bytes. 2015-11-22 19:30:00,432 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:38037 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:38037 from ubuntu got value #938 2015-11-22 19:30:00,432 DEBUG [Thread-4126] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getQueueInfo took 1ms 2015-11-22 19:30:00,432 INFO [Thread-4126] distributedshell.Client (Client.java:run(462)) - Queue info, queueName=default, queueCurrentCapacity=0.0, queueMaxCapacity=1.0, queueApplicationCount=0, queueChildQueueCount=0 2015-11-22 19:30:00,433 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:38037 from ubuntu sending #939 2015-11-22 19:30:00,433 DEBUG [Socket Reader #1 for port 38037] ipc.Server (Server.java:processOneRpc(1897)) - got #939 2015-11-22 19:30:00,433 DEBUG [IPC Server handler 1 on 38037] ipc.Server (Server.java:run(2185)) - IPC Server handler 1 on 38037: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getQueueUserAcls from 172.31.20.42:35310 Call#939 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:30:00,433 DEBUG [IPC Server handler 1 on 38037] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:30:00,433 DEBUG [IPC Server handler 1 on 38037] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getQueueUserAcls queueTime= 0 procesingTime= 0 2015-11-22 19:30:00,433 DEBUG [IPC Server handler 1 on 38037] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 1 on 38037: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getQueueUserAcls from 172.31.20.42:35310 Call#939 Retry#0 2015-11-22 19:30:00,434 DEBUG [IPC Server handler 1 on 38037] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 1 on 38037: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getQueueUserAcls from 172.31.20.42:35310 Call#939 Retry#0 Wrote 60 bytes. 2015-11-22 19:30:00,434 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:38037 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:38037 from ubuntu got value #939 2015-11-22 19:30:00,434 DEBUG [Thread-4126] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getQueueUserAcls took 1ms 2015-11-22 19:30:00,434 INFO [Thread-4126] distributedshell.Client (Client.java:run(472)) - User ACL Info for Queue, queueName=root, userAcl=SUBMIT_APPLICATIONS 2015-11-22 19:30:00,434 INFO [Thread-4126] distributedshell.Client (Client.java:run(472)) - User ACL Info for Queue, queueName=root, userAcl=ADMINISTER_QUEUE 2015-11-22 19:30:00,434 INFO [Thread-4126] distributedshell.Client (Client.java:run(472)) - User ACL Info for Queue, queueName=default, userAcl=SUBMIT_APPLICATIONS 2015-11-22 19:30:00,434 INFO [Thread-4126] distributedshell.Client (Client.java:run(472)) - User ACL Info for Queue, queueName=default, userAcl=ADMINISTER_QUEUE 2015-11-22 19:30:00,434 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:38037 from ubuntu sending #940 2015-11-22 19:30:00,434 DEBUG [Socket Reader #1 for port 38037] ipc.Server (Server.java:processOneRpc(1897)) - got #940 2015-11-22 19:30:00,435 DEBUG [IPC Server handler 4 on 38037] ipc.Server (Server.java:run(2185)) - IPC Server handler 4 on 38037: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getNewApplication from 172.31.20.42:35310 Call#940 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:30:00,435 DEBUG [IPC Server handler 4 on 38037] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:30:00,435 INFO [IPC Server handler 4 on 38037] resourcemanager.ClientRMService (ClientRMService.java:getNewApplicationId(305)) - Allocated new applicationId: 1 2015-11-22 19:30:00,435 DEBUG [IPC Server handler 4 on 38037] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getNewApplication queueTime= 1 procesingTime= 0 2015-11-22 19:30:00,435 DEBUG [IPC Server handler 4 on 38037] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 4 on 38037: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getNewApplication from 172.31.20.42:35310 Call#940 Retry#0 2015-11-22 19:30:00,435 DEBUG [IPC Server handler 4 on 38037] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 4 on 38037: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getNewApplication from 172.31.20.42:35310 Call#940 Retry#0 Wrote 51 bytes. 2015-11-22 19:30:00,435 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:38037 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:38037 from ubuntu got value #940 2015-11-22 19:30:00,435 DEBUG [Thread-4126] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getNewApplication took 1ms 2015-11-22 19:30:00,435 INFO [Thread-4126] distributedshell.Client (Client.java:run(491)) - Max mem capability of resources in this cluster 4096 2015-11-22 19:30:00,436 INFO [Thread-4126] distributedshell.Client (Client.java:run(502)) - Max virtual cores capability of resources in this cluster 4 2015-11-22 19:30:00,436 INFO [Thread-4126] distributedshell.Client (Client.java:run(528)) - Copy App Master jar from local filesystem and add to local environment 2015-11-22 19:30:00,442 INFO [Thread-4126] distributedshell.Client (Client.java:run(577)) - Set the environment for the application master 2015-11-22 19:30:00,443 INFO [Thread-4126] distributedshell.Client (Client.java:run(619)) - Setting up app master command 2015-11-22 19:30:00,443 INFO [Thread-4126] distributedshell.Client (Client.java:run(650)) - Completed setting up app master command {{JAVA_HOME}}/bin/java -Xmx512m org.apache.hadoop.yarn.applications.distributedshell.ContainerLaunchFailAppMaster --container_memory 128 --container_vcores 1 --num_containers 2 --priority 0 1>/AppMaster.stdout 2>/AppMaster.stderr 2015-11-22 19:30:00,443 INFO [Thread-4126] distributedshell.Client (Client.java:run(706)) - Submitting application to ASM 2015-11-22 19:30:00,443 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:38037 from ubuntu sending #941 2015-11-22 19:30:00,443 DEBUG [Socket Reader #1 for port 38037] ipc.Server (Server.java:processOneRpc(1897)) - got #941 2015-11-22 19:30:00,443 DEBUG [IPC Server handler 5 on 38037] ipc.Server (Server.java:run(2185)) - IPC Server handler 5 on 38037: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.submitApplication from 172.31.20.42:35310 Call#941 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:30:00,444 DEBUG [IPC Server handler 5 on 38037] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:30:00,444 INFO [IPC Server handler 5 on 38037] capacity.CapacityScheduler (CapacityScheduler.java:checkAndGetApplicationPriority(1941)) - Priority '0' is acceptable in queue : default for application: application_1448220594280_0001 for the user: ubuntu 2015-11-22 19:30:00,444 WARN [IPC Server handler 5 on 38037] rmapp.RMAppImpl (RMAppImpl.java:(420)) - The specific max attempts: 0 for application: 1 is invalid, because it is out of the range [1, 2]. Use the global max attempts instead. 2015-11-22 19:30:00,444 INFO [IPC Server handler 5 on 38037] resourcemanager.ClientRMService (ClientRMService.java:submitApplication(602)) - Application with id 1 submitted by user ubuntu 2015-11-22 19:30:00,444 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEvent.EventType: START 2015-11-22 19:30:00,444 INFO [IPC Server handler 5 on 38037] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(154)) - USER=ubuntu IP=172.31.20.42 OPERATION=Submit Application Request TARGET=ClientRMService RESULT=SUCCESS APPID=application_1448220594280_0001 2015-11-22 19:30:00,444 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(777)) - Processing event for application_1448220594280_0001 of type START 2015-11-22 19:30:00,444 DEBUG [IPC Server handler 5 on 38037] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: submitApplication queueTime= 1 procesingTime= 0 2015-11-22 19:30:00,445 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:transition(1079)) - Storing application with id application_1448220594280_0001 2015-11-22 19:30:00,445 DEBUG [IPC Server handler 5 on 38037] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 5 on 38037: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.submitApplication from 172.31.20.42:35310 Call#941 Retry#0 2015-11-22 19:30:00,445 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220594280_0001 State change from NEW to NEW_SAVING 2015-11-22 19:30:00,445 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:38037 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:38037 from ubuntu got value #941 2015-11-22 19:30:00,445 DEBUG [IPC Server handler 5 on 38037] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 5 on 38037: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.submitApplication from 172.31.20.42:35310 Call#941 Retry#0 Wrote 33 bytes. 2015-11-22 19:30:00,445 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStoreAppEvent.EventType: STORE_APP 2015-11-22 19:30:00,446 DEBUG [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:handleStoreEvent(1016)) - Processing event of type STORE_APP 2015-11-22 19:30:00,446 INFO [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:transition(208)) - Storing info for app: application_1448220594280_0001 2015-11-22 19:30:00,446 DEBUG [Thread-4126] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: submitApplication took 3ms 2015-11-22 19:30:00,446 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEvent.EventType: APP_NEW_SAVED 2015-11-22 19:30:00,446 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(777)) - Processing event for application_1448220594280_0001 of type APP_NEW_SAVED 2015-11-22 19:30:00,446 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220594280_0001 State change from NEW_SAVING to SUBMITTED 2015-11-22 19:30:00,446 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.AppAddedSchedulerEvent.EventType: APP_ADDED 2015-11-22 19:30:00,446 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:38037 from ubuntu sending #942 2015-11-22 19:30:00,446 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:addApplication(349)) - Application added - appId: application_1448220594280_0001 user: ubuntu leaf-queue of parent: root #applications: 1 2015-11-22 19:30:00,446 INFO [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:addApplication(765)) - Accepted application application_1448220594280_0001 from user: ubuntu, in queue: default 2015-11-22 19:30:00,446 DEBUG [Socket Reader #1 for port 38037] ipc.Server (Server.java:processOneRpc(1897)) - got #942 2015-11-22 19:30:00,446 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEvent.EventType: APP_ACCEPTED 2015-11-22 19:30:00,446 DEBUG [IPC Server handler 7 on 38037] ipc.Server (Server.java:run(2185)) - IPC Server handler 7 on 38037: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:35310 Call#942 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:30:00,447 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(777)) - Processing event for application_1448220594280_0001 of type APP_ACCEPTED 2015-11-22 19:30:00,447 DEBUG [IPC Server handler 7 on 38037] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:30:00,447 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220594280_0001 State change from SUBMITTED to ACCEPTED 2015-11-22 19:30:00,447 DEBUG [IPC Server handler 7 on 38037] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220594280_0001 owned by ubuntu 2015-11-22 19:30:00,447 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppStartAttemptEvent.EventType: START 2015-11-22 19:30:00,447 DEBUG [IPC Server handler 7 on 38037] scheduler.AbstractYarnScheduler (AbstractYarnScheduler.java:getAppResourceUsageReport(322)) - Request for appInfo of unknown attempt appattempt_1448220594280_0001_000001 2015-11-22 19:30:00,447 DEBUG [IPC Server handler 7 on 38037] scheduler.AbstractYarnScheduler (AbstractYarnScheduler.java:getAppResourceUsageReport(322)) - Request for appInfo of unknown attempt appattempt_1448220594280_0001_000001 2015-11-22 19:30:00,447 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220594280_0001_000001 of type START 2015-11-22 19:30:00,447 INFO [AsyncDispatcher event handler] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:registerAppAttempt(661)) - Registering app attempt : appattempt_1448220594280_0001_000001 2015-11-22 19:30:00,448 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220594280_0001_000001 State change from NEW to SUBMITTED 2015-11-22 19:30:00,448 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.AppAttemptAddedSchedulerEvent.EventType: APP_ATTEMPT_ADDED 2015-11-22 19:30:00,448 DEBUG [IPC Server handler 7 on 38037] scheduler.AbstractYarnScheduler (AbstractYarnScheduler.java:getAppResourceUsageReport(322)) - Request for appInfo of unknown attempt appattempt_1448220594280_0001_000001 2015-11-22 19:30:00,448 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:activateApplications(638)) - application AMResource maxAMResourcePerQueuePercent 0.1 amLimit lastClusterResource amIfStarted AM node-partition name 2015-11-22 19:30:00,448 DEBUG [IPC Server handler 7 on 38037] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 1 procesingTime= 1 2015-11-22 19:30:00,448 INFO [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:activateApplications(705)) - Application application_1448220594280_0001 from user: ubuntu activated in queue: default 2015-11-22 19:30:00,448 DEBUG [IPC Server handler 7 on 38037] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 7 on 38037: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:35310 Call#942 Retry#0 2015-11-22 19:30:00,448 INFO [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:addApplicationAttempt(720)) - Application added - appId: application_1448220594280_0001 user: ubuntu, leaf-queue: default #user-pending-applications: 0 #user-active-applications: 1 #queue-pending-applications: 0 #queue-active-applications: 1 2015-11-22 19:30:00,448 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:38037 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:38037 from ubuntu got value #942 2015-11-22 19:30:00,448 DEBUG [IPC Server handler 7 on 38037] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 7 on 38037: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:35310 Call#942 Retry#0 Wrote 393 bytes. 2015-11-22 19:30:00,448 DEBUG [Thread-4126] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:30:00,448 INFO [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:addApplicationAttempt(805)) - Added Application Attempt appattempt_1448220594280_0001_000001 to scheduler from user ubuntu in queue default 2015-11-22 19:30:00,449 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: ATTEMPT_ADDED 2015-11-22 19:30:00,449 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220594280_0001_000001 of type ATTEMPT_ADDED 2015-11-22 19:30:00,449 INFO [Thread-4126] impl.YarnClientImpl (YarnClientImpl.java:submitApplication(279)) - Submitted application application_1448220594280_0001 2015-11-22 19:30:00,449 DEBUG [AsyncDispatcher event handler] blacklist.SimpleBlacklistManager (SimpleBlacklistManager.java:getBlacklistUpdates(69)) - blacklist size 0 is less than failure threshold ratio 0.800000011920929 out of total usable nodes 1 2015-11-22 19:30:00,450 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:transition(1003)) - Using blacklist for AM: additions([]) and removals([]) 2015-11-22 19:30:00,450 DEBUG [AsyncDispatcher event handler] capacity.CapacityScheduler (CapacityScheduler.java:allocate(941)) - allocate: pre-update appattempt_1448220594280_0001_000001 ask size =1 2015-11-22 19:30:00,450 DEBUG [AsyncDispatcher event handler] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:updateResourceRequests(320)) - update: application=application_1448220594280_0001 request={Priority: 0, Capability: , # Containers: 1, Location: *, Relax Locality: true, Node Label Expression: } 2015-11-22 19:30:00,450 DEBUG [AsyncDispatcher event handler] scheduler.ActiveUsersManager (ActiveUsersManager.java:activateApplication(68)) - User ubuntu added to activeUsers, currently: 1 2015-11-22 19:30:00,452 DEBUG [AsyncDispatcher event handler] capacity.CapacityScheduler (CapacityScheduler.java:allocate(952)) - allocate: post-update 2015-11-22 19:30:00,453 DEBUG [AsyncDispatcher event handler] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(470)) - showRequests: application=application_1448220594280_0001 headRoom= currentConsumption=0 2015-11-22 19:30:00,453 DEBUG [AsyncDispatcher event handler] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(474)) - showRequests: application=application_1448220594280_0001 request={Priority: 0, Capability: , # Containers: 1, Location: *, Relax Locality: true, Node Label Expression: } 2015-11-22 19:30:00,453 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220594280_0001_000001 State change from SUBMITTED to SCHEDULED 2015-11-22 19:30:01,016 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:30:01,016 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 0 container statuses: [] 2015-11-22 19:30:01,017 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:30:01,017 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:35735 of type STATUS_UPDATE 2015-11-22 19:30:01,017 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:30:01,017 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:35735 clusterResources: 2015-11-22 19:30:01,017 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:35735 availableResource: 2015-11-22 19:30:01,017 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:30:01,017 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(408)) - Trying to assign containers to child-queue of root 2015-11-22 19:30:01,017 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:printChildQueues(617)) - printChildQueues - queue: root child-queues: root.defaultusedCapacity=(0.0), label=(*) 2015-11-22 19:30:01,017 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(564)) - Trying to assign to queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=1, numContainers=0 2015-11-22 19:30:01,017 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:assignContainers(817)) - assignContainers: node=ip-172-31-20-42.ap-northeast-1.compute.internal #applications=1 2015-11-22 19:30:01,018 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2015-11-22 19:30:01,018 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimitAndSetHeadroom(984)) - Headroom calculation for user ubuntu: userLimit= queueMaxAvailRes= consumed= headroom= 2015-11-22 19:30:01,018 DEBUG [ResourceManager Event Processor] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:assignContainers(477)) - pre-assignContainers for application application_1448220594280_0001 2015-11-22 19:30:01,019 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2015-11-22 19:30:01,019 DEBUG [ResourceManager Event Processor] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(470)) - showRequests: application=application_1448220594280_0001 headRoom= currentConsumption=0 2015-11-22 19:30:01,019 DEBUG [ResourceManager Event Processor] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(474)) - showRequests: application=application_1448220594280_0001 request={Priority: 0, Capability: , # Containers: 1, Location: *, Relax Locality: true, Node Label Expression: } 2015-11-22 19:30:01,019 DEBUG [ResourceManager Event Processor] allocator.IncreaseContainerAllocator (IncreaseContainerAllocator.java:assignContainers(184)) - Skip allocating increase request since we don't have any increase request on this node=ip-172-31-20-42.ap-northeast-1.compute.internal:35735 2015-11-22 19:30:01,019 DEBUG [ResourceManager Event Processor] allocator.RegularContainerAllocator (RegularContainerAllocator.java:assignContainer(407)) - assignContainers: node=ip-172-31-20-42.ap-northeast-1.compute.internal application=application_1448220594280_0001 priority=0 request={Priority: 0, Capability: , # Containers: 1, Location: *, Relax Locality: true, Node Label Expression: } type=OFF_SWITCH 2015-11-22 19:30:01,020 DEBUG [ResourceManager Event Processor] scheduler.ActiveUsersManager (ActiveUsersManager.java:deactivateApplication(94)) - User ubuntu removed from activeUsers, currently: 0 2015-11-22 19:30:01,022 DEBUG [ResourceManager Event Processor] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:allocate(594)) - allocate: applicationId=application_1448220594280_0001 container=container_1448220594280_0001_01_000001 host=ip-172-31-20-42.ap-northeast-1.compute.internal:35735 user=ubuntu resource= type=OFF_SWITCH 2015-11-22 19:30:01,022 DEBUG [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(424)) - Processing container_1448220594280_0001_01_000001 of type START 2015-11-22 19:30:01,022 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220594280_0001_01_000001 Container Transitioned from NEW to ALLOCATED 2015-11-22 19:30:01,022 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: CONTAINER_ALLOCATED 2015-11-22 19:30:01,022 DEBUG [ResourceManager Event Processor] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:allocate(208)) - allocate: applicationAttemptId=appattempt_1448220594280_0001_000001 container=container_1448220594280_0001_01_000001 host=ip-172-31-20-42.ap-northeast-1.compute.internal type=OFF_SWITCH 2015-11-22 19:30:01,022 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220594280_0001_000001 of type CONTAINER_ALLOCATED 2015-11-22 19:30:01,022 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220594280_0001 CONTAINERID=container_1448220594280_0001_01_000001 2015-11-22 19:30:01,023 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(154)) - Assigned container container_1448220594280_0001_01_000001 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:35735, which has 1 containers, used and available after allocation 2015-11-22 19:30:01,023 INFO [ResourceManager Event Processor] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(89)) - assignedContainer application attempt=appattempt_1448220594280_0001_000001 container=container_1448220594280_0001_01_000001 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@5ecb869b clusterResource= type=OFF_SWITCH 2015-11-22 19:30:01,023 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:assignContainers(881)) - post-assignContainers for application application_1448220594280_0001 2015-11-22 19:30:01,023 DEBUG [AsyncDispatcher event handler] security.BaseContainerTokenSecretManager (BaseContainerTokenSecretManager.java:createPassword(90)) - Creating password for container_1448220594280_0001_01_000001 for user container_1448220594280_0001_01_000001 (auth:SIMPLE) to be run on NM ip-172-31-20-42.ap-northeast-1.compute.internal:35735 2015-11-22 19:30:01,024 DEBUG [AsyncDispatcher event handler] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(197)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 1 cluster_timestamp: 1448220594280 } attemptId: 1 } id: 1 } nmHostAddr: "ip-172-31-20-42.ap-northeast-1.compute.internal:35735" appSubmitter: "ubuntu" resource { memory: 512 virtual_cores: 1 } expiryTimeStamp: 1448221201023 masterKeyId: -368252818 rmIdentifier: 1448220594280 priority { priority: 0 } creationTime: 1448220601020 nodeLabelExpression: "" containerType: APPLICATION_MASTER 2015-11-22 19:30:01,024 DEBUG [AsyncDispatcher event handler] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(197)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 1 cluster_timestamp: 1448220594280 } attemptId: 1 } id: 1 } nmHostAddr: "ip-172-31-20-42.ap-northeast-1.compute.internal:35735" appSubmitter: "ubuntu" resource { memory: 512 virtual_cores: 1 } expiryTimeStamp: 1448221201023 masterKeyId: -368252818 rmIdentifier: 1448220594280 priority { priority: 0 } creationTime: 1448220601020 nodeLabelExpression: "" containerType: APPLICATION_MASTER 2015-11-22 19:30:01,024 INFO [AsyncDispatcher event handler] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:createAndGetNMToken(200)) - Sending NMToken for nodeId : ip-172-31-20-42.ap-northeast-1.compute.internal:35735 for container : container_1448220594280_0001_01_000001 2015-11-22 19:30:01,025 DEBUG [AsyncDispatcher event handler] security.BaseNMTokenSecretManager (BaseNMTokenSecretManager.java:createPassword(75)) - creating password for appattempt_1448220594280_0001_000001 for user ubuntu to run on NM ip-172-31-20-42.ap-northeast-1.compute.internal:35735 2015-11-22 19:30:01,025 DEBUG [AsyncDispatcher event handler] security.NMTokenIdentifier (NMTokenIdentifier.java:write(96)) - Writing NMTokenIdentifier to RPC layer: appAttemptId { application_id { id: 1 cluster_timestamp: 1448220594280 } attemptId: 1 } nodeId { host: "ip-172-31-20-42.ap-northeast-1.compute.internal" port: 35735 } appSubmitter: "ubuntu" keyId: -1658271729 2015-11-22 19:30:01,025 DEBUG [AsyncDispatcher event handler] security.NMTokenIdentifier (NMTokenIdentifier.java:write(96)) - Writing NMTokenIdentifier to RPC layer: appAttemptId { application_id { id: 1 cluster_timestamp: 1448220594280 } attemptId: 1 } nodeId { host: "ip-172-31-20-42.ap-northeast-1.compute.internal" port: 35735 } appSubmitter: "ubuntu" keyId: -1658271729 2015-11-22 19:30:01,025 DEBUG [AsyncDispatcher event handler] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(424)) - Processing container_1448220594280_0001_01_000001 of type ACQUIRED 2015-11-22 19:30:01,025 INFO [AsyncDispatcher event handler] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220594280_0001_01_000001 Container Transitioned from ALLOCATED to ACQUIRED 2015-11-22 19:30:01,026 DEBUG [AsyncDispatcher event handler] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:30:01,026 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:30:01,026 DEBUG [ResourceManager Event Processor] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(470)) - showRequests: application=application_1448220594280_0001 headRoom= currentConsumption=512 2015-11-22 19:30:01,026 DEBUG [ResourceManager Event Processor] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(474)) - showRequests: application=application_1448220594280_0001 request={Priority: 0, Capability: , # Containers: 0, Location: *, Relax Locality: true, Node Label Expression: } 2015-11-22 19:30:01,026 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:30:01,026 INFO [AsyncDispatcher event handler] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:clearNodeSetForAttempt(146)) - Clear node set for appattempt_1448220594280_0001_000001 2015-11-22 19:30:01,027 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:30:01,027 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:storeAttempt(2026)) - Storing attempt: AppId: application_1448220594280_0001 AttemptId: appattempt_1448220594280_0001_000001 MasterContainer: Container: [ContainerId: container_1448220594280_0001_01_000001, NodeId: ip-172-31-20-42.ap-northeast-1.compute.internal:35735, NodeHttpAddress: ip-172-31-20-42.ap-northeast-1.compute.internal:43914, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.31.20.42:35735 }, ] 2015-11-22 19:30:01,027 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:30:01,027 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:allocateResource(1277)) - default user=ubuntu used= numContainers=1 headroom = user-resources= 2015-11-22 19:30:01,027 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(575)) - Assigned to queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.125, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 --> , OFF_SWITCH 2015-11-22 19:30:01,027 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStoreAppAttemptEvent.EventType: STORE_APP_ATTEMPT 2015-11-22 19:30:01,028 DEBUG [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:handleStoreEvent(1016)) - Processing event of type STORE_APP_ATTEMPT 2015-11-22 19:30:01,027 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220594280_0001_000001 State change from SCHEDULED to ALLOCATED_SAVING 2015-11-22 19:30:01,027 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(589)) - Re-sorting assigned queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.125, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 2015-11-22 19:30:01,028 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppRunningOnNodeEvent.EventType: APP_RUNNING_ON_NODE 2015-11-22 19:30:01,028 DEBUG [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:transition(295)) - Storing info for attempt: appattempt_1448220594280_0001_000001 2015-11-22 19:30:01,028 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(777)) - Processing event for application_1448220594280_0001 of type APP_RUNNING_ON_NODE 2015-11-22 19:30:01,028 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:printChildQueues(617)) - printChildQueues - queue: root child-queues: root.defaultusedCapacity=(0.125), label=(*) 2015-11-22 19:30:01,028 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: ATTEMPT_NEW_SAVED 2015-11-22 19:30:01,028 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(461)) - assignedContainer queue=root usedCapacity=0.125 absoluteUsedCapacity=0.125 used= cluster= 2015-11-22 19:30:01,028 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220594280_0001_000001 of type ATTEMPT_NEW_SAVED 2015-11-22 19:30:01,028 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(473)) - ParentQ=root assignedSoFarInThisIteration= usedCapacity=0.125 absoluteUsedCapacity=0.125 2015-11-22 19:30:01,029 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220594280_0001_000001 State change from ALLOCATED_SAVING to ALLOCATED 2015-11-22 19:30:01,029 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncherEvent.EventType: LAUNCH 2015-11-22 19:30:01,029 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(484)) - Not assigning more than one off-switch container, assignments so far: resource:; type:OFF_SWITCH; excessReservation:null; applicationid:null; skipped:false; fulfilled reservation:false; allocations(count/resource):1/; reservations(count/resource):0/ 2015-11-22 19:30:01,029 INFO [ApplicationMasterLauncher #0] amlauncher.AMLauncher (AMLauncher.java:run(254)) - Launching masterappattempt_1448220594280_0001_000001 2015-11-22 19:30:01,030 DEBUG [ApplicationMasterLauncher #0] ipc.YarnRPC (YarnRPC.java:create(59)) - Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC 2015-11-22 19:30:01,030 DEBUG [ApplicationMasterLauncher #0] security.BaseNMTokenSecretManager (BaseNMTokenSecretManager.java:createPassword(75)) - creating password for appattempt_1448220594280_0001_000001 for user ubuntu to run on NM ip-172-31-20-42.ap-northeast-1.compute.internal:35735 2015-11-22 19:30:01,030 DEBUG [ApplicationMasterLauncher #0] security.NMTokenIdentifier (NMTokenIdentifier.java:write(96)) - Writing NMTokenIdentifier to RPC layer: appAttemptId { application_id { id: 1 cluster_timestamp: 1448220594280 } attemptId: 1 } nodeId { host: "ip-172-31-20-42.ap-northeast-1.compute.internal" port: 35735 } appSubmitter: "ubuntu" keyId: -1658271729 2015-11-22 19:30:01,030 DEBUG [ApplicationMasterLauncher #0] security.NMTokenIdentifier (NMTokenIdentifier.java:write(96)) - Writing NMTokenIdentifier to RPC layer: appAttemptId { application_id { id: 1 cluster_timestamp: 1448220594280 } attemptId: 1 } nodeId { host: "ip-172-31-20-42.ap-northeast-1.compute.internal" port: 35735 } appSubmitter: "ubuntu" keyId: -1658271729 2015-11-22 19:30:01,031 DEBUG [ApplicationMasterLauncher #0] security.SecurityUtil (SecurityUtil.java:setTokenService(389)) - Acquired token Kind: NMToken, Service: 172.31.20.42:35735, Ident: (appAttemptId { application_id { id: 1 cluster_timestamp: 1448220594280 } attemptId: 1 } nodeId { host: "ip-172-31-20-42.ap-northeast-1.compute.internal" port: 35735 } appSubmitter: "ubuntu" keyId: -1658271729) 2015-11-22 19:30:01,031 DEBUG [ApplicationMasterLauncher #0] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220594280_0001_000001 (auth:SIMPLE) from:org.apache.hadoop.yarn.client.ServerProxy.createRetriableProxy(ServerProxy.java:93) 2015-11-22 19:30:01,031 DEBUG [ApplicationMasterLauncher #0] ipc.HadoopYarnProtoRPC (HadoopYarnProtoRPC.java:getProxy(47)) - Creating a HadoopYarnProtoRpc proxy for protocol interface org.apache.hadoop.yarn.api.ContainerManagementProtocol 2015-11-22 19:30:01,032 DEBUG [ApplicationMasterLauncher #0] ipc.Client (ClientCache.java:getClient(63)) - getting client out of cache: org.apache.hadoop.ipc.Client@67cf996b 2015-11-22 19:30:01,033 INFO [ApplicationMasterLauncher #0] amlauncher.AMLauncher (AMLauncher.java:launch(106)) - Setting up container Container: [ContainerId: container_1448220594280_0001_01_000001, NodeId: ip-172-31-20-42.ap-northeast-1.compute.internal:35735, NodeHttpAddress: ip-172-31-20-42.ap-northeast-1.compute.internal:43914, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.31.20.42:35735 }, ] for AM appattempt_1448220594280_0001_000001 2015-11-22 19:30:01,033 INFO [ApplicationMasterLauncher #0] amlauncher.AMLauncher (AMLauncher.java:createAMContainerLaunchContext(188)) - Command to launch container container_1448220594280_0001_01_000001 : {{JAVA_HOME}}/bin/java -Xmx512m org.apache.hadoop.yarn.applications.distributedshell.ContainerLaunchFailAppMaster --container_memory 128 --container_vcores 1 --num_containers 2 --priority 0 1>/AppMaster.stdout 2>/AppMaster.stderr 2015-11-22 19:30:01,033 INFO [ApplicationMasterLauncher #0] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:createAndGetAMRMToken(195)) - Create AMRMToken for ApplicationAttempt: appattempt_1448220594280_0001_000001 2015-11-22 19:30:01,033 INFO [ApplicationMasterLauncher #0] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:createPassword(307)) - Creating password for appattempt_1448220594280_0001_000001 2015-11-22 19:30:01,034 DEBUG [ApplicationMasterLauncher #0] ipc.Client (Client.java:(439)) - The ping interval is 60000 ms. 2015-11-22 19:30:01,034 DEBUG [ApplicationMasterLauncher #0] ipc.Client (Client.java:setupIOstreams(723)) - Connecting to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:35735 2015-11-22 19:30:01,034 DEBUG [IPC Server listener on 35735] ipc.Server (Server.java:register(2883)) - Server connection from 172.31.20.42:47740; # active connections: 1; # queued calls: 0 2015-11-22 19:30:01,035 DEBUG [ApplicationMasterLauncher #0] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220594280_0001_000001 (auth:SIMPLE) from:org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:745) 2015-11-22 19:30:01,035 DEBUG [ApplicationMasterLauncher #0] security.SaslRpcClient (SaslRpcClient.java:sendSaslMessage(454)) - Sending sasl message state: NEGOTIATE 2015-11-22 19:30:01,035 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:processOneRpc(1897)) - got #-33 2015-11-22 19:30:01,035 DEBUG [Socket Reader #1 for port 35735] security.SaslRpcServer (SaslRpcServer.java:create(175)) - Created SASL server with mechanism = DIGEST-MD5 2015-11-22 19:30:01,035 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:processResponse(1099)) - Socket Reader #1 for port 35735: responding to null from 172.31.20.42:47740 Call#-33 Retry#-1 2015-11-22 19:30:01,036 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:processResponse(1118)) - Socket Reader #1 for port 35735: responding to null from 172.31.20.42:47740 Call#-33 Retry#-1 Wrote 166 bytes. 2015-11-22 19:30:01,036 DEBUG [ApplicationMasterLauncher #0] security.SaslRpcClient (SaslRpcClient.java:getServerToken(264)) - Get token info proto:interface org.apache.hadoop.yarn.api.ContainerManagementProtocolPB info:org.apache.hadoop.yarn.security.ContainerManagerSecurityInfo$1@23f85235 2015-11-22 19:30:01,036 DEBUG [ApplicationMasterLauncher #0] security.NMTokenSelector (NMTokenSelector.java:selectToken(45)) - Looking for service: 172.31.20.42:35735. Current token is Kind: NMToken, Service: 172.31.20.42:35735, Ident: (appAttemptId { application_id { id: 1 cluster_timestamp: 1448220594280 } attemptId: 1 } nodeId { host: "ip-172-31-20-42.ap-northeast-1.compute.internal" port: 35735 } appSubmitter: "ubuntu" keyId: -1658271729) 2015-11-22 19:30:01,036 DEBUG [ApplicationMasterLauncher #0] security.SaslRpcClient (SaslRpcClient.java:createSaslClient(247)) - Creating SASL DIGEST-MD5(TOKEN) client to authenticate to service at default 2015-11-22 19:30:01,036 DEBUG [ApplicationMasterLauncher #0] security.SaslRpcClient (SaslRpcClient.java:selectSaslClient(176)) - Use TOKEN authentication for protocol ContainerManagementProtocolPB 2015-11-22 19:30:01,037 DEBUG [ApplicationMasterLauncher #0] security.SaslRpcClient (SaslRpcClient.java:handle(680)) - SASL client callback: setting username: Cg0KCQgBEOigsoWTKhABEjUKL2lwLTE3Mi0zMS0yMC00Mi5hcC1ub3J0aGVhc3QtMS5jb21wdXRlLmludGVybmFsEJeXAhoGdWJ1bnR1II+Qo+n5/////wE= 2015-11-22 19:30:01,037 DEBUG [ApplicationMasterLauncher #0] security.SaslRpcClient (SaslRpcClient.java:handle(685)) - SASL client callback: setting userPassword 2015-11-22 19:30:01,037 DEBUG [ApplicationMasterLauncher #0] security.SaslRpcClient (SaslRpcClient.java:handle(690)) - SASL client callback: setting realm: default 2015-11-22 19:30:01,037 DEBUG [ApplicationMasterLauncher #0] security.SaslRpcClient (SaslRpcClient.java:sendSaslMessage(454)) - Sending sasl message state: INITIATE token: "charset=utf-8,username=\"Cg0KCQgBEOigsoWTKhABEjUKL2lwLTE3Mi0zMS0yMC00Mi5hcC1ub3J0aGVhc3QtMS5jb21wdXRlLmludGVybmFsEJeXAhoGdWJ1bnR1II+Qo+n5/////wE=\",realm=\"default\",nonce=\"6PGSGD04RcbntIIg0upC9B5lipJkt6bJKJS3elwx\",nc=00000001,cnonce=\"0pWTG+Buf6rWszWgkUZbRhH6ezSSnfEF78USmig5\",digest-uri=\"/default\",maxbuf=65536,response=49ccd36fc04c407515a5901dc5b460e3,qop=auth" auths { method: "TOKEN" mechanism: "DIGEST-MD5" protocol: "" serverId: "default" } 2015-11-22 19:30:01,038 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:processOneRpc(1897)) - got #-33 2015-11-22 19:30:01,038 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:processSaslToken(1529)) - Have read input token of size 358 for processing by saslServer.evaluateResponse() 2015-11-22 19:30:01,038 DEBUG [Socket Reader #1 for port 35735] security.BaseNMTokenSecretManager (BaseNMTokenSecretManager.java:retrivePasswordInternal(103)) - creating password for appattempt_1448220594280_0001_000001 for user ubuntu to run on NM ip-172-31-20-42.ap-northeast-1.compute.internal:35735 2015-11-22 19:30:01,039 DEBUG [Socket Reader #1 for port 35735] security.NMTokenIdentifier (NMTokenIdentifier.java:write(96)) - Writing NMTokenIdentifier to RPC layer: appAttemptId { application_id { id: 1 cluster_timestamp: 1448220594280 } attemptId: 1 } nodeId { host: "ip-172-31-20-42.ap-northeast-1.compute.internal" port: 35735 } appSubmitter: "ubuntu" keyId: -1658271729 2015-11-22 19:30:01,039 DEBUG [Socket Reader #1 for port 35735] security.NMTokenSecretManagerInNM (NMTokenSecretManagerInNM.java:retrievePassword(180)) - NMToken password retrieved successfully!! 2015-11-22 19:30:01,039 DEBUG [Socket Reader #1 for port 35735] security.SaslRpcServer (SaslRpcServer.java:handle(310)) - SASL server DIGEST-MD5 callback: setting password for client: appattempt_1448220594280_0001_000001 (auth:SIMPLE) 2015-11-22 19:30:01,039 DEBUG [Socket Reader #1 for port 35735] security.SaslRpcServer (SaslRpcServer.java:handle(328)) - SASL server DIGEST-MD5 callback: setting canonicalized client ID: appattempt_1448220594280_0001_000001 2015-11-22 19:30:01,039 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:buildSaslResponse(1546)) - Will send SUCCESS token of size 40 from saslServer. 2015-11-22 19:30:01,039 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:saslProcess(1434)) - SASL server context established. Negotiated QoP is auth 2015-11-22 19:30:01,039 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:saslProcess(1439)) - SASL server successfully authenticated client: appattempt_1448220594280_0001_000001 (auth:SIMPLE) 2015-11-22 19:30:01,039 INFO [Socket Reader #1 for port 35735] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220594280_0001_000001 (auth:SIMPLE) 2015-11-22 19:30:01,040 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:processResponse(1099)) - Socket Reader #1 for port 35735: responding to null from 172.31.20.42:47740 Call#-33 Retry#-1 2015-11-22 19:30:01,040 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:processResponse(1118)) - Socket Reader #1 for port 35735: responding to null from 172.31.20.42:47740 Call#-33 Retry#-1 Wrote 64 bytes. 2015-11-22 19:30:01,040 DEBUG [ApplicationMasterLauncher #0] ipc.Client (Client.java:setupIOstreams(773)) - Negotiated QOP is :auth 2015-11-22 19:30:01,040 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:35735 from appattempt_1448220594280_0001_000001] ipc.Client (Client.java:run(1000)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:35735 from appattempt_1448220594280_0001_000001: starting, having connections 2 2015-11-22 19:30:01,042 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:35735 from appattempt_1448220594280_0001_000001 sending #943 2015-11-22 19:30:01,042 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:processOneRpc(1897)) - got #-3 2015-11-22 19:30:01,042 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:authorizeConnection(2097)) - Successfully authorized userInfo { } protocol: "org.apache.hadoop.yarn.api.ContainerManagementProtocolPB" 2015-11-22 19:30:01,042 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:processOneRpc(1897)) - got #943 2015-11-22 19:30:01,042 DEBUG [IPC Server handler 0 on 35735] ipc.Server (Server.java:run(2185)) - IPC Server handler 0 on 35735: org.apache.hadoop.yarn.api.ContainerManagementProtocolPB.startContainers from 172.31.20.42:47740 Call#943 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:30:01,043 DEBUG [IPC Server handler 0 on 35735] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220594280_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:30:01,044 DEBUG [IPC Server handler 0 on 35735] security.BaseContainerTokenSecretManager (BaseContainerTokenSecretManager.java:retrievePasswordInternal(118)) - Retrieving password for container_1448220594280_0001_01_000001 for user container_1448220594280_0001_01_000001 (auth:SIMPLE) to be run on NM ip-172-31-20-42.ap-northeast-1.compute.internal:35735 2015-11-22 19:30:01,044 DEBUG [IPC Server handler 0 on 35735] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(197)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 1 cluster_timestamp: 1448220594280 } attemptId: 1 } id: 1 } nmHostAddr: "ip-172-31-20-42.ap-northeast-1.compute.internal:35735" appSubmitter: "ubuntu" resource { memory: 512 virtual_cores: 1 } expiryTimeStamp: 1448221201023 masterKeyId: -368252818 rmIdentifier: 1448220594280 priority { priority: 0 } creationTime: 1448220601020 nodeLabelExpression: "" containerType: APPLICATION_MASTER 2015-11-22 19:30:01,045 DEBUG [IPC Server handler 0 on 35735] security.NMTokenSecretManagerInNM (NMTokenSecretManagerInNM.java:appAttemptStartContainer(226)) - NMToken key updated for application attempt : appattempt_1448220594280_0001_000001 2015-11-22 19:30:01,045 INFO [IPC Server handler 0 on 35735] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:startContainerInternal(889)) - Start request for container_1448220594280_0001_01_000001 by user ubuntu 2015-11-22 19:30:01,045 DEBUG [IPC Server handler 0 on 35735] utils.YarnServerSecurityUtils (YarnServerSecurityUtils.java:parseCredentials(135)) - = Kind: YARN_AM_RM_TOKEN, Service: , Ident: (appAttemptId { application_id { id: 1 cluster_timestamp: 1448220594280 } attemptId: 1 } keyId: -720358617) 2015-11-22 19:30:01,045 INFO [IPC Server handler 0 on 35735] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:startContainerInternal(930)) - Creating a new application reference for app application_1448220594280_0001 2015-11-22 19:30:01,046 INFO [IPC Server handler 0 on 35735] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu IP=172.31.20.42 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1448220594280_0001 CONTAINERID=container_1448220594280_0001_01_000001 2015-11-22 19:30:01,046 DEBUG [IPC Server handler 0 on 35735] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: startContainers queueTime= 1 procesingTime= 3 2015-11-22 19:30:01,046 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationInitEvent.EventType: INIT_APPLICATION 2015-11-22 19:30:01,048 DEBUG [IPC Server handler 0 on 35735] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 0 on 35735: responding to org.apache.hadoop.yarn.api.ContainerManagementProtocolPB.startContainers from 172.31.20.42:47740 Call#943 Retry#0 2015-11-22 19:30:01,048 DEBUG [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(452)) - Processing application_1448220594280_0001 of type INIT_APPLICATION 2015-11-22 19:30:01,048 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(463)) - Application application_1448220594280_0001 transitioned from NEW to INITING 2015-11-22 19:30:01,048 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationContainerInitEvent.EventType: INIT_CONTAINER 2015-11-22 19:30:01,048 DEBUG [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(452)) - Processing application_1448220594280_0001 of type INIT_CONTAINER 2015-11-22 19:30:01,048 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(303)) - Adding container_1448220594280_0001_01_000001 to application application_1448220594280_0001 2015-11-22 19:30:01,048 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.event.LogHandlerAppStartedEvent.EventType: APPLICATION_STARTED 2015-11-22 19:30:01,048 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationEvent.EventType: APPLICATION_LOG_HANDLING_INITED 2015-11-22 19:30:01,048 DEBUG [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(452)) - Processing application_1448220594280_0001 of type APPLICATION_LOG_HANDLING_INITED 2015-11-22 19:30:01,048 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.ApplicationLocalizationEvent.EventType: INIT_APPLICATION_RESOURCES 2015-11-22 19:30:01,048 DEBUG [IPC Server handler 0 on 35735] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 0 on 35735: responding to org.apache.hadoop.yarn.api.ContainerManagementProtocolPB.startContainers from 172.31.20.42:47740 Call#943 Retry#0 Wrote 52 bytes. 2015-11-22 19:30:01,049 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationInitedEvent.EventType: APPLICATION_INITED 2015-11-22 19:30:01,049 DEBUG [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(452)) - Processing application_1448220594280_0001 of type APPLICATION_INITED 2015-11-22 19:30:01,049 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(463)) - Application application_1448220594280_0001 transitioned from INITING to RUNNING 2015-11-22 19:30:01,049 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerInitEvent.EventType: INIT_CONTAINER 2015-11-22 19:30:01,049 DEBUG [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1131)) - Processing container_1448220594280_0001_01_000001 of type INIT_CONTAINER 2015-11-22 19:30:01,050 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220594280_0001_01_000001 transitioned from NEW to LOCALIZING 2015-11-22 19:30:01,050 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServicesEvent.EventType: CONTAINER_INIT 2015-11-22 19:30:01,050 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_INIT for appId application_1448220594280_0001 2015-11-22 19:30:01,050 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.ContainerLocalizationRequestEvent.EventType: INIT_CONTAINER_RESOURCES 2015-11-22 19:30:01,050 DEBUG [AsyncDispatcher event handler] localizer.LocalizedResource (LocalizedResource.java:handle(193)) - Processing file:/home/ubuntu/DistributedShell/application_1448220594280_0001/AppMaster.jar of type REQUEST 2015-11-22 19:30:01,050 INFO [AsyncDispatcher event handler] localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource file:/home/ubuntu/DistributedShell/application_1448220594280_0001/AppMaster.jar transitioned from INIT to DOWNLOADING 2015-11-22 19:30:01,051 DEBUG [AsyncDispatcher event handler] localizer.ResourceLocalizationService (ResourceLocalizationService.java:handleInitContainerResources(480)) - Localizing file:/home/ubuntu/DistributedShell/application_1448220594280_0001/AppMaster.jar for container container_1448220594280_0001_01_000001 2015-11-22 19:30:01,051 DEBUG [AsyncDispatcher event handler] localizer.LocalizedResource (LocalizedResource.java:handle(193)) - Processing file:/home/ubuntu/DistributedShell/application_1448220594280_0001/shellCommands of type REQUEST 2015-11-22 19:30:01,051 INFO [AsyncDispatcher event handler] localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource file:/home/ubuntu/DistributedShell/application_1448220594280_0001/shellCommands transitioned from INIT to DOWNLOADING 2015-11-22 19:30:01,051 DEBUG [AsyncDispatcher event handler] localizer.ResourceLocalizationService (ResourceLocalizationService.java:handleInitContainerResources(480)) - Localizing file:/home/ubuntu/DistributedShell/application_1448220594280_0001/shellCommands for container container_1448220594280_0001_01_000001 2015-11-22 19:30:01,051 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizerResourceRequestEvent.EventType: REQUEST_RESOURCE_LOCALIZATION 2015-11-22 19:30:01,051 INFO [AsyncDispatcher event handler] localizer.ResourceLocalizationService (ResourceLocalizationService.java:handle(742)) - Created localizer for container_1448220594280_0001_01_000001 2015-11-22 19:30:01,051 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizerResourceRequestEvent.EventType: REQUEST_RESOURCE_LOCALIZATION 2015-11-22 19:30:01,052 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:35735 from appattempt_1448220594280_0001_000001] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:35735 from appattempt_1448220594280_0001_000001 got value #943 2015-11-22 19:30:01,052 DEBUG [ApplicationMasterLauncher #0] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: startContainers took 18ms 2015-11-22 19:30:01,052 INFO [ApplicationMasterLauncher #0] amlauncher.AMLauncher (AMLauncher.java:launch(127)) - Done launching container Container: [ContainerId: container_1448220594280_0001_01_000001, NodeId: ip-172-31-20-42.ap-northeast-1.compute.internal:35735, NodeHttpAddress: ip-172-31-20-42.ap-northeast-1.compute.internal:43914, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.31.20.42:35735 }, ] for AM appattempt_1448220594280_0001_000001 2015-11-22 19:30:01,052 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: LAUNCHED 2015-11-22 19:30:01,052 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220594280_0001_000001 of type LAUNCHED 2015-11-22 19:30:01,053 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220594280_0001_000001 State change from ALLOCATED to LAUNCHED 2015-11-22 19:30:01,053 DEBUG [LocalizerRunner for container_1448220594280_0001_01_000001] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.fs.FileContext.getAbstractFileSystem(FileContext.java:339) 2015-11-22 19:30:01,059 INFO [LocalizerRunner for container_1448220594280_0001_01_000001] localizer.ResourceLocalizationService (ResourceLocalizationService.java:writeCredentials(1221)) - Writing credentials to the nmPrivate file /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/container_1448220594280_0001_01_000001.tokens. Credentials list: 2015-11-22 19:30:01,059 DEBUG [LocalizerRunner for container_1448220594280_0001_01_000001] localizer.ResourceLocalizationService (ResourceLocalizationService.java:writeCredentials(1226)) - Kind: YARN_AM_RM_TOKEN, Service: , Ident: (appAttemptId { application_id { id: 1 cluster_timestamp: 1448220594280 } attemptId: 1 } keyId: -720358617) : df d6 aa 9b ca 3a 29 36 42 ff 2015-11-22 19:30:01,061 INFO [LocalizerRunner for container_1448220594280_0001_01_000001] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:createUserCacheDirs(632)) - Initializing user ubuntu 2015-11-22 19:30:01,068 INFO [LocalizerRunner for container_1448220594280_0001_01_000001] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:startLocalizer(126)) - Copying from /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/container_1448220594280_0001_01_000001.tokens to /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220594280_0001/container_1448220594280_0001_01_000001.tokens 2015-11-22 19:30:01,068 INFO [LocalizerRunner for container_1448220594280_0001_01_000001] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:startLocalizer(133)) - Localizer CWD set to /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220594280_0001 = file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220594280_0001 2015-11-22 19:30:01,080 DEBUG [LocalizerRunner for container_1448220594280_0001_01_000001] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.runLocalization(ContainerLocalizer.java:152) 2015-11-22 19:30:01,080 DEBUG [LocalizerRunner for container_1448220594280_0001_01_000001] ipc.YarnRPC (YarnRPC.java:create(59)) - Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC 2015-11-22 19:30:01,080 DEBUG [LocalizerRunner for container_1448220594280_0001_01_000001] ipc.HadoopYarnProtoRPC (HadoopYarnProtoRPC.java:getProxy(47)) - Creating a HadoopYarnProtoRpc proxy for protocol interface org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocol 2015-11-22 19:30:01,080 DEBUG [LocalizerRunner for container_1448220594280_0001_01_000001] ipc.Client (ClientCache.java:getClient(63)) - getting client out of cache: org.apache.hadoop.ipc.Client@67cf996b 2015-11-22 19:30:01,081 DEBUG [LocalizerRunner for container_1448220594280_0001_01_000001] ipc.Client (Client.java:(439)) - The ping interval is 60000 ms. 2015-11-22 19:30:01,081 DEBUG [LocalizerRunner for container_1448220594280_0001_01_000001] ipc.Client (Client.java:setupIOstreams(723)) - Connecting to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:40868 2015-11-22 19:30:01,081 DEBUG [IPC Server listener on 40868] ipc.Server (Server.java:register(2883)) - Server connection from 172.31.20.42:49300; # active connections: 1; # queued calls: 0 2015-11-22 19:30:01,082 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:40868 from ubuntu] ipc.Client (Client.java:run(1000)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:40868 from ubuntu: starting, having connections 3 2015-11-22 19:30:01,082 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:40868 from ubuntu sending #944 2015-11-22 19:30:01,083 DEBUG [Socket Reader #1 for port 40868] ipc.Server (Server.java:processOneRpc(1897)) - got #-3 2015-11-22 19:30:01,083 DEBUG [Socket Reader #1 for port 40868] ipc.Server (Server.java:authorizeConnection(2097)) - Successfully authorized userInfo { effectiveUser: "ubuntu" } protocol: "org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB" 2015-11-22 19:30:01,083 DEBUG [Socket Reader #1 for port 40868] ipc.Server (Server.java:processOneRpc(1897)) - got #944 2015-11-22 19:30:01,083 DEBUG [IPC Server handler 0 on 40868] ipc.Server (Server.java:run(2185)) - IPC Server handler 0 on 40868: org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 172.31.20.42:49300 Call#944 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:30:01,083 DEBUG [IPC Server handler 0 on 40868] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:30:01,085 DEBUG [IPC Server handler 0 on 40868] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: heartbeat queueTime= 0 procesingTime= 2 2015-11-22 19:30:01,086 DEBUG [IPC Server handler 0 on 40868] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 0 on 40868: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 172.31.20.42:49300 Call#944 Retry#0 2015-11-22 19:30:01,086 DEBUG [IPC Server handler 0 on 40868] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 0 on 40868: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 172.31.20.42:49300 Call#944 Retry#0 Wrote 421 bytes. 2015-11-22 19:30:01,086 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:40868 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:40868 from ubuntu got value #944 2015-11-22 19:30:01,086 DEBUG [LocalizerRunner for container_1448220594280_0001_01_000001] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: heartbeat took 5ms 2015-11-22 19:30:01,090 DEBUG [ContainerLocalizer Downloader] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:359) 2015-11-22 19:30:01,094 DEBUG [ContainerLocalizer Downloader] util.FSDownload (FSDownload.java:changePermissions(412)) - Changing permissions for path file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220594280_0001/filecache/10_tmp/AppMaster.jar to perm r-x------ 2015-11-22 19:30:01,095 DEBUG [ContainerLocalizer Downloader] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.yarn.util.FSDownload.changePermissions(FSDownload.java:419) 2015-11-22 19:30:01,100 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:40868 from ubuntu sending #945 2015-11-22 19:30:01,100 DEBUG [Socket Reader #1 for port 40868] ipc.Server (Server.java:processOneRpc(1897)) - got #945 2015-11-22 19:30:01,100 DEBUG [IPC Server handler 3 on 40868] ipc.Server (Server.java:run(2185)) - IPC Server handler 3 on 40868: org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 172.31.20.42:49300 Call#945 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:30:01,101 DEBUG [IPC Server handler 3 on 40868] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:30:01,101 DEBUG [IPC Server handler 3 on 40868] localizer.LocalizedResource (LocalizedResource.java:handle(193)) - Processing file:/home/ubuntu/DistributedShell/application_1448220594280_0001/AppMaster.jar of type LOCALIZED 2015-11-22 19:30:01,101 INFO [IPC Server handler 3 on 40868] localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource file:/home/ubuntu/DistributedShell/application_1448220594280_0001/AppMaster.jar(->/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220594280_0001/filecache/10/AppMaster.jar) transitioned from DOWNLOADING to LOCALIZED 2015-11-22 19:30:01,101 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerResourceLocalizedEvent.EventType: RESOURCE_LOCALIZED 2015-11-22 19:30:01,102 DEBUG [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1131)) - Processing container_1448220594280_0001_01_000001 of type RESOURCE_LOCALIZED 2015-11-22 19:30:01,102 DEBUG [IPC Server handler 3 on 40868] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: heartbeat queueTime= 1 procesingTime= 1 2015-11-22 19:30:01,104 DEBUG [IPC Server handler 3 on 40868] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 3 on 40868: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 172.31.20.42:49300 Call#945 Retry#0 2015-11-22 19:30:01,104 DEBUG [IPC Server handler 3 on 40868] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 3 on 40868: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 172.31.20.42:49300 Call#945 Retry#0 Wrote 421 bytes. 2015-11-22 19:30:01,104 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:40868 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:40868 from ubuntu got value #945 2015-11-22 19:30:01,105 DEBUG [LocalizerRunner for container_1448220594280_0001_01_000001] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: heartbeat took 5ms 2015-11-22 19:30:01,108 DEBUG [ContainerLocalizer Downloader] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:359) 2015-11-22 19:30:01,112 DEBUG [ContainerLocalizer Downloader] util.FSDownload (FSDownload.java:changePermissions(412)) - Changing permissions for path file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220594280_0001/filecache/11_tmp/shellCommands to perm r-x------ 2015-11-22 19:30:01,112 DEBUG [ContainerLocalizer Downloader] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.yarn.util.FSDownload.changePermissions(FSDownload.java:419) 2015-11-22 19:30:01,117 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:40868 from ubuntu sending #946 2015-11-22 19:30:01,117 DEBUG [Socket Reader #1 for port 40868] ipc.Server (Server.java:processOneRpc(1897)) - got #946 2015-11-22 19:30:01,117 DEBUG [IPC Server handler 1 on 40868] ipc.Server (Server.java:run(2185)) - IPC Server handler 1 on 40868: org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 172.31.20.42:49300 Call#946 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:30:01,117 DEBUG [IPC Server handler 1 on 40868] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:30:01,118 DEBUG [IPC Server handler 1 on 40868] localizer.LocalizedResource (LocalizedResource.java:handle(193)) - Processing file:/home/ubuntu/DistributedShell/application_1448220594280_0001/shellCommands of type LOCALIZED 2015-11-22 19:30:01,118 INFO [IPC Server handler 1 on 40868] localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource file:/home/ubuntu/DistributedShell/application_1448220594280_0001/shellCommands(->/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220594280_0001/filecache/11/shellCommands) transitioned from DOWNLOADING to LOCALIZED 2015-11-22 19:30:01,118 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerResourceLocalizedEvent.EventType: RESOURCE_LOCALIZED 2015-11-22 19:30:01,118 DEBUG [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1131)) - Processing container_1448220594280_0001_01_000001 of type RESOURCE_LOCALIZED 2015-11-22 19:30:01,118 DEBUG [IPC Server handler 1 on 40868] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: heartbeat queueTime= 1 procesingTime= 0 2015-11-22 19:30:01,118 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220594280_0001_01_000001 transitioned from LOCALIZING to LOCALIZED 2015-11-22 19:30:01,119 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.ContainerLocalizationEvent.EventType: CONTAINER_RESOURCES_LOCALIZED 2015-11-22 19:30:01,119 DEBUG [IPC Server handler 1 on 40868] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 1 on 40868: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 172.31.20.42:49300 Call#946 Retry#0 2015-11-22 19:30:01,119 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncherEvent.EventType: LAUNCH_CONTAINER 2015-11-22 19:30:01,119 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:40868 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:40868 from ubuntu got value #946 2015-11-22 19:30:01,119 DEBUG [IPC Server handler 1 on 40868] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 1 on 40868: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 172.31.20.42:49300 Call#946 Retry#0 Wrote 35 bytes. 2015-11-22 19:30:01,119 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.sharedcache.SharedCacheUploadEvent.EventType: UPLOAD 2015-11-22 19:30:01,119 DEBUG [LocalizerRunner for container_1448220594280_0001_01_000001] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: heartbeat took 2ms 2015-11-22 19:30:01,120 DEBUG [ContainersLauncher #0] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.fs.FileContext.getAbstractFileSystem(FileContext.java:339) 2015-11-22 19:30:01,137 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerEvent.EventType: CONTAINER_LAUNCHED 2015-11-22 19:30:01,137 DEBUG [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1131)) - Processing container_1448220594280_0001_01_000001 of type CONTAINER_LAUNCHED 2015-11-22 19:30:01,138 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220594280_0001_01_000001 transitioned from LOCALIZED to RUNNING 2015-11-22 19:30:01,138 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerStartMonitoringEvent.EventType: START_MONITORING_CONTAINER 2015-11-22 19:30:01,153 INFO [ContainersLauncher #0] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:buildCommandExecutor(281)) - launchContainer: [nice, -n, 0, bash, /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220594280_0001/container_1448220594280_0001_01_000001/default_container_executor.sh] 2015-11-22 19:30:01,450 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:38037 from ubuntu sending #947 2015-11-22 19:30:01,450 DEBUG [Socket Reader #1 for port 38037] ipc.Server (Server.java:processOneRpc(1897)) - got #947 2015-11-22 19:30:01,450 DEBUG [IPC Server handler 8 on 38037] ipc.Server (Server.java:run(2185)) - IPC Server handler 8 on 38037: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:35310 Call#947 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:30:01,450 DEBUG [IPC Server handler 8 on 38037] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:30:01,451 DEBUG [IPC Server handler 8 on 38037] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220594280_0001 owned by ubuntu 2015-11-22 19:30:01,451 DEBUG [IPC Server handler 8 on 38037] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 1 2015-11-22 19:30:01,452 DEBUG [IPC Server handler 8 on 38037] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 8 on 38037: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:35310 Call#947 Retry#0 2015-11-22 19:30:01,452 DEBUG [IPC Server handler 8 on 38037] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 8 on 38037: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:35310 Call#947 Retry#0 Wrote 333 bytes. 2015-11-22 19:30:01,452 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:38037 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:38037 from ubuntu got value #947 2015-11-22 19:30:01,452 DEBUG [Thread-4126] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:30:01,452 INFO [Thread-4126] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=N/A, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220600444, yarnAppState=ACCEPTED, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40017/proxy/application_1448220594280_0001/, appUser=ubuntu 2015-11-22 19:30:02,017 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:30:02,018 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220594280_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:30:02,018 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:30:02,018 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:35735 of type STATUS_UPDATE 2015-11-22 19:30:02,018 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handleContainerStatus(1208)) - Container container_1448220594280_0001_01_000001 is the first container get launched for application application_1448220594280_0001 2015-11-22 19:30:02,019 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:30:02,019 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:35735 clusterResources: 2015-11-22 19:30:02,019 DEBUG [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(424)) - Processing container_1448220594280_0001_01_000001 of type LAUNCHED 2015-11-22 19:30:02,019 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220594280_0001_01_000001 Container Transitioned from ACQUIRED to RUNNING 2015-11-22 19:30:02,019 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:35735 availableResource: 2015-11-22 19:30:02,019 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:30:02,019 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:30:02,120 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:40868 from ubuntu sending #948 2015-11-22 19:30:02,120 DEBUG [Socket Reader #1 for port 40868] ipc.Server (Server.java:processOneRpc(1897)) - got #948 2015-11-22 19:30:02,120 DEBUG [IPC Server handler 4 on 40868] ipc.Server (Server.java:run(2185)) - IPC Server handler 4 on 40868: org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 172.31.20.42:49300 Call#948 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:30:02,120 DEBUG [IPC Server handler 4 on 40868] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:30:02,120 DEBUG [IPC Server handler 4 on 40868] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: heartbeat queueTime= 0 procesingTime= 0 2015-11-22 19:30:02,121 DEBUG [IPC Server handler 4 on 40868] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 4 on 40868: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 172.31.20.42:49300 Call#948 Retry#0 2015-11-22 19:30:02,121 DEBUG [IPC Server handler 4 on 40868] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 4 on 40868: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 172.31.20.42:49300 Call#948 Retry#0 Wrote 35 bytes. 2015-11-22 19:30:02,121 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:40868 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:40868 from ubuntu got value #948 2015-11-22 19:30:02,121 DEBUG [LocalizerRunner for container_1448220594280_0001_01_000001] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: heartbeat took 2ms 2015-11-22 19:30:02,121 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:40868 from ubuntu sending #949 2015-11-22 19:30:02,122 DEBUG [Socket Reader #1 for port 40868] ipc.Server (Server.java:processOneRpc(1897)) - got #949 2015-11-22 19:30:02,122 DEBUG [IPC Server handler 2 on 40868] ipc.Server (Server.java:run(2185)) - IPC Server handler 2 on 40868: org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 172.31.20.42:49300 Call#949 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:30:02,122 DEBUG [IPC Server handler 2 on 40868] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:30:02,122 DEBUG [IPC Server handler 2 on 40868] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: heartbeat queueTime= 0 procesingTime= 0 2015-11-22 19:30:02,122 DEBUG [IPC Server handler 2 on 40868] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 2 on 40868: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 172.31.20.42:49300 Call#949 Retry#0 2015-11-22 19:30:02,122 DEBUG [IPC Server handler 2 on 40868] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 2 on 40868: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 172.31.20.42:49300 Call#949 Retry#0 Wrote 35 bytes. 2015-11-22 19:30:02,122 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:40868 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:40868 from ubuntu got value #949 2015-11-22 19:30:02,122 DEBUG [LocalizerRunner for container_1448220594280_0001_01_000001] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: heartbeat took 1ms 2015-11-22 19:30:02,137 DEBUG [DeletionService #0] nodemanager.DeletionService (DeletionService.java:run(266)) - FileDeletionTask : user : null subDir : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/container_1448220594280_0001_01_000001.tokens baseDir : null 2015-11-22 19:30:02,138 DEBUG [DeletionService #0] nodemanager.DeletionService (DeletionService.java:run(271)) - NM deleting absolute path : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/container_1448220594280_0001_01_000001.tokens 2015-11-22 19:30:02,453 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:38037 from ubuntu sending #950 2015-11-22 19:30:02,453 DEBUG [Socket Reader #1 for port 38037] ipc.Server (Server.java:processOneRpc(1897)) - got #950 2015-11-22 19:30:02,453 DEBUG [IPC Server handler 10 on 38037] ipc.Server (Server.java:run(2185)) - IPC Server handler 10 on 38037: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:35310 Call#950 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:30:02,453 DEBUG [IPC Server handler 10 on 38037] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:30:02,454 DEBUG [IPC Server handler 10 on 38037] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220594280_0001 owned by ubuntu 2015-11-22 19:30:02,454 DEBUG [IPC Server handler 10 on 38037] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 1 procesingTime= 0 2015-11-22 19:30:02,454 DEBUG [IPC Server handler 10 on 38037] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 10 on 38037: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:35310 Call#950 Retry#0 2015-11-22 19:30:02,455 DEBUG [IPC Server handler 10 on 38037] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 10 on 38037: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:35310 Call#950 Retry#0 Wrote 333 bytes. 2015-11-22 19:30:02,455 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:38037 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:38037 from ubuntu got value #950 2015-11-22 19:30:02,455 DEBUG [Thread-4126] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:30:02,455 INFO [Thread-4126] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=N/A, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220600444, yarnAppState=ACCEPTED, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40017/proxy/application_1448220594280_0001/, appUser=ubuntu 2015-11-22 19:30:03,018 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:30:03,019 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220594280_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:30:03,019 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:30:03,019 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:35735 of type STATUS_UPDATE 2015-11-22 19:30:03,019 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:30:03,019 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:35735 clusterResources: 2015-11-22 19:30:03,020 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:35735 availableResource: 2015-11-22 19:30:03,020 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:30:03,020 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:30:03,078 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - REQUEST /ws/v1/timeline/ on org.mortbay.jetty.HttpConnection@244c278a 2015-11-22 19:30:03,078 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - sessionManager=org.mortbay.jetty.servlet.HashSessionManager@536027b9 2015-11-22 19:30:03,078 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - session=null 2015-11-22 19:30:03,079 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - servlet=org.mortbay.jetty.servlet.DefaultServlet-1061138277 2015-11-22 19:30:03,079 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - chain=NoCacheFilter->NoCacheFilter->safety->Timeline Authentication Filter->RMAuthenticationFilter->static_user_filter->guice->org.mortbay.jetty.servlet.DefaultServlet-1061138277 2015-11-22 19:30:03,079 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - servlet holder=org.mortbay.jetty.servlet.DefaultServlet-1061138277 2015-11-22 19:30:03,079 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter NoCacheFilter 2015-11-22 19:30:03,079 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter NoCacheFilter 2015-11-22 19:30:03,079 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter safety 2015-11-22 19:30:03,079 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter Timeline Authentication Filter 2015-11-22 19:30:03,080 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/?user.name=ubuntu] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(557)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:49959/ws/v1/timeline/?user.name=ubuntu] triggering authentication 2015-11-22 19:30:03,080 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/?user.name=ubuntu] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(576)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:49959/ws/v1/timeline/?user.name=ubuntu] user [ubuntu] authenticated 2015-11-22 19:30:03,080 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter RMAuthenticationFilter 2015-11-22 19:30:03,080 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/?user.name=ubuntu] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(557)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:49959/ws/v1/timeline/?user.name=ubuntu] triggering authentication 2015-11-22 19:30:03,080 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/?user.name=ubuntu] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(576)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:49959/ws/v1/timeline/?user.name=ubuntu] user [ubuntu] authenticated 2015-11-22 19:30:03,081 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter static_user_filter 2015-11-22 19:30:03,081 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter guice 2015-11-22 19:30:03,086 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - getResource(META-INF/services/javax.xml.bind.JAXBContext)=jar:file:/home/ubuntu/.m2/repository/com/sun/xml/bind/jaxb-impl/2.2.3-1/jaxb-impl-2.2.3-1.jar!/META-INF/services/javax.xml.bind.JAXBContext 2015-11-22 19:30:03,087 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.xml.bind.v2.ContextFactory from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:30:03,126 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - RESPONSE /ws/v1/timeline/ 200 2015-11-22 19:30:03,420 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - REQUEST /ws/v1/timeline/ on org.mortbay.jetty.HttpConnection@92bc3e6 2015-11-22 19:30:03,420 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - sessionManager=org.mortbay.jetty.servlet.HashSessionManager@536027b9 2015-11-22 19:30:03,420 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - session=null 2015-11-22 19:30:03,421 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - servlet=org.mortbay.jetty.servlet.DefaultServlet-1061138277 2015-11-22 19:30:03,421 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - chain=NoCacheFilter->NoCacheFilter->safety->Timeline Authentication Filter->RMAuthenticationFilter->static_user_filter->guice->org.mortbay.jetty.servlet.DefaultServlet-1061138277 2015-11-22 19:30:03,421 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - servlet holder=org.mortbay.jetty.servlet.DefaultServlet-1061138277 2015-11-22 19:30:03,421 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter NoCacheFilter 2015-11-22 19:30:03,421 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter NoCacheFilter 2015-11-22 19:30:03,421 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter safety 2015-11-22 19:30:03,422 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter Timeline Authentication Filter 2015-11-22 19:30:03,422 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(576)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:49959/ws/v1/timeline/] user [ubuntu] authenticated 2015-11-22 19:30:03,422 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter RMAuthenticationFilter 2015-11-22 19:30:03,422 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(576)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:49959/ws/v1/timeline/] user [ubuntu] authenticated 2015-11-22 19:30:03,423 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter static_user_filter 2015-11-22 19:30:03,423 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter guice 2015-11-22 19:30:03,433 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/] security.TimelineACLsManager (TimelineACLsManager.java:checkAccess(106)) - Verifying the access of ubuntu on the timeline entity { id: appattempt_1448220594280_0001_000001, type: DS_APP_ATTEMPT } 2015-11-22 19:30:03,436 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - RESPONSE /ws/v1/timeline/ 200 2015-11-22 19:30:03,456 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:38037 from ubuntu sending #951 2015-11-22 19:30:03,456 DEBUG [Socket Reader #1 for port 38037] ipc.Server (Server.java:processOneRpc(1897)) - got #951 2015-11-22 19:30:03,456 DEBUG [IPC Server handler 15 on 38037] ipc.Server (Server.java:run(2185)) - IPC Server handler 15 on 38037: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:35310 Call#951 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:30:03,456 DEBUG [IPC Server handler 15 on 38037] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:30:03,456 DEBUG [IPC Server handler 15 on 38037] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220594280_0001 owned by ubuntu 2015-11-22 19:30:03,457 DEBUG [IPC Server handler 15 on 38037] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 1 2015-11-22 19:30:03,457 DEBUG [IPC Server handler 15 on 38037] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 15 on 38037: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:35310 Call#951 Retry#0 2015-11-22 19:30:03,457 DEBUG [IPC Server handler 15 on 38037] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 15 on 38037: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:35310 Call#951 Retry#0 Wrote 333 bytes. 2015-11-22 19:30:03,457 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:38037 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:38037 from ubuntu got value #951 2015-11-22 19:30:03,457 DEBUG [Thread-4126] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:30:03,458 INFO [Thread-4126] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=N/A, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220600444, yarnAppState=ACCEPTED, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40017/proxy/application_1448220594280_0001/, appUser=ubuntu 2015-11-22 19:30:03,572 DEBUG [1957145000@qtp-248140027-0] mortbay.log (Slf4jLog.java:debug(40)) - EOF 2015-11-22 19:30:03,575 DEBUG [IPC Server listener on 53664] ipc.Server (Server.java:register(2883)) - Server connection from 172.31.20.42:57589; # active connections: 1; # queued calls: 0 2015-11-22 19:30:03,586 DEBUG [Socket Reader #1 for port 53664] ipc.Server (Server.java:processOneRpc(1897)) - got #-33 2015-11-22 19:30:03,586 DEBUG [Socket Reader #1 for port 53664] security.SaslRpcServer (SaslRpcServer.java:create(175)) - Created SASL server with mechanism = DIGEST-MD5 2015-11-22 19:30:03,587 DEBUG [Socket Reader #1 for port 53664] ipc.Server (Server.java:processResponse(1099)) - Socket Reader #1 for port 53664: responding to null from 172.31.20.42:57589 Call#-33 Retry#-1 2015-11-22 19:30:03,587 DEBUG [Socket Reader #1 for port 53664] ipc.Server (Server.java:processResponse(1118)) - Socket Reader #1 for port 53664: responding to null from 172.31.20.42:57589 Call#-33 Retry#-1 Wrote 166 bytes. 2015-11-22 19:30:03,605 DEBUG [Socket Reader #1 for port 53664] ipc.Server (Server.java:processOneRpc(1897)) - got #-33 2015-11-22 19:30:03,606 DEBUG [Socket Reader #1 for port 53664] ipc.Server (Server.java:processSaslToken(1529)) - Have read input token of size 274 for processing by saslServer.evaluateResponse() 2015-11-22 19:30:03,606 DEBUG [Socket Reader #1 for port 53664] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:retrievePassword(247)) - Trying to retrieve password for appattempt_1448220594280_0001_000001 2015-11-22 19:30:03,606 DEBUG [Socket Reader #1 for port 53664] security.SaslRpcServer (SaslRpcServer.java:handle(310)) - SASL server DIGEST-MD5 callback: setting password for client: appattempt_1448220594280_0001_000001 (auth:SIMPLE) 2015-11-22 19:30:03,606 DEBUG [Socket Reader #1 for port 53664] security.SaslRpcServer (SaslRpcServer.java:handle(328)) - SASL server DIGEST-MD5 callback: setting canonicalized client ID: appattempt_1448220594280_0001_000001 2015-11-22 19:30:03,606 DEBUG [Socket Reader #1 for port 53664] ipc.Server (Server.java:buildSaslResponse(1546)) - Will send SUCCESS token of size 40 from saslServer. 2015-11-22 19:30:03,607 DEBUG [Socket Reader #1 for port 53664] ipc.Server (Server.java:saslProcess(1434)) - SASL server context established. Negotiated QoP is auth 2015-11-22 19:30:03,607 DEBUG [Socket Reader #1 for port 53664] ipc.Server (Server.java:saslProcess(1439)) - SASL server successfully authenticated client: appattempt_1448220594280_0001_000001 (auth:SIMPLE) 2015-11-22 19:30:03,607 INFO [Socket Reader #1 for port 53664] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220594280_0001_000001 (auth:SIMPLE) 2015-11-22 19:30:03,607 DEBUG [Socket Reader #1 for port 53664] ipc.Server (Server.java:processResponse(1099)) - Socket Reader #1 for port 53664: responding to null from 172.31.20.42:57589 Call#-33 Retry#-1 2015-11-22 19:30:03,607 DEBUG [Socket Reader #1 for port 53664] ipc.Server (Server.java:processResponse(1118)) - Socket Reader #1 for port 53664: responding to null from 172.31.20.42:57589 Call#-33 Retry#-1 Wrote 64 bytes. 2015-11-22 19:30:03,613 DEBUG [Socket Reader #1 for port 53664] ipc.Server (Server.java:processOneRpc(1897)) - got #-3 2015-11-22 19:30:03,614 DEBUG [Socket Reader #1 for port 53664] ipc.Server (Server.java:authorizeConnection(2097)) - Successfully authorized userInfo { } protocol: "org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB" 2015-11-22 19:30:03,614 DEBUG [Socket Reader #1 for port 53664] ipc.Server (Server.java:processOneRpc(1897)) - got #0 2015-11-22 19:30:03,614 DEBUG [IPC Server handler 1 on 53664] ipc.Server (Server.java:run(2185)) - IPC Server handler 1 on 53664: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.registerApplicationMaster from 172.31.20.42:57589 Call#0 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:30:03,615 DEBUG [IPC Server handler 1 on 53664] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220594280_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:30:03,615 INFO [IPC Server handler 1 on 53664] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:registerApplicationMaster(220)) - AM registration appattempt_1448220594280_0001_000001 2015-11-22 19:30:03,615 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptRegistrationEvent.EventType: REGISTERED 2015-11-22 19:30:03,615 INFO [IPC Server handler 1 on 53664] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(133)) - USER=ubuntu IP=172.31.20.42 OPERATION=Register App Master TARGET=ApplicationMasterService RESULT=SUCCESS APPID=application_1448220594280_0001 APPATTEMPTID=appattempt_1448220594280_0001_000001 2015-11-22 19:30:03,615 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220594280_0001_000001 of type REGISTERED 2015-11-22 19:30:03,615 DEBUG [IPC Server handler 1 on 53664] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: registerApplicationMaster queueTime= 1 procesingTime= 0 2015-11-22 19:30:03,616 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220594280_0001_000001 State change from LAUNCHED to RUNNING 2015-11-22 19:30:03,618 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEvent.EventType: ATTEMPT_REGISTERED 2015-11-22 19:30:03,618 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(777)) - Processing event for application_1448220594280_0001 of type ATTEMPT_REGISTERED 2015-11-22 19:30:03,619 DEBUG [IPC Server handler 1 on 53664] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 1 on 53664: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.registerApplicationMaster from 172.31.20.42:57589 Call#0 Retry#0 2015-11-22 19:30:03,619 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220594280_0001 State change from ACCEPTED to RUNNING 2015-11-22 19:30:03,619 DEBUG [IPC Server handler 1 on 53664] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 1 on 53664: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.registerApplicationMaster from 172.31.20.42:57589 Call#0 Retry#0 Wrote 50 bytes. 2015-11-22 19:30:03,645 DEBUG [Socket Reader #1 for port 53664] ipc.Server (Server.java:processOneRpc(1897)) - got #1 2015-11-22 19:30:03,645 DEBUG [IPC Server handler 0 on 53664] ipc.Server (Server.java:run(2185)) - IPC Server handler 0 on 53664: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:57589 Call#1 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:30:03,645 DEBUG [IPC Server handler 0 on 53664] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220594280_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:30:03,646 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:30:03,646 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220594280_0001_000001 of type STATUS_UPDATE 2015-11-22 19:30:03,646 DEBUG [IPC Server handler 0 on 53664] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:30:03,646 DEBUG [IPC Server handler 0 on 53664] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 1 2015-11-22 19:30:03,647 DEBUG [IPC Server handler 0 on 53664] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 0 on 53664: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:57589 Call#1 Retry#0 2015-11-22 19:30:03,647 DEBUG [IPC Server handler 0 on 53664] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 0 on 53664: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:57589 Call#1 Retry#0 Wrote 47 bytes. 2015-11-22 19:30:04,019 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:30:04,019 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220594280_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:30:04,020 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:30:04,020 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:35735 of type STATUS_UPDATE 2015-11-22 19:30:04,020 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:30:04,020 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:35735 clusterResources: 2015-11-22 19:30:04,020 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:35735 availableResource: 2015-11-22 19:30:04,020 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:30:04,020 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:30:04,293 DEBUG [IPC Server idle connection scanner for port 50902] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 50902: task running 2015-11-22 19:30:04,307 DEBUG [IPC Server idle connection scanner for port 53664] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 53664: task running 2015-11-22 19:30:04,322 DEBUG [IPC Server idle connection scanner for port 38037] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 38037: task running 2015-11-22 19:30:04,458 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:38037 from ubuntu sending #952 2015-11-22 19:30:04,458 DEBUG [Socket Reader #1 for port 38037] ipc.Server (Server.java:processOneRpc(1897)) - got #952 2015-11-22 19:30:04,458 DEBUG [IPC Server handler 13 on 38037] ipc.Server (Server.java:run(2185)) - IPC Server handler 13 on 38037: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:35310 Call#952 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:30:04,459 DEBUG [IPC Server handler 13 on 38037] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:30:04,459 DEBUG [IPC Server handler 13 on 38037] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220594280_0001 owned by ubuntu 2015-11-22 19:30:04,459 DEBUG [IPC Server handler 13 on 38037] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 1 procesingTime= 0 2015-11-22 19:30:04,459 DEBUG [IPC Server handler 13 on 38037] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 13 on 38037: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:35310 Call#952 Retry#0 2015-11-22 19:30:04,459 DEBUG [IPC Server handler 13 on 38037] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 13 on 38037: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:35310 Call#952 Retry#0 Wrote 359 bytes. 2015-11-22 19:30:04,459 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:38037 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:38037 from ubuntu got value #952 2015-11-22 19:30:04,460 DEBUG [Thread-4126] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:30:04,460 INFO [Thread-4126] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220600444, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40017/proxy/application_1448220594280_0001/, appUser=ubuntu 2015-11-22 19:30:04,533 DEBUG [IPC Server idle connection scanner for port 38415] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 38415: task running 2015-11-22 19:30:04,662 DEBUG [Socket Reader #1 for port 53664] ipc.Server (Server.java:processOneRpc(1897)) - got #2 2015-11-22 19:30:04,662 DEBUG [IPC Server handler 9 on 53664] ipc.Server (Server.java:run(2185)) - IPC Server handler 9 on 53664: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:57589 Call#2 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:30:04,662 DEBUG [IPC Server handler 9 on 53664] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220594280_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:30:04,663 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:30:04,663 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220594280_0001_000001 of type STATUS_UPDATE 2015-11-22 19:30:04,663 DEBUG [IPC Server handler 9 on 53664] capacity.CapacityScheduler (CapacityScheduler.java:allocate(941)) - allocate: pre-update appattempt_1448220594280_0001_000001 ask size =1 2015-11-22 19:30:04,663 DEBUG [IPC Server handler 9 on 53664] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:30:04,663 DEBUG [IPC Server handler 9 on 53664] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(470)) - showRequests: application=application_1448220594280_0001 headRoom= currentConsumption=512 2015-11-22 19:30:04,663 DEBUG [IPC Server handler 9 on 53664] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(474)) - showRequests: application=application_1448220594280_0001 request={Priority: 0, Capability: , # Containers: 0, Location: *, Relax Locality: true, Node Label Expression: } 2015-11-22 19:30:04,663 DEBUG [IPC Server handler 9 on 53664] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:updateResourceRequests(320)) - update: application=application_1448220594280_0001 request={Priority: 0, Capability: , # Containers: 2, Location: *, Relax Locality: true, Node Label Expression: } 2015-11-22 19:30:04,663 DEBUG [IPC Server handler 9 on 53664] scheduler.ActiveUsersManager (ActiveUsersManager.java:activateApplication(68)) - User ubuntu added to activeUsers, currently: 1 2015-11-22 19:30:04,664 DEBUG [IPC Server handler 9 on 53664] capacity.CapacityScheduler (CapacityScheduler.java:allocate(952)) - allocate: post-update 2015-11-22 19:30:04,664 DEBUG [IPC Server handler 9 on 53664] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2015-11-22 19:30:04,664 DEBUG [IPC Server handler 9 on 53664] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(470)) - showRequests: application=application_1448220594280_0001 headRoom= currentConsumption=512 2015-11-22 19:30:04,664 DEBUG [IPC Server handler 9 on 53664] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(474)) - showRequests: application=application_1448220594280_0001 request={Priority: 0, Capability: , # Containers: 2, Location: *, Relax Locality: true, Node Label Expression: } 2015-11-22 19:30:04,665 DEBUG [IPC Server handler 9 on 53664] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2015-11-22 19:30:04,665 DEBUG [IPC Server handler 9 on 53664] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 3 2015-11-22 19:30:04,665 DEBUG [IPC Server handler 9 on 53664] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 9 on 53664: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:57589 Call#2 Retry#0 2015-11-22 19:30:04,666 DEBUG [IPC Server handler 9 on 53664] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 9 on 53664: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:57589 Call#2 Retry#0 Wrote 47 bytes. 2015-11-22 19:30:05,020 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:30:05,020 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220594280_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:30:05,020 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:30:05,020 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:35735 of type STATUS_UPDATE 2015-11-22 19:30:05,020 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:30:05,021 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:35735 clusterResources: 2015-11-22 19:30:05,021 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:35735 availableResource: 2015-11-22 19:30:05,021 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:30:05,021 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(408)) - Trying to assign containers to child-queue of root 2015-11-22 19:30:05,021 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:printChildQueues(617)) - printChildQueues - queue: root child-queues: root.defaultusedCapacity=(0.125), label=(*) 2015-11-22 19:30:05,021 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(564)) - Trying to assign to queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.125, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 2015-11-22 19:30:05,021 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:assignContainers(817)) - assignContainers: node=ip-172-31-20-42.ap-northeast-1.compute.internal #applications=1 2015-11-22 19:30:05,021 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2015-11-22 19:30:05,021 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimitAndSetHeadroom(984)) - Headroom calculation for user ubuntu: userLimit= queueMaxAvailRes= consumed= headroom= 2015-11-22 19:30:05,021 DEBUG [ResourceManager Event Processor] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:assignContainers(477)) - pre-assignContainers for application application_1448220594280_0001 2015-11-22 19:30:05,021 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2015-11-22 19:30:05,021 DEBUG [ResourceManager Event Processor] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(470)) - showRequests: application=application_1448220594280_0001 headRoom= currentConsumption=512 2015-11-22 19:30:05,022 DEBUG [ResourceManager Event Processor] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(474)) - showRequests: application=application_1448220594280_0001 request={Priority: 0, Capability: , # Containers: 2, Location: *, Relax Locality: true, Node Label Expression: } 2015-11-22 19:30:05,022 DEBUG [ResourceManager Event Processor] allocator.IncreaseContainerAllocator (IncreaseContainerAllocator.java:assignContainers(184)) - Skip allocating increase request since we don't have any increase request on this node=ip-172-31-20-42.ap-northeast-1.compute.internal:35735 2015-11-22 19:30:05,022 DEBUG [ResourceManager Event Processor] allocator.RegularContainerAllocator (RegularContainerAllocator.java:assignContainer(407)) - assignContainers: node=ip-172-31-20-42.ap-northeast-1.compute.internal application=application_1448220594280_0001 priority=0 request={Priority: 0, Capability: , # Containers: 2, Location: *, Relax Locality: true, Node Label Expression: } type=OFF_SWITCH 2015-11-22 19:30:05,023 DEBUG [ResourceManager Event Processor] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:allocate(594)) - allocate: applicationId=application_1448220594280_0001 container=container_1448220594280_0001_01_000002 host=ip-172-31-20-42.ap-northeast-1.compute.internal:35735 user=ubuntu resource= type=OFF_SWITCH 2015-11-22 19:30:05,023 DEBUG [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(424)) - Processing container_1448220594280_0001_01_000002 of type START 2015-11-22 19:30:05,023 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220594280_0001_01_000002 Container Transitioned from NEW to ALLOCATED 2015-11-22 19:30:05,023 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: CONTAINER_ALLOCATED 2015-11-22 19:30:05,023 DEBUG [ResourceManager Event Processor] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:allocate(208)) - allocate: applicationAttemptId=appattempt_1448220594280_0001_000001 container=container_1448220594280_0001_01_000002 host=ip-172-31-20-42.ap-northeast-1.compute.internal type=OFF_SWITCH 2015-11-22 19:30:05,023 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220594280_0001_000001 of type CONTAINER_ALLOCATED 2015-11-22 19:30:05,023 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220594280_0001 CONTAINERID=container_1448220594280_0001_01_000002 2015-11-22 19:30:05,023 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(154)) - Assigned container container_1448220594280_0001_01_000002 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:35735, which has 2 containers, used and available after allocation 2015-11-22 19:30:05,023 INFO [ResourceManager Event Processor] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(89)) - assignedContainer application attempt=appattempt_1448220594280_0001_000001 container=container_1448220594280_0001_01_000002 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@5ecb869b clusterResource= type=OFF_SWITCH 2015-11-22 19:30:05,023 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:assignContainers(881)) - post-assignContainers for application application_1448220594280_0001 2015-11-22 19:30:05,024 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2015-11-22 19:30:05,024 DEBUG [ResourceManager Event Processor] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(470)) - showRequests: application=application_1448220594280_0001 headRoom= currentConsumption=640 2015-11-22 19:30:05,024 DEBUG [ResourceManager Event Processor] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(474)) - showRequests: application=application_1448220594280_0001 request={Priority: 0, Capability: , # Containers: 1, Location: *, Relax Locality: true, Node Label Expression: } 2015-11-22 19:30:05,024 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2015-11-22 19:30:05,024 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2015-11-22 19:30:05,024 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2015-11-22 19:30:05,024 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:allocateResource(1277)) - default user=ubuntu used= numContainers=2 headroom = user-resources= 2015-11-22 19:30:05,024 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(575)) - Assigned to queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.15625, absoluteUsedCapacity=0.15625, numApps=1, numContainers=2 --> , OFF_SWITCH 2015-11-22 19:30:05,024 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(589)) - Re-sorting assigned queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.15625, absoluteUsedCapacity=0.15625, numApps=1, numContainers=2 2015-11-22 19:30:05,024 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:printChildQueues(617)) - printChildQueues - queue: root child-queues: root.defaultusedCapacity=(0.15625), label=(*) 2015-11-22 19:30:05,025 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(461)) - assignedContainer queue=root usedCapacity=0.15625 absoluteUsedCapacity=0.15625 used= cluster= 2015-11-22 19:30:05,025 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(473)) - ParentQ=root assignedSoFarInThisIteration= usedCapacity=0.15625 absoluteUsedCapacity=0.15625 2015-11-22 19:30:05,025 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(484)) - Not assigning more than one off-switch container, assignments so far: resource:; type:OFF_SWITCH; excessReservation:null; applicationid:null; skipped:false; fulfilled reservation:false; allocations(count/resource):1/; reservations(count/resource):0/ 2015-11-22 19:30:05,460 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:38037 from ubuntu sending #953 2015-11-22 19:30:05,460 DEBUG [Socket Reader #1 for port 38037] ipc.Server (Server.java:processOneRpc(1897)) - got #953 2015-11-22 19:30:05,460 DEBUG [IPC Server handler 9 on 38037] ipc.Server (Server.java:run(2185)) - IPC Server handler 9 on 38037: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:35310 Call#953 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:30:05,461 DEBUG [IPC Server handler 9 on 38037] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:30:05,461 DEBUG [IPC Server handler 9 on 38037] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220594280_0001 owned by ubuntu 2015-11-22 19:30:05,461 DEBUG [IPC Server handler 9 on 38037] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 1 procesingTime= 0 2015-11-22 19:30:05,461 DEBUG [IPC Server handler 9 on 38037] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 9 on 38037: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:35310 Call#953 Retry#0 2015-11-22 19:30:05,462 DEBUG [IPC Server handler 9 on 38037] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 9 on 38037: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:35310 Call#953 Retry#0 Wrote 359 bytes. 2015-11-22 19:30:05,462 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:38037 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:38037 from ubuntu got value #953 2015-11-22 19:30:05,462 DEBUG [Thread-4126] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:30:05,462 INFO [Thread-4126] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220600444, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40017/proxy/application_1448220594280_0001/, appUser=ubuntu 2015-11-22 19:30:05,666 DEBUG [Socket Reader #1 for port 53664] ipc.Server (Server.java:processOneRpc(1897)) - got #3 2015-11-22 19:30:05,667 DEBUG [IPC Server handler 4 on 53664] ipc.Server (Server.java:run(2185)) - IPC Server handler 4 on 53664: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:57589 Call#3 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:30:05,667 DEBUG [IPC Server handler 4 on 53664] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220594280_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:30:05,667 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:30:05,667 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220594280_0001_000001 of type STATUS_UPDATE 2015-11-22 19:30:05,667 DEBUG [IPC Server handler 4 on 53664] security.BaseContainerTokenSecretManager (BaseContainerTokenSecretManager.java:createPassword(90)) - Creating password for container_1448220594280_0001_01_000002 for user container_1448220594280_0001_01_000002 (auth:SIMPLE) to be run on NM ip-172-31-20-42.ap-northeast-1.compute.internal:35735 2015-11-22 19:30:05,667 DEBUG [IPC Server handler 4 on 53664] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(197)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 1 cluster_timestamp: 1448220594280 } attemptId: 1 } id: 2 } nmHostAddr: "ip-172-31-20-42.ap-northeast-1.compute.internal:35735" appSubmitter: "ubuntu" resource { memory: 128 virtual_cores: 1 } expiryTimeStamp: 1448221205667 masterKeyId: -368252818 rmIdentifier: 1448220594280 priority { priority: 0 } creationTime: 1448220605022 nodeLabelExpression: "" containerType: TASK 2015-11-22 19:30:05,668 DEBUG [IPC Server handler 4 on 53664] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(197)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 1 cluster_timestamp: 1448220594280 } attemptId: 1 } id: 2 } nmHostAddr: "ip-172-31-20-42.ap-northeast-1.compute.internal:35735" appSubmitter: "ubuntu" resource { memory: 128 virtual_cores: 1 } expiryTimeStamp: 1448221205667 masterKeyId: -368252818 rmIdentifier: 1448220594280 priority { priority: 0 } creationTime: 1448220605022 nodeLabelExpression: "" containerType: TASK 2015-11-22 19:30:05,668 INFO [IPC Server handler 4 on 53664] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:createAndGetNMToken(200)) - Sending NMToken for nodeId : ip-172-31-20-42.ap-northeast-1.compute.internal:35735 for container : container_1448220594280_0001_01_000002 2015-11-22 19:30:05,668 DEBUG [IPC Server handler 4 on 53664] security.BaseNMTokenSecretManager (BaseNMTokenSecretManager.java:createPassword(75)) - creating password for appattempt_1448220594280_0001_000001 for user ubuntu to run on NM ip-172-31-20-42.ap-northeast-1.compute.internal:35735 2015-11-22 19:30:05,668 DEBUG [IPC Server handler 4 on 53664] security.NMTokenIdentifier (NMTokenIdentifier.java:write(96)) - Writing NMTokenIdentifier to RPC layer: appAttemptId { application_id { id: 1 cluster_timestamp: 1448220594280 } attemptId: 1 } nodeId { host: "ip-172-31-20-42.ap-northeast-1.compute.internal" port: 35735 } appSubmitter: "ubuntu" keyId: -1658271729 2015-11-22 19:30:05,668 DEBUG [IPC Server handler 4 on 53664] security.NMTokenIdentifier (NMTokenIdentifier.java:write(96)) - Writing NMTokenIdentifier to RPC layer: appAttemptId { application_id { id: 1 cluster_timestamp: 1448220594280 } attemptId: 1 } nodeId { host: "ip-172-31-20-42.ap-northeast-1.compute.internal" port: 35735 } appSubmitter: "ubuntu" keyId: -1658271729 2015-11-22 19:30:05,668 DEBUG [IPC Server handler 4 on 53664] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(424)) - Processing container_1448220594280_0001_01_000002 of type ACQUIRED 2015-11-22 19:30:05,669 INFO [IPC Server handler 4 on 53664] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220594280_0001_01_000002 Container Transitioned from ALLOCATED to ACQUIRED 2015-11-22 19:30:05,669 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppRunningOnNodeEvent.EventType: APP_RUNNING_ON_NODE 2015-11-22 19:30:05,669 DEBUG [IPC Server handler 4 on 53664] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2015-11-22 19:30:05,669 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(777)) - Processing event for application_1448220594280_0001 of type APP_RUNNING_ON_NODE 2015-11-22 19:30:05,670 DEBUG [IPC Server handler 4 on 53664] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 3 2015-11-22 19:30:05,670 DEBUG [IPC Server handler 4 on 53664] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 4 on 53664: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:57589 Call#3 Retry#0 2015-11-22 19:30:05,670 DEBUG [IPC Server handler 4 on 53664] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 4 on 53664: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:57589 Call#3 Retry#0 Wrote 587 bytes. 2015-11-22 19:30:05,728 DEBUG [IPC Server listener on 35735] ipc.Server (Server.java:register(2883)) - Server connection from 172.31.20.42:47745; # active connections: 2; # queued calls: 0 2015-11-22 19:30:05,728 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:processOneRpc(1897)) - got #-33 2015-11-22 19:30:05,728 DEBUG [Socket Reader #1 for port 35735] security.SaslRpcServer (SaslRpcServer.java:create(175)) - Created SASL server with mechanism = DIGEST-MD5 2015-11-22 19:30:05,728 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:processResponse(1099)) - Socket Reader #1 for port 35735: responding to null from 172.31.20.42:47745 Call#-33 Retry#-1 2015-11-22 19:30:05,728 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:processResponse(1118)) - Socket Reader #1 for port 35735: responding to null from 172.31.20.42:47745 Call#-33 Retry#-1 Wrote 166 bytes. 2015-11-22 19:30:05,731 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:processOneRpc(1897)) - got #-33 2015-11-22 19:30:05,731 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:processSaslToken(1529)) - Have read input token of size 358 for processing by saslServer.evaluateResponse() 2015-11-22 19:30:05,731 DEBUG [Socket Reader #1 for port 35735] security.BaseNMTokenSecretManager (BaseNMTokenSecretManager.java:retrivePasswordInternal(103)) - creating password for appattempt_1448220594280_0001_000001 for user ubuntu to run on NM ip-172-31-20-42.ap-northeast-1.compute.internal:35735 2015-11-22 19:30:05,732 DEBUG [Socket Reader #1 for port 35735] security.NMTokenIdentifier (NMTokenIdentifier.java:write(96)) - Writing NMTokenIdentifier to RPC layer: appAttemptId { application_id { id: 1 cluster_timestamp: 1448220594280 } attemptId: 1 } nodeId { host: "ip-172-31-20-42.ap-northeast-1.compute.internal" port: 35735 } appSubmitter: "ubuntu" keyId: -1658271729 2015-11-22 19:30:05,732 DEBUG [Socket Reader #1 for port 35735] security.NMTokenSecretManagerInNM (NMTokenSecretManagerInNM.java:retrievePassword(180)) - NMToken password retrieved successfully!! 2015-11-22 19:30:05,732 DEBUG [Socket Reader #1 for port 35735] security.SaslRpcServer (SaslRpcServer.java:handle(310)) - SASL server DIGEST-MD5 callback: setting password for client: appattempt_1448220594280_0001_000001 (auth:SIMPLE) 2015-11-22 19:30:05,732 DEBUG [Socket Reader #1 for port 35735] security.SaslRpcServer (SaslRpcServer.java:handle(328)) - SASL server DIGEST-MD5 callback: setting canonicalized client ID: appattempt_1448220594280_0001_000001 2015-11-22 19:30:05,732 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:buildSaslResponse(1546)) - Will send SUCCESS token of size 40 from saslServer. 2015-11-22 19:30:05,732 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:saslProcess(1434)) - SASL server context established. Negotiated QoP is auth 2015-11-22 19:30:05,732 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:saslProcess(1439)) - SASL server successfully authenticated client: appattempt_1448220594280_0001_000001 (auth:SIMPLE) 2015-11-22 19:30:05,733 INFO [Socket Reader #1 for port 35735] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220594280_0001_000001 (auth:SIMPLE) 2015-11-22 19:30:05,733 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:processResponse(1099)) - Socket Reader #1 for port 35735: responding to null from 172.31.20.42:47745 Call#-33 Retry#-1 2015-11-22 19:30:05,733 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:processResponse(1118)) - Socket Reader #1 for port 35735: responding to null from 172.31.20.42:47745 Call#-33 Retry#-1 Wrote 64 bytes. 2015-11-22 19:30:05,735 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:processOneRpc(1897)) - got #-3 2015-11-22 19:30:05,736 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:authorizeConnection(2097)) - Successfully authorized userInfo { } protocol: "org.apache.hadoop.yarn.api.ContainerManagementProtocolPB" 2015-11-22 19:30:05,736 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:processOneRpc(1897)) - got #4 2015-11-22 19:30:05,736 DEBUG [IPC Server handler 2 on 35735] ipc.Server (Server.java:run(2185)) - IPC Server handler 2 on 35735: org.apache.hadoop.yarn.api.ContainerManagementProtocolPB.startContainers from 172.31.20.42:47745 Call#4 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:30:05,737 DEBUG [IPC Server handler 2 on 35735] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220594280_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:30:05,737 DEBUG [IPC Server handler 2 on 35735] security.BaseContainerTokenSecretManager (BaseContainerTokenSecretManager.java:retrievePasswordInternal(118)) - Retrieving password for container_1448220594280_0001_01_000002 for user container_1448220594280_0001_01_000002 (auth:SIMPLE) to be run on NM ip-172-31-20-42.ap-northeast-1.compute.internal:35735 2015-11-22 19:30:05,738 DEBUG [IPC Server handler 2 on 35735] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(197)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 1 cluster_timestamp: 1448220594280 } attemptId: 1 } id: 2 } nmHostAddr: "ip-172-31-20-42.ap-northeast-1.compute.internal:35735" appSubmitter: "ubuntu" resource { memory: 128 virtual_cores: 1 } expiryTimeStamp: 1448221205667 masterKeyId: -368252818 rmIdentifier: 1448220594280 priority { priority: 0 } creationTime: 1448220605022 nodeLabelExpression: "" containerType: TASK 2015-11-22 19:30:05,738 INFO [IPC Server handler 2 on 35735] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:startContainerInternal(889)) - Start request for container_1448220594280_0001_01_000002 by user ubuntu 2015-11-22 19:30:05,739 DEBUG [IPC Server handler 2 on 35735] utils.YarnServerSecurityUtils (YarnServerSecurityUtils.java:parseCredentials(135)) - = Kind: YARN_AM_RM_TOKEN, Service: , Ident: (appAttemptId { application_id { id: 1 cluster_timestamp: 1448220594280 } attemptId: 1 } keyId: -720358617) 2015-11-22 19:30:05,739 INFO [IPC Server handler 2 on 35735] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu IP=172.31.20.42 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1448220594280_0001 CONTAINERID=container_1448220594280_0001_01_000002 2015-11-22 19:30:05,739 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationContainerInitEvent.EventType: INIT_CONTAINER 2015-11-22 19:30:05,740 DEBUG [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(452)) - Processing application_1448220594280_0001 of type INIT_CONTAINER 2015-11-22 19:30:05,740 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(303)) - Adding container_1448220594280_0001_01_000002 to application application_1448220594280_0001 2015-11-22 19:30:05,740 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerInitEvent.EventType: INIT_CONTAINER 2015-11-22 19:30:05,740 DEBUG [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1131)) - Processing container_1448220594280_0001_01_000002 of type INIT_CONTAINER 2015-11-22 19:30:05,740 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220594280_0001_01_000002 transitioned from NEW to LOCALIZED 2015-11-22 19:30:05,740 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServicesEvent.EventType: CONTAINER_INIT 2015-11-22 19:30:05,740 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_INIT for appId application_1448220594280_0001 2015-11-22 19:30:05,740 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncherEvent.EventType: LAUNCH_CONTAINER 2015-11-22 19:30:05,739 DEBUG [IPC Server handler 2 on 35735] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: startContainers queueTime= 1 procesingTime= 2 2015-11-22 19:30:05,744 DEBUG [IPC Server handler 2 on 35735] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 2 on 35735: responding to org.apache.hadoop.yarn.api.ContainerManagementProtocolPB.startContainers from 172.31.20.42:47745 Call#4 Retry#0 2015-11-22 19:30:05,744 DEBUG [IPC Server handler 2 on 35735] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 2 on 35735: responding to org.apache.hadoop.yarn.api.ContainerManagementProtocolPB.startContainers from 172.31.20.42:47745 Call#4 Retry#0 Wrote 51 bytes. 2015-11-22 19:30:05,745 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:close(2894)) - Socket Reader #1 for port 35735: disconnecting client 172.31.20.42:47745. Number of active connections: 1 2015-11-22 19:30:05,745 DEBUG [ContainersLauncher #1] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.fs.FileContext.getAbstractFileSystem(FileContext.java:339) 2015-11-22 19:30:05,770 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerEvent.EventType: CONTAINER_LAUNCHED 2015-11-22 19:30:05,771 DEBUG [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1131)) - Processing container_1448220594280_0001_01_000002 of type CONTAINER_LAUNCHED 2015-11-22 19:30:05,771 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220594280_0001_01_000002 transitioned from LOCALIZED to RUNNING 2015-11-22 19:30:05,771 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerStartMonitoringEvent.EventType: START_MONITORING_CONTAINER 2015-11-22 19:30:05,790 INFO [ContainersLauncher #1] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:buildCommandExecutor(281)) - launchContainer: [nice, -n, 0, bash, /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220594280_0001/container_1448220594280_0001_01_000002/default_container_executor.sh] 2015-11-22 19:30:05,791 DEBUG [IPC Server idle connection scanner for port 35735] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 35735: task running 2015-11-22 19:30:05,800 DEBUG [ContainersLauncher #1] launcher.ContainerLaunch (ContainerLaunch.java:call(333)) - Container container_1448220594280_0001_01_000002 completed with exit code 0 2015-11-22 19:30:05,800 INFO [ContainersLauncher #1] launcher.ContainerLaunch (ContainerLaunch.java:call(356)) - Container container_1448220594280_0001_01_000002 succeeded 2015-11-22 19:30:05,800 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerEvent.EventType: CONTAINER_EXITED_WITH_SUCCESS 2015-11-22 19:30:05,801 DEBUG [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1131)) - Processing container_1448220594280_0001_01_000002 of type CONTAINER_EXITED_WITH_SUCCESS 2015-11-22 19:30:05,801 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220594280_0001_01_000002 transitioned from RUNNING to EXITED_WITH_SUCCESS 2015-11-22 19:30:05,801 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncherEvent.EventType: CLEANUP_CONTAINER 2015-11-22 19:30:05,801 INFO [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(380)) - Cleaning up container container_1448220594280_0001_01_000002 2015-11-22 19:30:05,801 DEBUG [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(397)) - Marking container container_1448220594280_0001_01_000002 as inactive 2015-11-22 19:30:05,801 DEBUG [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(403)) - Getting pid for container container_1448220594280_0001_01_000002 to kill from pid file /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/application_1448220594280_0001/container_1448220594280_0001_01_000002/container_1448220594280_0001_01_000002.pid 2015-11-22 19:30:05,801 DEBUG [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:getContainerPid(569)) - Accessing pid for container container_1448220594280_0001_01_000002 from pid file /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/application_1448220594280_0001/container_1448220594280_0001_01_000002/container_1448220594280_0001_01_000002.pid 2015-11-22 19:30:05,801 DEBUG [AsyncDispatcher event handler] util.ProcessIdFileReader (ProcessIdFileReader.java:getProcessId(51)) - Accessing pid from pid file /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/application_1448220594280_0001/container_1448220594280_0001_01_000002/container_1448220594280_0001_01_000002.pid 2015-11-22 19:30:05,801 DEBUG [AsyncDispatcher event handler] util.ProcessIdFileReader (ProcessIdFileReader.java:getProcessId(99)) - Got pid 30738 from path /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/application_1448220594280_0001/container_1448220594280_0001_01_000002/container_1448220594280_0001_01_000002.pid 2015-11-22 19:30:05,801 DEBUG [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:getContainerPid(579)) - Got pid 30738 for container container_1448220594280_0001_01_000002 2015-11-22 19:30:05,801 DEBUG [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(421)) - Sending signal to pid 30738 as user ubuntu for container container_1448220594280_0001_01_000002 2015-11-22 19:30:05,801 DEBUG [AsyncDispatcher event handler] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:signalContainer(417)) - Sending signal 15 to pid 30738 as user ubuntu 2015-11-22 19:30:05,804 DEBUG [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(437)) - Sent signal SIGTERM to pid 30738 as user ubuntu for container container_1448220594280_0001_01_000002, result=failed 2015-11-22 19:30:05,804 DEBUG [AsyncDispatcher event handler] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.fs.FileContext.getAbstractFileSystem(FileContext.java:339) 2015-11-22 19:30:05,809 DEBUG [IPC Server idle connection scanner for port 40868] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 40868: task running 2015-11-22 19:30:05,815 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.ContainerLocalizationCleanupEvent.EventType: CLEANUP_CONTAINER_RESOURCES 2015-11-22 19:30:05,815 DEBUG [DeletionService #0] nodemanager.DeletionService (DeletionService.java:run(266)) - FileDeletionTask : user : ubuntu subDir : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220594280_0001/container_1448220594280_0001_01_000002 baseDir : null 2015-11-22 19:30:05,816 DEBUG [DeletionService #0] nodemanager.DeletionService (DeletionService.java:run(292)) - Deleting path: [/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220594280_0001/container_1448220594280_0001_01_000002] as user: [ubuntu] 2015-11-22 19:30:05,816 INFO [DeletionService #0] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(480)) - Deleting absolute path : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220594280_0001/container_1448220594280_0001_01_000002 2015-11-22 19:30:05,816 DEBUG [DeletionService #1] nodemanager.DeletionService (DeletionService.java:run(266)) - FileDeletionTask : user : null subDir : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/application_1448220594280_0001/container_1448220594280_0001_01_000002 baseDir : null 2015-11-22 19:30:05,816 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerEvent.EventType: CONTAINER_RESOURCES_CLEANEDUP 2015-11-22 19:30:05,816 DEBUG [DeletionService #1] nodemanager.DeletionService (DeletionService.java:run(271)) - NM deleting absolute path : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/application_1448220594280_0001/container_1448220594280_0001_01_000002 2015-11-22 19:30:05,816 DEBUG [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1131)) - Processing container_1448220594280_0001_01_000002 of type CONTAINER_RESOURCES_CLEANEDUP 2015-11-22 19:30:05,816 INFO [AsyncDispatcher event handler] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu OPERATION=Container Finished - Succeeded TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1448220594280_0001 CONTAINERID=container_1448220594280_0001_01_000002 2015-11-22 19:30:05,816 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220594280_0001_01_000002 transitioned from EXITED_WITH_SUCCESS to DONE 2015-11-22 19:30:05,816 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationContainerFinishedEvent.EventType: APPLICATION_CONTAINER_FINISHED 2015-11-22 19:30:05,816 DEBUG [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(452)) - Processing application_1448220594280_0001 of type APPLICATION_CONTAINER_FINISHED 2015-11-22 19:30:05,817 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(346)) - Removing container_1448220594280_0001_01_000002 from application application_1448220594280_0001 2015-11-22 19:30:05,817 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerStopMonitoringEvent.EventType: STOP_MONITORING_CONTAINER 2015-11-22 19:30:05,817 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.event.LogHandlerContainerFinishedEvent.EventType: CONTAINER_FINISHED 2015-11-22 19:30:05,817 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServicesEvent.EventType: CONTAINER_STOP 2015-11-22 19:30:05,817 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_STOP for appId application_1448220594280_0001 2015-11-22 19:30:06,020 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:30:06,021 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 2 container statuses: [ContainerStatus: [ContainerId: container_1448220594280_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ], ContainerStatus: [ContainerId: container_1448220594280_0001_01_000002, State: COMPLETE, Capability: , Diagnostics: , ExitStatus: 0, ]] 2015-11-22 19:30:06,021 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:30:06,021 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:35735 of type STATUS_UPDATE 2015-11-22 19:30:06,021 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:30:06,021 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:35735 clusterResources: 2015-11-22 19:30:06,021 DEBUG [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(424)) - Processing container_1448220594280_0001_01_000002 of type FINISHED 2015-11-22 19:30:06,022 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220594280_0001_01_000002 Container Transitioned from ACQUIRED to COMPLETED 2015-11-22 19:30:06,022 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220594280_0001 CONTAINERID=container_1448220594280_0001_01_000002 2015-11-22 19:30:06,022 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:releaseContainer(248)) - Released container container_1448220594280_0001_01_000002 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:35735, which currently has 1 containers, used and available, release resources=true 2015-11-22 19:30:06,022 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2015-11-22 19:30:06,022 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:releaseResource(1313)) - default used= numContainers=1 user=ubuntu user-resources= 2015-11-22 19:30:06,022 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:internalReleaseResource(630)) - completedContainer root: numChildQueue= 1, capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.125, numApps=1, numContainers=1, cluster= 2015-11-22 19:30:06,022 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:internalReleaseResource(643)) - Re-sorting completed queue: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.125, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 2015-11-22 19:30:06,022 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:35735 availableResource: 2015-11-22 19:30:06,022 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:30:06,022 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(408)) - Trying to assign containers to child-queue of root 2015-11-22 19:30:06,023 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:printChildQueues(617)) - printChildQueues - queue: root child-queues: root.defaultusedCapacity=(0.125), label=(*) 2015-11-22 19:30:06,023 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(564)) - Trying to assign to queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.125, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 2015-11-22 19:30:06,023 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:assignContainers(817)) - assignContainers: node=ip-172-31-20-42.ap-northeast-1.compute.internal #applications=1 2015-11-22 19:30:06,023 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2015-11-22 19:30:06,023 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimitAndSetHeadroom(984)) - Headroom calculation for user ubuntu: userLimit= queueMaxAvailRes= consumed= headroom= 2015-11-22 19:30:06,023 DEBUG [ResourceManager Event Processor] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:assignContainers(477)) - pre-assignContainers for application application_1448220594280_0001 2015-11-22 19:30:06,023 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 1 clusterCapacity: 2015-11-22 19:30:06,023 DEBUG [ResourceManager Event Processor] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(470)) - showRequests: application=application_1448220594280_0001 headRoom= currentConsumption=512 2015-11-22 19:30:06,023 DEBUG [ResourceManager Event Processor] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(474)) - showRequests: application=application_1448220594280_0001 request={Priority: 0, Capability: , # Containers: 1, Location: *, Relax Locality: true, Node Label Expression: } 2015-11-22 19:30:06,023 DEBUG [ResourceManager Event Processor] allocator.IncreaseContainerAllocator (IncreaseContainerAllocator.java:assignContainers(184)) - Skip allocating increase request since we don't have any increase request on this node=ip-172-31-20-42.ap-northeast-1.compute.internal:35735 2015-11-22 19:30:06,024 DEBUG [ResourceManager Event Processor] allocator.RegularContainerAllocator (RegularContainerAllocator.java:assignContainer(407)) - assignContainers: node=ip-172-31-20-42.ap-northeast-1.compute.internal application=application_1448220594280_0001 priority=0 request={Priority: 0, Capability: , # Containers: 1, Location: *, Relax Locality: true, Node Label Expression: } type=OFF_SWITCH 2015-11-22 19:30:06,024 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptContainerFinishedEvent.EventType: CONTAINER_FINISHED 2015-11-22 19:30:06,024 DEBUG [ResourceManager Event Processor] scheduler.ActiveUsersManager (ActiveUsersManager.java:deactivateApplication(94)) - User ubuntu removed from activeUsers, currently: 0 2015-11-22 19:30:06,024 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220594280_0001_000001 of type CONTAINER_FINISHED 2015-11-22 19:30:06,024 DEBUG [ResourceManager Event Processor] scheduler.AppSchedulingInfo (AppSchedulingInfo.java:allocate(594)) - allocate: applicationId=application_1448220594280_0001 container=container_1448220594280_0001_01_000003 host=ip-172-31-20-42.ap-northeast-1.compute.internal:35735 user=ubuntu resource= type=OFF_SWITCH 2015-11-22 19:30:06,024 DEBUG [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(424)) - Processing container_1448220594280_0001_01_000003 of type START 2015-11-22 19:30:06,024 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220594280_0001_01_000003 Container Transitioned from NEW to ALLOCATED 2015-11-22 19:30:06,024 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: CONTAINER_ALLOCATED 2015-11-22 19:30:06,024 DEBUG [ResourceManager Event Processor] fica.FiCaSchedulerApp (FiCaSchedulerApp.java:allocate(208)) - allocate: applicationAttemptId=appattempt_1448220594280_0001_000001 container=container_1448220594280_0001_01_000003 host=ip-172-31-20-42.ap-northeast-1.compute.internal type=OFF_SWITCH 2015-11-22 19:30:06,024 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220594280_0001_000001 of type CONTAINER_ALLOCATED 2015-11-22 19:30:06,025 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220594280_0001 CONTAINERID=container_1448220594280_0001_01_000003 2015-11-22 19:30:06,025 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:allocateContainer(154)) - Assigned container container_1448220594280_0001_01_000003 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:35735, which has 2 containers, used and available after allocation 2015-11-22 19:30:06,025 INFO [ResourceManager Event Processor] allocator.AbstractContainerAllocator (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(89)) - assignedContainer application attempt=appattempt_1448220594280_0001_000001 container=container_1448220594280_0001_01_000003 queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@5ecb869b clusterResource= type=OFF_SWITCH 2015-11-22 19:30:06,025 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:assignContainers(881)) - post-assignContainers for application application_1448220594280_0001 2015-11-22 19:30:06,025 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:30:06,025 DEBUG [ResourceManager Event Processor] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(470)) - showRequests: application=application_1448220594280_0001 headRoom= currentConsumption=640 2015-11-22 19:30:06,025 DEBUG [ResourceManager Event Processor] scheduler.SchedulerApplicationAttempt (SchedulerApplicationAttempt.java:showRequests(474)) - showRequests: application=application_1448220594280_0001 request={Priority: 0, Capability: , # Containers: 0, Location: *, Relax Locality: true, Node Label Expression: } 2015-11-22 19:30:06,025 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:30:06,025 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:30:06,025 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:30:06,026 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:allocateResource(1277)) - default user=ubuntu used= numContainers=2 headroom = user-resources= 2015-11-22 19:30:06,026 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(575)) - Assigned to queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.15625, absoluteUsedCapacity=0.15625, numApps=1, numContainers=2 --> , OFF_SWITCH 2015-11-22 19:30:06,026 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainersToChildQueues(589)) - Re-sorting assigned queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.15625, absoluteUsedCapacity=0.15625, numApps=1, numContainers=2 2015-11-22 19:30:06,026 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:printChildQueues(617)) - printChildQueues - queue: root child-queues: root.defaultusedCapacity=(0.15625), label=(*) 2015-11-22 19:30:06,026 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(461)) - assignedContainer queue=root usedCapacity=0.15625 absoluteUsedCapacity=0.15625 used= cluster= 2015-11-22 19:30:06,026 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(473)) - ParentQ=root assignedSoFarInThisIteration= usedCapacity=0.15625 absoluteUsedCapacity=0.15625 2015-11-22 19:30:06,026 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(484)) - Not assigning more than one off-switch container, assignments so far: resource:; type:OFF_SWITCH; excessReservation:null; applicationid:null; skipped:false; fulfilled reservation:false; allocations(count/resource):1/; reservations(count/resource):0/ 2015-11-22 19:30:06,054 DEBUG [Task killer for 30738] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:signalContainer(417)) - Sending signal 9 to pid 30738 as user ubuntu 2015-11-22 19:30:06,462 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:38037 from ubuntu sending #954 2015-11-22 19:30:06,462 DEBUG [Socket Reader #1 for port 38037] ipc.Server (Server.java:processOneRpc(1897)) - got #954 2015-11-22 19:30:06,463 DEBUG [IPC Server handler 12 on 38037] ipc.Server (Server.java:run(2185)) - IPC Server handler 12 on 38037: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:35310 Call#954 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:30:06,463 DEBUG [IPC Server handler 12 on 38037] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:30:06,463 DEBUG [IPC Server handler 12 on 38037] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220594280_0001 owned by ubuntu 2015-11-22 19:30:06,463 DEBUG [IPC Server handler 12 on 38037] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 0 2015-11-22 19:30:06,464 DEBUG [IPC Server handler 12 on 38037] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 12 on 38037: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:35310 Call#954 Retry#0 2015-11-22 19:30:06,464 DEBUG [IPC Server handler 12 on 38037] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 12 on 38037: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:35310 Call#954 Retry#0 Wrote 359 bytes. 2015-11-22 19:30:06,464 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:38037 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:38037 from ubuntu got value #954 2015-11-22 19:30:06,464 DEBUG [Thread-4126] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:30:06,464 INFO [Thread-4126] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220600444, yarnAppState=RUNNING, distributedFinalState=UNDEFINED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40017/proxy/application_1448220594280_0001/, appUser=ubuntu 2015-11-22 19:30:06,676 DEBUG [Socket Reader #1 for port 53664] ipc.Server (Server.java:processOneRpc(1897)) - got #5 2015-11-22 19:30:06,677 DEBUG [IPC Server handler 7 on 53664] ipc.Server (Server.java:run(2185)) - IPC Server handler 7 on 53664: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:57589 Call#5 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:30:06,677 DEBUG [IPC Server handler 7 on 53664] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220594280_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:30:06,677 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2015-11-22 19:30:06,677 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220594280_0001_000001 of type STATUS_UPDATE 2015-11-22 19:30:06,677 DEBUG [IPC Server handler 7 on 53664] security.BaseContainerTokenSecretManager (BaseContainerTokenSecretManager.java:createPassword(90)) - Creating password for container_1448220594280_0001_01_000003 for user container_1448220594280_0001_01_000003 (auth:SIMPLE) to be run on NM ip-172-31-20-42.ap-northeast-1.compute.internal:35735 2015-11-22 19:30:06,677 DEBUG [IPC Server handler 7 on 53664] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(197)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 1 cluster_timestamp: 1448220594280 } attemptId: 1 } id: 3 } nmHostAddr: "ip-172-31-20-42.ap-northeast-1.compute.internal:35735" appSubmitter: "ubuntu" resource { memory: 128 virtual_cores: 1 } expiryTimeStamp: 1448221206677 masterKeyId: -368252818 rmIdentifier: 1448220594280 priority { priority: 0 } creationTime: 1448220606024 nodeLabelExpression: "" containerType: TASK 2015-11-22 19:30:06,678 DEBUG [IPC Server handler 7 on 53664] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(197)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 1 cluster_timestamp: 1448220594280 } attemptId: 1 } id: 3 } nmHostAddr: "ip-172-31-20-42.ap-northeast-1.compute.internal:35735" appSubmitter: "ubuntu" resource { memory: 128 virtual_cores: 1 } expiryTimeStamp: 1448221206677 masterKeyId: -368252818 rmIdentifier: 1448220594280 priority { priority: 0 } creationTime: 1448220606024 nodeLabelExpression: "" containerType: TASK 2015-11-22 19:30:06,678 DEBUG [IPC Server handler 7 on 53664] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(424)) - Processing container_1448220594280_0001_01_000003 of type ACQUIRED 2015-11-22 19:30:06,678 INFO [IPC Server handler 7 on 53664] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220594280_0001_01_000003 Container Transitioned from ALLOCATED to ACQUIRED 2015-11-22 19:30:06,678 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppRunningOnNodeEvent.EventType: APP_RUNNING_ON_NODE 2015-11-22 19:30:06,679 DEBUG [IPC Server handler 7 on 53664] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:30:06,679 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(777)) - Processing event for application_1448220594280_0001 of type APP_RUNNING_ON_NODE 2015-11-22 19:30:06,679 DEBUG [IPC Server handler 7 on 53664] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: allocate queueTime= 0 procesingTime= 2 2015-11-22 19:30:06,679 DEBUG [IPC Server handler 7 on 53664] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 7 on 53664: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:57589 Call#5 Retry#0 2015-11-22 19:30:06,680 DEBUG [IPC Server handler 7 on 53664] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 7 on 53664: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 172.31.20.42:57589 Call#5 Retry#0 Wrote 418 bytes. 2015-11-22 19:30:06,685 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - REQUEST /ws/v1/timeline/ on org.mortbay.jetty.HttpConnection@92bc3e6 2015-11-22 19:30:06,685 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - sessionManager=org.mortbay.jetty.servlet.HashSessionManager@536027b9 2015-11-22 19:30:06,685 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - session=null 2015-11-22 19:30:06,685 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - servlet=org.mortbay.jetty.servlet.DefaultServlet-1061138277 2015-11-22 19:30:06,685 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - chain=NoCacheFilter->NoCacheFilter->safety->Timeline Authentication Filter->RMAuthenticationFilter->static_user_filter->guice->org.mortbay.jetty.servlet.DefaultServlet-1061138277 2015-11-22 19:30:06,685 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - servlet holder=org.mortbay.jetty.servlet.DefaultServlet-1061138277 2015-11-22 19:30:06,685 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter NoCacheFilter 2015-11-22 19:30:06,685 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter NoCacheFilter 2015-11-22 19:30:06,686 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter safety 2015-11-22 19:30:06,686 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter Timeline Authentication Filter 2015-11-22 19:30:06,686 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/?user.name=ubuntu] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(557)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:49959/ws/v1/timeline/?user.name=ubuntu] triggering authentication 2015-11-22 19:30:06,686 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/?user.name=ubuntu] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(576)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:49959/ws/v1/timeline/?user.name=ubuntu] user [ubuntu] authenticated 2015-11-22 19:30:06,686 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter RMAuthenticationFilter 2015-11-22 19:30:06,686 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/?user.name=ubuntu] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(557)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:49959/ws/v1/timeline/?user.name=ubuntu] triggering authentication 2015-11-22 19:30:06,686 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/?user.name=ubuntu] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(576)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:49959/ws/v1/timeline/?user.name=ubuntu] user [ubuntu] authenticated 2015-11-22 19:30:06,686 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter static_user_filter 2015-11-22 19:30:06,687 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter guice 2015-11-22 19:30:06,689 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - getResource(META-INF/services/javax.xml.bind.JAXBContext)=jar:file:/home/ubuntu/.m2/repository/com/sun/xml/bind/jaxb-impl/2.2.3-1/jaxb-impl-2.2.3-1.jar!/META-INF/services/javax.xml.bind.JAXBContext 2015-11-22 19:30:06,690 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.xml.bind.v2.ContextFactory from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:30:06,714 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - RESPONSE /ws/v1/timeline/ 200 2015-11-22 19:30:06,717 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - REQUEST /ws/v1/timeline/ on org.mortbay.jetty.HttpConnection@3384e57c 2015-11-22 19:30:06,717 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - sessionManager=org.mortbay.jetty.servlet.HashSessionManager@536027b9 2015-11-22 19:30:06,717 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - session=null 2015-11-22 19:30:06,717 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - servlet=org.mortbay.jetty.servlet.DefaultServlet-1061138277 2015-11-22 19:30:06,717 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - chain=NoCacheFilter->NoCacheFilter->safety->Timeline Authentication Filter->RMAuthenticationFilter->static_user_filter->guice->org.mortbay.jetty.servlet.DefaultServlet-1061138277 2015-11-22 19:30:06,717 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - servlet holder=org.mortbay.jetty.servlet.DefaultServlet-1061138277 2015-11-22 19:30:06,717 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter NoCacheFilter 2015-11-22 19:30:06,718 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter NoCacheFilter 2015-11-22 19:30:06,718 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter safety 2015-11-22 19:30:06,718 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter Timeline Authentication Filter 2015-11-22 19:30:06,718 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(576)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:49959/ws/v1/timeline/] user [ubuntu] authenticated 2015-11-22 19:30:06,718 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter RMAuthenticationFilter 2015-11-22 19:30:06,718 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(576)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:49959/ws/v1/timeline/] user [ubuntu] authenticated 2015-11-22 19:30:06,719 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter static_user_filter 2015-11-22 19:30:06,719 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter guice 2015-11-22 19:30:06,720 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/] security.TimelineACLsManager (TimelineACLsManager.java:checkAccess(106)) - Verifying the access of ubuntu on the timeline entity { id: container_1448220594280_0001_01_000002, type: DS_CONTAINER } 2015-11-22 19:30:06,722 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - RESPONSE /ws/v1/timeline/ 200 2015-11-22 19:30:06,725 DEBUG [1957145000@qtp-248140027-0] mortbay.log (Slf4jLog.java:debug(40)) - EOF 2015-11-22 19:30:06,727 DEBUG [IPC Server listener on 35735] ipc.Server (Server.java:register(2883)) - Server connection from 172.31.20.42:47747; # active connections: 2; # queued calls: 0 2015-11-22 19:30:06,727 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:processOneRpc(1897)) - got #-33 2015-11-22 19:30:06,728 DEBUG [Socket Reader #1 for port 35735] security.SaslRpcServer (SaslRpcServer.java:create(175)) - Created SASL server with mechanism = DIGEST-MD5 2015-11-22 19:30:06,728 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:processResponse(1099)) - Socket Reader #1 for port 35735: responding to null from 172.31.20.42:47747 Call#-33 Retry#-1 2015-11-22 19:30:06,728 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:processResponse(1118)) - Socket Reader #1 for port 35735: responding to null from 172.31.20.42:47747 Call#-33 Retry#-1 Wrote 166 bytes. 2015-11-22 19:30:06,729 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:processOneRpc(1897)) - got #-33 2015-11-22 19:30:06,729 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:processSaslToken(1529)) - Have read input token of size 358 for processing by saslServer.evaluateResponse() 2015-11-22 19:30:06,730 DEBUG [Socket Reader #1 for port 35735] security.BaseNMTokenSecretManager (BaseNMTokenSecretManager.java:retrivePasswordInternal(103)) - creating password for appattempt_1448220594280_0001_000001 for user ubuntu to run on NM ip-172-31-20-42.ap-northeast-1.compute.internal:35735 2015-11-22 19:30:06,730 DEBUG [Socket Reader #1 for port 35735] security.NMTokenIdentifier (NMTokenIdentifier.java:write(96)) - Writing NMTokenIdentifier to RPC layer: appAttemptId { application_id { id: 1 cluster_timestamp: 1448220594280 } attemptId: 1 } nodeId { host: "ip-172-31-20-42.ap-northeast-1.compute.internal" port: 35735 } appSubmitter: "ubuntu" keyId: -1658271729 2015-11-22 19:30:06,730 DEBUG [Socket Reader #1 for port 35735] security.NMTokenSecretManagerInNM (NMTokenSecretManagerInNM.java:retrievePassword(180)) - NMToken password retrieved successfully!! 2015-11-22 19:30:06,730 DEBUG [Socket Reader #1 for port 35735] security.SaslRpcServer (SaslRpcServer.java:handle(310)) - SASL server DIGEST-MD5 callback: setting password for client: appattempt_1448220594280_0001_000001 (auth:SIMPLE) 2015-11-22 19:30:06,731 DEBUG [Socket Reader #1 for port 35735] security.SaslRpcServer (SaslRpcServer.java:handle(328)) - SASL server DIGEST-MD5 callback: setting canonicalized client ID: appattempt_1448220594280_0001_000001 2015-11-22 19:30:06,731 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:buildSaslResponse(1546)) - Will send SUCCESS token of size 40 from saslServer. 2015-11-22 19:30:06,731 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:saslProcess(1434)) - SASL server context established. Negotiated QoP is auth 2015-11-22 19:30:06,731 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:saslProcess(1439)) - SASL server successfully authenticated client: appattempt_1448220594280_0001_000001 (auth:SIMPLE) 2015-11-22 19:30:06,731 INFO [Socket Reader #1 for port 35735] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220594280_0001_000001 (auth:SIMPLE) 2015-11-22 19:30:06,731 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:processResponse(1099)) - Socket Reader #1 for port 35735: responding to null from 172.31.20.42:47747 Call#-33 Retry#-1 2015-11-22 19:30:06,731 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:processResponse(1118)) - Socket Reader #1 for port 35735: responding to null from 172.31.20.42:47747 Call#-33 Retry#-1 Wrote 64 bytes. 2015-11-22 19:30:06,733 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:processOneRpc(1897)) - got #-3 2015-11-22 19:30:06,733 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:authorizeConnection(2097)) - Successfully authorized userInfo { } protocol: "org.apache.hadoop.yarn.api.ContainerManagementProtocolPB" 2015-11-22 19:30:06,733 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:processOneRpc(1897)) - got #6 2015-11-22 19:30:06,734 DEBUG [IPC Server handler 2 on 35735] ipc.Server (Server.java:run(2185)) - IPC Server handler 2 on 35735: org.apache.hadoop.yarn.api.ContainerManagementProtocolPB.startContainers from 172.31.20.42:47747 Call#6 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:30:06,736 DEBUG [IPC Server handler 2 on 35735] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220594280_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:30:06,736 DEBUG [IPC Server handler 2 on 35735] security.BaseContainerTokenSecretManager (BaseContainerTokenSecretManager.java:retrievePasswordInternal(118)) - Retrieving password for container_1448220594280_0001_01_000003 for user container_1448220594280_0001_01_000003 (auth:SIMPLE) to be run on NM ip-172-31-20-42.ap-northeast-1.compute.internal:35735 2015-11-22 19:30:06,737 DEBUG [IPC Server handler 2 on 35735] security.ContainerTokenIdentifier (ContainerTokenIdentifier.java:write(197)) - Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 1 cluster_timestamp: 1448220594280 } attemptId: 1 } id: 3 } nmHostAddr: "ip-172-31-20-42.ap-northeast-1.compute.internal:35735" appSubmitter: "ubuntu" resource { memory: 128 virtual_cores: 1 } expiryTimeStamp: 1448221206677 masterKeyId: -368252818 rmIdentifier: 1448220594280 priority { priority: 0 } creationTime: 1448220606024 nodeLabelExpression: "" containerType: TASK 2015-11-22 19:30:06,737 INFO [IPC Server handler 2 on 35735] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:startContainerInternal(889)) - Start request for container_1448220594280_0001_01_000003 by user ubuntu 2015-11-22 19:30:06,737 DEBUG [IPC Server handler 2 on 35735] utils.YarnServerSecurityUtils (YarnServerSecurityUtils.java:parseCredentials(135)) - = Kind: YARN_AM_RM_TOKEN, Service: , Ident: (appAttemptId { application_id { id: 1 cluster_timestamp: 1448220594280 } attemptId: 1 } keyId: -720358617) 2015-11-22 19:30:06,737 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationContainerInitEvent.EventType: INIT_CONTAINER 2015-11-22 19:30:06,737 INFO [IPC Server handler 2 on 35735] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu IP=172.31.20.42 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1448220594280_0001 CONTAINERID=container_1448220594280_0001_01_000003 2015-11-22 19:30:06,738 DEBUG [IPC Server handler 2 on 35735] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: startContainers queueTime= 2 procesingTime= 2 2015-11-22 19:30:06,738 DEBUG [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(452)) - Processing application_1448220594280_0001 of type INIT_CONTAINER 2015-11-22 19:30:06,738 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(303)) - Adding container_1448220594280_0001_01_000003 to application application_1448220594280_0001 2015-11-22 19:30:06,738 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerInitEvent.EventType: INIT_CONTAINER 2015-11-22 19:30:06,739 DEBUG [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1131)) - Processing container_1448220594280_0001_01_000003 of type INIT_CONTAINER 2015-11-22 19:30:06,739 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220594280_0001_01_000003 transitioned from NEW to LOCALIZED 2015-11-22 19:30:06,739 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServicesEvent.EventType: CONTAINER_INIT 2015-11-22 19:30:06,739 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_INIT for appId application_1448220594280_0001 2015-11-22 19:30:06,739 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncherEvent.EventType: LAUNCH_CONTAINER 2015-11-22 19:30:06,740 DEBUG [IPC Server handler 2 on 35735] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 2 on 35735: responding to org.apache.hadoop.yarn.api.ContainerManagementProtocolPB.startContainers from 172.31.20.42:47747 Call#6 Retry#0 2015-11-22 19:30:06,740 DEBUG [IPC Server handler 2 on 35735] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 2 on 35735: responding to org.apache.hadoop.yarn.api.ContainerManagementProtocolPB.startContainers from 172.31.20.42:47747 Call#6 Retry#0 Wrote 51 bytes. 2015-11-22 19:30:06,740 DEBUG [ContainersLauncher #1] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.fs.FileContext.getAbstractFileSystem(FileContext.java:339) 2015-11-22 19:30:06,742 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:close(2894)) - Socket Reader #1 for port 35735: disconnecting client 172.31.20.42:47747. Number of active connections: 1 2015-11-22 19:30:06,759 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerEvent.EventType: CONTAINER_LAUNCHED 2015-11-22 19:30:06,759 DEBUG [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1131)) - Processing container_1448220594280_0001_01_000003 of type CONTAINER_LAUNCHED 2015-11-22 19:30:06,759 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220594280_0001_01_000003 transitioned from LOCALIZED to RUNNING 2015-11-22 19:30:06,759 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerStartMonitoringEvent.EventType: START_MONITORING_CONTAINER 2015-11-22 19:30:06,775 INFO [ContainersLauncher #1] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:buildCommandExecutor(281)) - launchContainer: [nice, -n, 0, bash, /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220594280_0001/container_1448220594280_0001_01_000003/default_container_executor.sh] 2015-11-22 19:30:06,784 DEBUG [ContainersLauncher #1] launcher.ContainerLaunch (ContainerLaunch.java:call(333)) - Container container_1448220594280_0001_01_000003 completed with exit code 0 2015-11-22 19:30:06,784 INFO [ContainersLauncher #1] launcher.ContainerLaunch (ContainerLaunch.java:call(356)) - Container container_1448220594280_0001_01_000003 succeeded 2015-11-22 19:30:06,784 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerEvent.EventType: CONTAINER_EXITED_WITH_SUCCESS 2015-11-22 19:30:06,784 DEBUG [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1131)) - Processing container_1448220594280_0001_01_000003 of type CONTAINER_EXITED_WITH_SUCCESS 2015-11-22 19:30:06,784 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220594280_0001_01_000003 transitioned from RUNNING to EXITED_WITH_SUCCESS 2015-11-22 19:30:06,784 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncherEvent.EventType: CLEANUP_CONTAINER 2015-11-22 19:30:06,784 INFO [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(380)) - Cleaning up container container_1448220594280_0001_01_000003 2015-11-22 19:30:06,785 DEBUG [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(397)) - Marking container container_1448220594280_0001_01_000003 as inactive 2015-11-22 19:30:06,785 DEBUG [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(403)) - Getting pid for container container_1448220594280_0001_01_000003 to kill from pid file /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/application_1448220594280_0001/container_1448220594280_0001_01_000003/container_1448220594280_0001_01_000003.pid 2015-11-22 19:30:06,785 DEBUG [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:getContainerPid(569)) - Accessing pid for container container_1448220594280_0001_01_000003 from pid file /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/application_1448220594280_0001/container_1448220594280_0001_01_000003/container_1448220594280_0001_01_000003.pid 2015-11-22 19:30:06,785 DEBUG [AsyncDispatcher event handler] util.ProcessIdFileReader (ProcessIdFileReader.java:getProcessId(51)) - Accessing pid from pid file /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/application_1448220594280_0001/container_1448220594280_0001_01_000003/container_1448220594280_0001_01_000003.pid 2015-11-22 19:30:06,785 DEBUG [AsyncDispatcher event handler] util.ProcessIdFileReader (ProcessIdFileReader.java:getProcessId(99)) - Got pid 30790 from path /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/application_1448220594280_0001/container_1448220594280_0001_01_000003/container_1448220594280_0001_01_000003.pid 2015-11-22 19:30:06,785 DEBUG [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:getContainerPid(579)) - Got pid 30790 for container container_1448220594280_0001_01_000003 2015-11-22 19:30:06,785 DEBUG [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(421)) - Sending signal to pid 30790 as user ubuntu for container container_1448220594280_0001_01_000003 2015-11-22 19:30:06,785 DEBUG [AsyncDispatcher event handler] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:signalContainer(417)) - Sending signal 15 to pid 30790 as user ubuntu 2015-11-22 19:30:06,787 DEBUG [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(437)) - Sent signal SIGTERM to pid 30790 as user ubuntu for container container_1448220594280_0001_01_000003, result=failed 2015-11-22 19:30:06,787 DEBUG [AsyncDispatcher event handler] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.fs.FileContext.getAbstractFileSystem(FileContext.java:339) 2015-11-22 19:30:06,789 DEBUG [IPC Server idle connection scanner for port 56823] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 56823: task running 2015-11-22 19:30:06,798 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.ContainerLocalizationCleanupEvent.EventType: CLEANUP_CONTAINER_RESOURCES 2015-11-22 19:30:06,799 DEBUG [DeletionService #0] nodemanager.DeletionService (DeletionService.java:run(266)) - FileDeletionTask : user : ubuntu subDir : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220594280_0001/container_1448220594280_0001_01_000003 baseDir : null 2015-11-22 19:30:06,799 DEBUG [DeletionService #0] nodemanager.DeletionService (DeletionService.java:run(292)) - Deleting path: [/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220594280_0001/container_1448220594280_0001_01_000003] as user: [ubuntu] 2015-11-22 19:30:06,799 INFO [DeletionService #0] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(480)) - Deleting absolute path : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220594280_0001/container_1448220594280_0001_01_000003 2015-11-22 19:30:06,799 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerEvent.EventType: CONTAINER_RESOURCES_CLEANEDUP 2015-11-22 19:30:06,799 DEBUG [DeletionService #1] nodemanager.DeletionService (DeletionService.java:run(266)) - FileDeletionTask : user : null subDir : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/application_1448220594280_0001/container_1448220594280_0001_01_000003 baseDir : null 2015-11-22 19:30:06,799 DEBUG [DeletionService #1] nodemanager.DeletionService (DeletionService.java:run(271)) - NM deleting absolute path : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/application_1448220594280_0001/container_1448220594280_0001_01_000003 2015-11-22 19:30:06,799 DEBUG [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1131)) - Processing container_1448220594280_0001_01_000003 of type CONTAINER_RESOURCES_CLEANEDUP 2015-11-22 19:30:06,799 INFO [AsyncDispatcher event handler] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu OPERATION=Container Finished - Succeeded TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1448220594280_0001 CONTAINERID=container_1448220594280_0001_01_000003 2015-11-22 19:30:06,800 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220594280_0001_01_000003 transitioned from EXITED_WITH_SUCCESS to DONE 2015-11-22 19:30:06,800 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationContainerFinishedEvent.EventType: APPLICATION_CONTAINER_FINISHED 2015-11-22 19:30:06,800 DEBUG [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(452)) - Processing application_1448220594280_0001 of type APPLICATION_CONTAINER_FINISHED 2015-11-22 19:30:06,800 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(346)) - Removing container_1448220594280_0001_01_000003 from application application_1448220594280_0001 2015-11-22 19:30:06,800 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerStopMonitoringEvent.EventType: STOP_MONITORING_CONTAINER 2015-11-22 19:30:06,800 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.event.LogHandlerContainerFinishedEvent.EventType: CONTAINER_FINISHED 2015-11-22 19:30:06,800 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServicesEvent.EventType: CONTAINER_STOP 2015-11-22 19:30:06,800 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_STOP for appId application_1448220594280_0001 2015-11-22 19:30:06,845 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - REQUEST /ws/v1/timeline/ on org.mortbay.jetty.HttpConnection@338efe59 2015-11-22 19:30:06,845 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - sessionManager=org.mortbay.jetty.servlet.HashSessionManager@536027b9 2015-11-22 19:30:06,845 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - session=null 2015-11-22 19:30:06,846 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - servlet=org.mortbay.jetty.servlet.DefaultServlet-1061138277 2015-11-22 19:30:06,846 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - chain=NoCacheFilter->NoCacheFilter->safety->Timeline Authentication Filter->RMAuthenticationFilter->static_user_filter->guice->org.mortbay.jetty.servlet.DefaultServlet-1061138277 2015-11-22 19:30:06,846 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - servlet holder=org.mortbay.jetty.servlet.DefaultServlet-1061138277 2015-11-22 19:30:06,846 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter NoCacheFilter 2015-11-22 19:30:06,846 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter NoCacheFilter 2015-11-22 19:30:06,846 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter safety 2015-11-22 19:30:06,846 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter Timeline Authentication Filter 2015-11-22 19:30:06,846 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/?user.name=ubuntu] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(557)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:49959/ws/v1/timeline/?user.name=ubuntu] triggering authentication 2015-11-22 19:30:06,846 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/?user.name=ubuntu] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(576)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:49959/ws/v1/timeline/?user.name=ubuntu] user [ubuntu] authenticated 2015-11-22 19:30:06,847 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter RMAuthenticationFilter 2015-11-22 19:30:06,847 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/?user.name=ubuntu] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(557)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:49959/ws/v1/timeline/?user.name=ubuntu] triggering authentication 2015-11-22 19:30:06,847 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/?user.name=ubuntu] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(576)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:49959/ws/v1/timeline/?user.name=ubuntu] user [ubuntu] authenticated 2015-11-22 19:30:06,847 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter static_user_filter 2015-11-22 19:30:06,847 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - call filter guice 2015-11-22 19:30:06,850 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - getResource(META-INF/services/javax.xml.bind.JAXBContext)=jar:file:/home/ubuntu/.m2/repository/com/sun/xml/bind/jaxb-impl/2.2.3-1/jaxb-impl-2.2.3-1.jar!/META-INF/services/javax.xml.bind.JAXBContext 2015-11-22 19:30:06,850 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - loaded class com.sun.xml.bind.v2.ContextFactory from sun.misc.Launcher$AppClassLoader@42a57993 2015-11-22 19:30:06,874 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/?user.name=ubuntu] mortbay.log (Slf4jLog.java:debug(40)) - RESPONSE /ws/v1/timeline/ 200 2015-11-22 19:30:06,876 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - REQUEST /ws/v1/timeline/ on org.mortbay.jetty.HttpConnection@4d134818 2015-11-22 19:30:06,876 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - sessionManager=org.mortbay.jetty.servlet.HashSessionManager@536027b9 2015-11-22 19:30:06,876 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - session=null 2015-11-22 19:30:06,877 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - servlet=org.mortbay.jetty.servlet.DefaultServlet-1061138277 2015-11-22 19:30:06,877 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - chain=NoCacheFilter->NoCacheFilter->safety->Timeline Authentication Filter->RMAuthenticationFilter->static_user_filter->guice->org.mortbay.jetty.servlet.DefaultServlet-1061138277 2015-11-22 19:30:06,877 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - servlet holder=org.mortbay.jetty.servlet.DefaultServlet-1061138277 2015-11-22 19:30:06,877 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter NoCacheFilter 2015-11-22 19:30:06,877 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter NoCacheFilter 2015-11-22 19:30:06,877 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter safety 2015-11-22 19:30:06,877 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter Timeline Authentication Filter 2015-11-22 19:30:06,878 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(576)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:49959/ws/v1/timeline/] user [ubuntu] authenticated 2015-11-22 19:30:06,878 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter RMAuthenticationFilter 2015-11-22 19:30:06,878 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/] server.AuthenticationFilter (AuthenticationFilter.java:doFilter(576)) - Request [http://ip-172-31-20-42.ap-northeast-1.compute.internal:49959/ws/v1/timeline/] user [ubuntu] authenticated 2015-11-22 19:30:06,878 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter static_user_filter 2015-11-22 19:30:06,878 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - call filter guice 2015-11-22 19:30:06,879 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/] security.TimelineACLsManager (TimelineACLsManager.java:checkAccess(106)) - Verifying the access of ubuntu on the timeline entity { id: appattempt_1448220594280_0001_000001, type: DS_APP_ATTEMPT } 2015-11-22 19:30:06,884 DEBUG [1957145000@qtp-248140027-0 - /ws/v1/timeline/] mortbay.log (Slf4jLog.java:debug(40)) - RESPONSE /ws/v1/timeline/ 200 2015-11-22 19:30:06,884 DEBUG [1957145000@qtp-248140027-0] mortbay.log (Slf4jLog.java:debug(40)) - EOF 2015-11-22 19:30:06,895 DEBUG [IPC Server listener on 35735] ipc.Server (Server.java:register(2883)) - Server connection from 172.31.20.42:47750; # active connections: 2; # queued calls: 0 2015-11-22 19:30:06,896 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:processOneRpc(1897)) - got #-33 2015-11-22 19:30:06,896 DEBUG [Socket Reader #1 for port 35735] security.SaslRpcServer (SaslRpcServer.java:create(175)) - Created SASL server with mechanism = DIGEST-MD5 2015-11-22 19:30:06,896 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:processResponse(1099)) - Socket Reader #1 for port 35735: responding to null from 172.31.20.42:47750 Call#-33 Retry#-1 2015-11-22 19:30:06,896 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:processResponse(1118)) - Socket Reader #1 for port 35735: responding to null from 172.31.20.42:47750 Call#-33 Retry#-1 Wrote 166 bytes. 2015-11-22 19:30:06,898 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:processOneRpc(1897)) - got #-33 2015-11-22 19:30:06,898 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:processSaslToken(1529)) - Have read input token of size 358 for processing by saslServer.evaluateResponse() 2015-11-22 19:30:06,898 DEBUG [Socket Reader #1 for port 35735] security.BaseNMTokenSecretManager (BaseNMTokenSecretManager.java:retrivePasswordInternal(103)) - creating password for appattempt_1448220594280_0001_000001 for user ubuntu to run on NM ip-172-31-20-42.ap-northeast-1.compute.internal:35735 2015-11-22 19:30:06,899 DEBUG [Socket Reader #1 for port 35735] security.NMTokenIdentifier (NMTokenIdentifier.java:write(96)) - Writing NMTokenIdentifier to RPC layer: appAttemptId { application_id { id: 1 cluster_timestamp: 1448220594280 } attemptId: 1 } nodeId { host: "ip-172-31-20-42.ap-northeast-1.compute.internal" port: 35735 } appSubmitter: "ubuntu" keyId: -1658271729 2015-11-22 19:30:06,899 DEBUG [Socket Reader #1 for port 35735] security.NMTokenSecretManagerInNM (NMTokenSecretManagerInNM.java:retrievePassword(180)) - NMToken password retrieved successfully!! 2015-11-22 19:30:06,899 DEBUG [Socket Reader #1 for port 35735] security.SaslRpcServer (SaslRpcServer.java:handle(310)) - SASL server DIGEST-MD5 callback: setting password for client: appattempt_1448220594280_0001_000001 (auth:SIMPLE) 2015-11-22 19:30:06,899 DEBUG [Socket Reader #1 for port 35735] security.SaslRpcServer (SaslRpcServer.java:handle(328)) - SASL server DIGEST-MD5 callback: setting canonicalized client ID: appattempt_1448220594280_0001_000001 2015-11-22 19:30:06,899 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:buildSaslResponse(1546)) - Will send SUCCESS token of size 40 from saslServer. 2015-11-22 19:30:06,899 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:saslProcess(1434)) - SASL server context established. Negotiated QoP is auth 2015-11-22 19:30:06,899 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:saslProcess(1439)) - SASL server successfully authenticated client: appattempt_1448220594280_0001_000001 (auth:SIMPLE) 2015-11-22 19:30:06,899 INFO [Socket Reader #1 for port 35735] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220594280_0001_000001 (auth:SIMPLE) 2015-11-22 19:30:06,899 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:processResponse(1099)) - Socket Reader #1 for port 35735: responding to null from 172.31.20.42:47750 Call#-33 Retry#-1 2015-11-22 19:30:06,899 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:processResponse(1118)) - Socket Reader #1 for port 35735: responding to null from 172.31.20.42:47750 Call#-33 Retry#-1 Wrote 64 bytes. 2015-11-22 19:30:06,902 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:processOneRpc(1897)) - got #-3 2015-11-22 19:30:06,902 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:authorizeConnection(2097)) - Successfully authorized userInfo { } protocol: "org.apache.hadoop.yarn.api.ContainerManagementProtocolPB" 2015-11-22 19:30:06,902 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:processOneRpc(1897)) - got #7 2015-11-22 19:30:06,902 DEBUG [IPC Server handler 0 on 35735] ipc.Server (Server.java:run(2185)) - IPC Server handler 0 on 35735: org.apache.hadoop.yarn.api.ContainerManagementProtocolPB.stopContainers from 172.31.20.42:47750 Call#7 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:30:06,903 DEBUG [IPC Server handler 0 on 35735] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220594280_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:30:06,903 INFO [IPC Server handler 0 on 35735] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:stopContainerInternal(1163)) - Stopping container with container Id: container_1448220594280_0001_01_000002 2015-11-22 19:30:06,903 INFO [IPC Server handler 0 on 35735] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu IP=172.31.20.42 OPERATION=Stop Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1448220594280_0001 CONTAINERID=container_1448220594280_0001_01_000002 2015-11-22 19:30:06,903 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerKillEvent.EventType: KILL_CONTAINER 2015-11-22 19:30:06,903 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:30:06,903 DEBUG [IPC Server handler 0 on 35735] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: stopContainers queueTime= 1 procesingTime= 0 2015-11-22 19:30:06,904 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 2 container statuses: [ContainerStatus: [ContainerId: container_1448220594280_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ], ContainerStatus: [ContainerId: container_1448220594280_0001_01_000003, State: COMPLETE, Capability: , Diagnostics: , ExitStatus: 0, ]] 2015-11-22 19:30:06,904 DEBUG [IPC Server handler 0 on 35735] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 0 on 35735: responding to org.apache.hadoop.yarn.api.ContainerManagementProtocolPB.stopContainers from 172.31.20.42:47750 Call#7 Retry#0 2015-11-22 19:30:06,904 DEBUG [IPC Server handler 0 on 35735] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 0 on 35735: responding to org.apache.hadoop.yarn.api.ContainerManagementProtocolPB.stopContainers from 172.31.20.42:47750 Call#7 Retry#0 Wrote 51 bytes. 2015-11-22 19:30:06,904 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:30:06,905 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:35735 of type STATUS_UPDATE 2015-11-22 19:30:06,905 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:close(2894)) - Socket Reader #1 for port 35735: disconnecting client 172.31.20.42:47750. Number of active connections: 1 2015-11-22 19:30:06,905 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:30:06,905 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:35735 clusterResources: 2015-11-22 19:30:06,905 DEBUG [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(424)) - Processing container_1448220594280_0001_01_000003 of type FINISHED 2015-11-22 19:30:06,905 INFO [ResourceManager Event Processor] rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(436)) - container_1448220594280_0001_01_000003 Container Transitioned from ACQUIRED to COMPLETED 2015-11-22 19:30:06,905 INFO [ResourceManager Event Processor] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(112)) - USER=ubuntu OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1448220594280_0001 CONTAINERID=container_1448220594280_0001_01_000003 2015-11-22 19:30:06,905 INFO [ResourceManager Event Processor] scheduler.SchedulerNode (SchedulerNode.java:releaseContainer(248)) - Released container container_1448220594280_0001_01_000003 of capacity on host ip-172-31-20-42.ap-northeast-1.compute.internal:35735, which currently has 1 containers, used and available, release resources=true 2015-11-22 19:30:06,906 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:computeUserLimit(1095)) - User limit computation for ubuntu in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: currentCapacity: activeUsers: 0 clusterCapacity: 2015-11-22 19:30:06,906 DEBUG [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:releaseResource(1313)) - default used= numContainers=1 user=ubuntu user-resources= 2015-11-22 19:30:06,906 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:internalReleaseResource(630)) - completedContainer root: numChildQueue= 1, capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.125, numApps=1, numContainers=1, cluster= 2015-11-22 19:30:06,906 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:internalReleaseResource(643)) - Re-sorting completed queue: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.125, absoluteUsedCapacity=0.125, numApps=1, numContainers=1 2015-11-22 19:30:06,906 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:35735 availableResource: 2015-11-22 19:30:06,906 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:30:06,906 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:30:06,908 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptContainerFinishedEvent.EventType: CONTAINER_FINISHED 2015-11-22 19:30:06,908 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220594280_0001_000001 of type CONTAINER_FINISHED 2015-11-22 19:30:06,905 DEBUG [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1131)) - Processing container_1448220594280_0001_01_000002 of type KILL_CONTAINER 2015-11-22 19:30:06,910 DEBUG [IPC Server listener on 35735] ipc.Server (Server.java:register(2883)) - Server connection from 172.31.20.42:47751; # active connections: 2; # queued calls: 0 2015-11-22 19:30:06,911 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:processOneRpc(1897)) - got #-33 2015-11-22 19:30:06,911 DEBUG [Socket Reader #1 for port 35735] security.SaslRpcServer (SaslRpcServer.java:create(175)) - Created SASL server with mechanism = DIGEST-MD5 2015-11-22 19:30:06,912 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:processResponse(1099)) - Socket Reader #1 for port 35735: responding to null from 172.31.20.42:47751 Call#-33 Retry#-1 2015-11-22 19:30:06,912 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:processResponse(1118)) - Socket Reader #1 for port 35735: responding to null from 172.31.20.42:47751 Call#-33 Retry#-1 Wrote 166 bytes. 2015-11-22 19:30:06,913 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:processOneRpc(1897)) - got #-33 2015-11-22 19:30:06,913 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:processSaslToken(1529)) - Have read input token of size 358 for processing by saslServer.evaluateResponse() 2015-11-22 19:30:06,913 DEBUG [Socket Reader #1 for port 35735] security.BaseNMTokenSecretManager (BaseNMTokenSecretManager.java:retrivePasswordInternal(103)) - creating password for appattempt_1448220594280_0001_000001 for user ubuntu to run on NM ip-172-31-20-42.ap-northeast-1.compute.internal:35735 2015-11-22 19:30:06,914 DEBUG [Socket Reader #1 for port 35735] security.NMTokenIdentifier (NMTokenIdentifier.java:write(96)) - Writing NMTokenIdentifier to RPC layer: appAttemptId { application_id { id: 1 cluster_timestamp: 1448220594280 } attemptId: 1 } nodeId { host: "ip-172-31-20-42.ap-northeast-1.compute.internal" port: 35735 } appSubmitter: "ubuntu" keyId: -1658271729 2015-11-22 19:30:06,914 DEBUG [Socket Reader #1 for port 35735] security.NMTokenSecretManagerInNM (NMTokenSecretManagerInNM.java:retrievePassword(180)) - NMToken password retrieved successfully!! 2015-11-22 19:30:06,914 DEBUG [Socket Reader #1 for port 35735] security.SaslRpcServer (SaslRpcServer.java:handle(310)) - SASL server DIGEST-MD5 callback: setting password for client: appattempt_1448220594280_0001_000001 (auth:SIMPLE) 2015-11-22 19:30:06,914 DEBUG [Socket Reader #1 for port 35735] security.SaslRpcServer (SaslRpcServer.java:handle(328)) - SASL server DIGEST-MD5 callback: setting canonicalized client ID: appattempt_1448220594280_0001_000001 2015-11-22 19:30:06,914 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:buildSaslResponse(1546)) - Will send SUCCESS token of size 40 from saslServer. 2015-11-22 19:30:06,914 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:saslProcess(1434)) - SASL server context established. Negotiated QoP is auth 2015-11-22 19:30:06,914 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:saslProcess(1439)) - SASL server successfully authenticated client: appattempt_1448220594280_0001_000001 (auth:SIMPLE) 2015-11-22 19:30:06,915 INFO [Socket Reader #1 for port 35735] ipc.Server (Server.java:saslProcess(1442)) - Auth successful for appattempt_1448220594280_0001_000001 (auth:SIMPLE) 2015-11-22 19:30:06,915 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:processResponse(1099)) - Socket Reader #1 for port 35735: responding to null from 172.31.20.42:47751 Call#-33 Retry#-1 2015-11-22 19:30:06,915 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:processResponse(1118)) - Socket Reader #1 for port 35735: responding to null from 172.31.20.42:47751 Call#-33 Retry#-1 Wrote 64 bytes. 2015-11-22 19:30:06,916 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:processOneRpc(1897)) - got #-3 2015-11-22 19:30:06,916 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:authorizeConnection(2097)) - Successfully authorized userInfo { } protocol: "org.apache.hadoop.yarn.api.ContainerManagementProtocolPB" 2015-11-22 19:30:06,917 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:processOneRpc(1897)) - got #8 2015-11-22 19:30:06,917 DEBUG [IPC Server handler 2 on 35735] ipc.Server (Server.java:run(2185)) - IPC Server handler 2 on 35735: org.apache.hadoop.yarn.api.ContainerManagementProtocolPB.stopContainers from 172.31.20.42:47751 Call#8 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:30:06,917 DEBUG [IPC Server handler 2 on 35735] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220594280_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:30:06,917 INFO [IPC Server handler 2 on 35735] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:stopContainerInternal(1163)) - Stopping container with container Id: container_1448220594280_0001_01_000003 2015-11-22 19:30:06,917 INFO [IPC Server handler 2 on 35735] nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=ubuntu IP=172.31.20.42 OPERATION=Stop Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1448220594280_0001 CONTAINERID=container_1448220594280_0001_01_000003 2015-11-22 19:30:06,918 DEBUG [IPC Server handler 2 on 35735] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: stopContainers queueTime= 0 procesingTime= 1 2015-11-22 19:30:06,918 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerKillEvent.EventType: KILL_CONTAINER 2015-11-22 19:30:06,918 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(428)) - Node's health-status : true, 2015-11-22 19:30:06,918 DEBUG [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1131)) - Processing container_1448220594280_0001_01_000003 of type KILL_CONTAINER 2015-11-22 19:30:06,919 DEBUG [IPC Server handler 2 on 35735] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 2 on 35735: responding to org.apache.hadoop.yarn.api.ContainerManagementProtocolPB.stopContainers from 172.31.20.42:47751 Call#8 Retry#0 2015-11-22 19:30:06,919 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getContainerStatuses(516)) - Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1448220594280_0001_01_000001, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2015-11-22 19:30:06,919 DEBUG [IPC Server handler 2 on 35735] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 2 on 35735: responding to org.apache.hadoop.yarn.api.ContainerManagementProtocolPB.stopContainers from 172.31.20.42:47751 Call#8 Retry#0 Wrote 51 bytes. 2015-11-22 19:30:06,919 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2015-11-22 19:30:06,919 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(554)) - Processing ip-172-31-20-42.ap-northeast-1.compute.internal:35735 of type STATUS_UPDATE 2015-11-22 19:30:06,919 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:close(2894)) - Socket Reader #1 for port 35735: disconnecting client 172.31.20.42:47751. Number of active connections: 1 2015-11-22 19:30:06,919 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2015-11-22 19:30:06,919 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1015)) - nodeUpdate: ip-172-31-20-42.ap-northeast-1.compute.internal:35735 clusterResources: 2015-11-22 19:30:06,919 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:nodeUpdate(1068)) - Node being looked for scheduling ip-172-31-20-42.ap-northeast-1.compute.internal:35735 availableResource: 2015-11-22 19:30:06,919 DEBUG [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:allocateContainersToNode(1215)) - Trying to schedule on node: ip-172-31-20-42.ap-northeast-1.compute.internal, available: 2015-11-22 19:30:06,919 DEBUG [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:assignContainers(396)) - Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2015-11-22 19:30:06,923 DEBUG [Socket Reader #1 for port 53664] ipc.Server (Server.java:processOneRpc(1897)) - got #9 2015-11-22 19:30:06,923 DEBUG [IPC Server handler 3 on 53664] ipc.Server (Server.java:run(2185)) - IPC Server handler 3 on 53664: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.finishApplicationMaster from 172.31.20.42:57589 Call#9 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:30:06,923 DEBUG [IPC Server handler 3 on 53664] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220594280_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:30:06,923 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptUnregistrationEvent.EventType: UNREGISTERED 2015-11-22 19:30:06,923 DEBUG [IPC Server handler 3 on 53664] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: finishApplicationMaster queueTime= 0 procesingTime= 0 2015-11-22 19:30:06,923 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220594280_0001_000001 of type UNREGISTERED 2015-11-22 19:30:06,923 DEBUG [IPC Server handler 3 on 53664] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 3 on 53664: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.finishApplicationMaster from 172.31.20.42:57589 Call#9 Retry#0 2015-11-22 19:30:06,924 DEBUG [IPC Server handler 3 on 53664] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 3 on 53664: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.finishApplicationMaster from 172.31.20.42:57589 Call#9 Retry#0 Wrote 34 bytes. 2015-11-22 19:30:06,924 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:rememberTargetTransitionsAndStoreState(1258)) - Updating application attempt appattempt_1448220594280_0001_000001 with final state: FINISHING, and exit status: -1000 2015-11-22 19:30:06,924 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220594280_0001_000001 State change from RUNNING to FINAL_SAVING 2015-11-22 19:30:06,924 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEvent.EventType: ATTEMPT_UNREGISTERED 2015-11-22 19:30:06,924 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(777)) - Processing event for application_1448220594280_0001 of type ATTEMPT_UNREGISTERED 2015-11-22 19:30:06,924 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:rememberTargetTransitionsAndStoreState(1098)) - Updating application application_1448220594280_0001 with final state: FINISHING 2015-11-22 19:30:06,924 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220594280_0001 State change from RUNNING to FINAL_SAVING 2015-11-22 19:30:06,924 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateUpdateAppAttemptEvent.EventType: UPDATE_APP_ATTEMPT 2015-11-22 19:30:06,924 DEBUG [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:handleStoreEvent(1016)) - Processing event of type UPDATE_APP_ATTEMPT 2015-11-22 19:30:06,925 DEBUG [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:transition(326)) - Updating info for attempt: appattempt_1448220594280_0001_000001 2015-11-22 19:30:06,925 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateUpdateAppEvent.EventType: UPDATE_APP 2015-11-22 19:30:06,925 DEBUG [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:handleStoreEvent(1016)) - Processing event of type UPDATE_APP 2015-11-22 19:30:06,925 INFO [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:transition(237)) - Updating info for app: application_1448220594280_0001 2015-11-22 19:30:06,925 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: ATTEMPT_UPDATE_SAVED 2015-11-22 19:30:06,925 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(835)) - Processing event for appattempt_1448220594280_0001_000001 of type ATTEMPT_UPDATE_SAVED 2015-11-22 19:30:06,925 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(847)) - appattempt_1448220594280_0001_000001 State change from FINAL_SAVING to FINISHING 2015-11-22 19:30:06,926 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEvent.EventType: APP_UPDATE_SAVED 2015-11-22 19:30:06,926 DEBUG [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(777)) - Processing event for application_1448220594280_0001 of type APP_UPDATE_SAVED 2015-11-22 19:30:06,926 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(789)) - application_1448220594280_0001 State change from FINAL_SAVING to FINISHING 2015-11-22 19:30:07,026 DEBUG [Socket Reader #1 for port 53664] ipc.Server (Server.java:processOneRpc(1897)) - got #10 2015-11-22 19:30:07,026 DEBUG [IPC Server handler 6 on 53664] ipc.Server (Server.java:run(2185)) - IPC Server handler 6 on 53664: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.finishApplicationMaster from 172.31.20.42:57589 Call#10 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:30:07,026 DEBUG [IPC Server handler 6 on 53664] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:appattempt_1448220594280_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:30:07,026 INFO [IPC Server handler 6 on 53664] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:finishApplicationMaster(300)) - application_1448220594280_0001 unregistered successfully. 2015-11-22 19:30:07,026 DEBUG [IPC Server handler 6 on 53664] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: finishApplicationMaster queueTime= 0 procesingTime= 0 2015-11-22 19:30:07,027 DEBUG [IPC Server handler 6 on 53664] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 6 on 53664: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.finishApplicationMaster from 172.31.20.42:57589 Call#10 Retry#0 2015-11-22 19:30:07,027 DEBUG [IPC Server handler 6 on 53664] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 6 on 53664: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.finishApplicationMaster from 172.31.20.42:57589 Call#10 Retry#0 Wrote 34 bytes. 2015-11-22 19:30:07,037 DEBUG [Task killer for 30790] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:signalContainer(417)) - Sending signal 9 to pid 30790 as user ubuntu 2015-11-22 19:30:07,041 DEBUG [Socket Reader #1 for port 53664] ipc.Server (Server.java:close(2894)) - Socket Reader #1 for port 53664: disconnecting client 172.31.20.42:57589. Number of active connections: 0 2015-11-22 19:30:07,041 DEBUG [1957145000@qtp-248140027-0] mortbay.log (Slf4jLog.java:debug(40)) - EOF 2015-11-22 19:30:07,041 DEBUG [1957145000@qtp-248140027-0] mortbay.log (Slf4jLog.java:debug(40)) - EOF 2015-11-22 19:30:07,045 WARN [ContainersLauncher #0] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(237)) - Exit code from container container_1448220594280_0001_01_000001 is : 2 2015-11-22 19:30:07,045 WARN [ContainersLauncher #0] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(243)) - Exception from container-launch with container ID: container_1448220594280_0001_01_000001 and exit code: 2 ExitCodeException exitCode=2: at org.apache.hadoop.util.Shell.runCommand(Shell.java:909) at org.apache.hadoop.util.Shell.run(Shell.java:820) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1099) at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:225) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:304) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:84) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 2015-11-22 19:30:07,045 INFO [ContainersLauncher #0] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - Exception from container-launch. 2015-11-22 19:30:07,045 INFO [ContainersLauncher #0] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - Container id: container_1448220594280_0001_01_000001 2015-11-22 19:30:07,045 INFO [ContainersLauncher #0] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - Exit code: 2 2015-11-22 19:30:07,045 INFO [ContainersLauncher #0] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - Stack trace: ExitCodeException exitCode=2: 2015-11-22 19:30:07,045 INFO [ContainersLauncher #0] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - at org.apache.hadoop.util.Shell.runCommand(Shell.java:909) 2015-11-22 19:30:07,045 INFO [ContainersLauncher #0] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - at org.apache.hadoop.util.Shell.run(Shell.java:820) 2015-11-22 19:30:07,046 INFO [ContainersLauncher #0] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1099) 2015-11-22 19:30:07,046 INFO [ContainersLauncher #0] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:225) 2015-11-22 19:30:07,046 INFO [ContainersLauncher #0] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:304) 2015-11-22 19:30:07,046 INFO [ContainersLauncher #0] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:84) 2015-11-22 19:30:07,046 INFO [ContainersLauncher #0] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - at java.util.concurrent.FutureTask.run(FutureTask.java:266) 2015-11-22 19:30:07,046 INFO [ContainersLauncher #0] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 2015-11-22 19:30:07,046 INFO [ContainersLauncher #0] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 2015-11-22 19:30:07,046 INFO [ContainersLauncher #0] nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(333)) - at java.lang.Thread.run(Thread.java:745) 2015-11-22 19:30:07,046 DEBUG [ContainersLauncher #0] container.ContainerImpl (ContainerImpl.java:handle(1131)) - Processing container_1448220594280_0001_01_000001 of type UPDATE_DIAGNOSTICS_MSG 2015-11-22 19:30:07,046 DEBUG [ContainersLauncher #0] launcher.ContainerLaunch (ContainerLaunch.java:call(333)) - Container container_1448220594280_0001_01_000001 completed with exit code 2 2015-11-22 19:30:07,046 WARN [ContainersLauncher #0] launcher.ContainerLaunch (ContainerLaunch.java:call(348)) - Container exited with a non-zero exit code 2 2015-11-22 19:30:07,046 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerExitEvent.EventType: CONTAINER_EXITED_WITH_FAILURE 2015-11-22 19:30:07,046 DEBUG [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1131)) - Processing container_1448220594280_0001_01_000001 of type CONTAINER_EXITED_WITH_FAILURE 2015-11-22 19:30:07,047 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220594280_0001_01_000001 transitioned from RUNNING to EXITED_WITH_FAILURE 2015-11-22 19:30:07,047 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncherEvent.EventType: CLEANUP_CONTAINER 2015-11-22 19:30:07,047 INFO [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(380)) - Cleaning up container container_1448220594280_0001_01_000001 2015-11-22 19:30:07,047 DEBUG [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(397)) - Marking container container_1448220594280_0001_01_000001 as inactive 2015-11-22 19:30:07,047 DEBUG [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(403)) - Getting pid for container container_1448220594280_0001_01_000001 to kill from pid file /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/application_1448220594280_0001/container_1448220594280_0001_01_000001/container_1448220594280_0001_01_000001.pid 2015-11-22 19:30:07,047 DEBUG [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:getContainerPid(569)) - Accessing pid for container container_1448220594280_0001_01_000001 from pid file /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/application_1448220594280_0001/container_1448220594280_0001_01_000001/container_1448220594280_0001_01_000001.pid 2015-11-22 19:30:07,047 DEBUG [AsyncDispatcher event handler] util.ProcessIdFileReader (ProcessIdFileReader.java:getProcessId(51)) - Accessing pid from pid file /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/application_1448220594280_0001/container_1448220594280_0001_01_000001/container_1448220594280_0001_01_000001.pid 2015-11-22 19:30:07,047 DEBUG [AsyncDispatcher event handler] util.ProcessIdFileReader (ProcessIdFileReader.java:getProcessId(99)) - Got pid 30665 from path /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/application_1448220594280_0001/container_1448220594280_0001_01_000001/container_1448220594280_0001_01_000001.pid 2015-11-22 19:30:07,047 DEBUG [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:getContainerPid(579)) - Got pid 30665 for container container_1448220594280_0001_01_000001 2015-11-22 19:30:07,047 DEBUG [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(421)) - Sending signal to pid 30665 as user ubuntu for container container_1448220594280_0001_01_000001 2015-11-22 19:30:07,047 DEBUG [AsyncDispatcher event handler] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:signalContainer(417)) - Sending signal 15 to pid 30665 as user ubuntu 2015-11-22 19:30:07,049 DEBUG [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(437)) - Sent signal SIGTERM to pid 30665 as user ubuntu for container container_1448220594280_0001_01_000001, result=failed 2015-11-22 19:30:07,050 DEBUG [AsyncDispatcher event handler] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.fs.FileContext.getAbstractFileSystem(FileContext.java:339) 2015-11-22 19:30:07,061 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.ContainerLocalizationCleanupEvent.EventType: CLEANUP_CONTAINER_RESOURCES 2015-11-22 19:30:07,061 DEBUG [AsyncDispatcher event handler] localizer.LocalizedResource (LocalizedResource.java:handle(193)) - Processing file:/home/ubuntu/DistributedShell/application_1448220594280_0001/AppMaster.jar of type RELEASE 2015-11-22 19:30:07,061 DEBUG [AsyncDispatcher event handler] localizer.LocalizedResource (LocalizedResource.java:handle(193)) - Processing file:/home/ubuntu/DistributedShell/application_1448220594280_0001/shellCommands of type RELEASE 2015-11-22 19:30:07,061 DEBUG [DeletionService #2] nodemanager.DeletionService (DeletionService.java:run(266)) - FileDeletionTask : user : ubuntu subDir : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220594280_0001/container_1448220594280_0001_01_000001 baseDir : null 2015-11-22 19:30:07,061 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerEvent.EventType: CONTAINER_RESOURCES_CLEANEDUP 2015-11-22 19:30:07,062 DEBUG [DeletionService #2] nodemanager.DeletionService (DeletionService.java:run(292)) - Deleting path: [/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220594280_0001/container_1448220594280_0001_01_000001] as user: [ubuntu] 2015-11-22 19:30:07,062 INFO [DeletionService #2] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(480)) - Deleting absolute path : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220594280_0001/container_1448220594280_0001_01_000001 2015-11-22 19:30:07,061 DEBUG [DeletionService #3] nodemanager.DeletionService (DeletionService.java:run(266)) - FileDeletionTask : user : null subDir : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/application_1448220594280_0001/container_1448220594280_0001_01_000001 baseDir : null 2015-11-22 19:30:07,062 DEBUG [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1131)) - Processing container_1448220594280_0001_01_000001 of type CONTAINER_RESOURCES_CLEANEDUP 2015-11-22 19:30:07,062 WARN [AsyncDispatcher event handler] nodemanager.NMAuditLogger (NMAuditLogger.java:logFailure(150)) - USER=ubuntu OPERATION=Container Finished - Failed TARGET=ContainerImpl RESULT=FAILURE DESCRIPTION=Container failed with state: EXITED_WITH_FAILURE APPID=application_1448220594280_0001 CONTAINERID=container_1448220594280_0001_01_000001 2015-11-22 19:30:07,062 INFO [AsyncDispatcher event handler] container.ContainerImpl (ContainerImpl.java:handle(1143)) - Container container_1448220594280_0001_01_000001 transitioned from EXITED_WITH_FAILURE to DONE 2015-11-22 19:30:07,062 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationContainerFinishedEvent.EventType: APPLICATION_CONTAINER_FINISHED 2015-11-22 19:30:07,062 DEBUG [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(452)) - Processing application_1448220594280_0001 of type APPLICATION_CONTAINER_FINISHED 2015-11-22 19:30:07,062 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:transition(346)) - Removing container_1448220594280_0001_01_000001 from application application_1448220594280_0001 2015-11-22 19:30:07,062 DEBUG [DeletionService #3] nodemanager.DeletionService (DeletionService.java:run(271)) - NM deleting absolute path : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/application_1448220594280_0001/container_1448220594280_0001_01_000001 2015-11-22 19:30:07,062 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerStopMonitoringEvent.EventType: STOP_MONITORING_CONTAINER 2015-11-22 19:30:07,063 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.event.LogHandlerContainerFinishedEvent.EventType: CONTAINER_FINISHED 2015-11-22 19:30:07,063 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServicesEvent.EventType: CONTAINER_STOP 2015-11-22 19:30:07,063 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_STOP for appId application_1448220594280_0001 2015-11-22 19:30:07,300 DEBUG [Task killer for 30665] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:signalContainer(417)) - Sending signal 9 to pid 30665 as user ubuntu 2015-11-22 19:30:07,464 DEBUG [IPC Parameter Sending Thread #0] ipc.Client (Client.java:run(1063)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:38037 from ubuntu sending #955 2015-11-22 19:30:07,465 DEBUG [Socket Reader #1 for port 38037] ipc.Server (Server.java:processOneRpc(1897)) - got #955 2015-11-22 19:30:07,465 DEBUG [IPC Server handler 17 on 38037] ipc.Server (Server.java:run(2185)) - IPC Server handler 17 on 38037: org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:35310 Call#955 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2015-11-22 19:30:07,465 DEBUG [IPC Server handler 17 on 38037] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2215) 2015-11-22 19:30:07,465 DEBUG [IPC Server handler 17 on 38037] security.ApplicationACLsManager (ApplicationACLsManager.java:checkAccess(102)) - Verifying access-type VIEW_APP for ubuntu (auth:SIMPLE) on application application_1448220594280_0001 owned by ubuntu 2015-11-22 19:30:07,465 DEBUG [IPC Server handler 17 on 38037] ipc.Server (ProtobufRpcEngine.java:call(632)) - Served: getApplicationReport queueTime= 0 procesingTime= 0 2015-11-22 19:30:07,466 DEBUG [IPC Server handler 17 on 38037] ipc.Server (Server.java:processResponse(1099)) - IPC Server handler 17 on 38037: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:35310 Call#955 Retry#0 2015-11-22 19:30:07,466 DEBUG [IPC Server handler 17 on 38037] ipc.Server (Server.java:processResponse(1118)) - IPC Server handler 17 on 38037: responding to org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 172.31.20.42:35310 Call#955 Retry#0 Wrote 364 bytes. 2015-11-22 19:30:07,466 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:38037 from ubuntu] ipc.Client (Client.java:receiveRpcResponse(1120)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:38037 from ubuntu got value #955 2015-11-22 19:30:07,466 DEBUG [Thread-4126] ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(251)) - Call: getApplicationReport took 2ms 2015-11-22 19:30:07,466 INFO [Thread-4126] distributedshell.Client (Client.java:monitorApplication(742)) - Got application report from ASM for, appId=1, clientToAMToken=null, appDiagnostics=, appMasterHost=ip-172-31-20-42/172.31.20.42, appQueue=default, appMasterRpcPort=-1, appStartTime=1448220600444, yarnAppState=FINISHED, distributedFinalState=FAILED, appTrackingUrl=http://ip-172-31-20-42.ap-northeast-1.compute.internal:40017/proxy/application_1448220594280_0001/, appUser=ubuntu 2015-11-22 19:30:07,466 INFO [Thread-4126] distributedshell.Client (Client.java:monitorApplication(763)) - Application did finished unsuccessfully. YarnState=FINISHED, DSFinalStatus=FAILED. Breaking monitoring loop 2015-11-22 19:30:07,466 INFO [Thread-4126] distributedshell.TestDistributedShell (TestDistributedShell.java:testContainerLaunchFailureHandling(811)) - Client run completed. Result=false 2015-11-22 19:30:07,466 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: TestDistributedShell entered state STOPPED 2015-11-22 19:30:07,466 DEBUG [main] service.CompositeService (CompositeService.java:serviceStop(129)) - TestDistributedShell: stopping services, size=3 2015-11-22 19:30:07,467 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #2: Service org.apache.hadoop.yarn.server.MiniYARNCluster$ApplicationHistoryServerWrapper in state org.apache.hadoop.yarn.server.MiniYARNCluster$ApplicationHistoryServerWrapper: STARTED 2015-11-22 19:30:07,467 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.MiniYARNCluster$ApplicationHistoryServerWrapper entered state STOPPED 2015-11-22 19:30:07,467 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer entered state STOPPED 2015-11-22 19:30:07,468 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.mortbay.jetty.nio.SelectChannelConnector$1@4ba8cb34 2015-11-22 19:30:07,469 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping org.mortbay.jetty.webapp.WebAppContext@733f6120{/,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes/webapps/applicationhistory} 2015-11-22 19:30:07,469 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping SessionHandler@5b5e6b93 2015-11-22 19:30:07,469 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping SecurityHandler@612fa7b 2015-11-22 19:30:07,469 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping ServletHandler@2f2eba34 2015-11-22 19:30:07,470 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped guice 2015-11-22 19:30:07,470 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped Timeline Authentication Filter 2015-11-22 19:30:07,470 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped static_user_filter 2015-11-22 19:30:07,470 DEBUG [main] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:stopThreads(619)) - Stopping expired delegation token remover thread 2015-11-22 19:30:07,470 ERROR [Thread[Thread-4122,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(670)) - ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted 2015-11-22 19:30:07,470 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped RMAuthenticationFilter 2015-11-22 19:30:07,471 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped safety 2015-11-22 19:30:07,471 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped NoCacheFilter 2015-11-22 19:30:07,471 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped NoCacheFilter 2015-11-22 19:30:07,471 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped conf 2015-11-22 19:30:07,471 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped jmx 2015-11-22 19:30:07,471 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped metrics 2015-11-22 19:30:07,471 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped logLevel 2015-11-22 19:30:07,471 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped stacks 2015-11-22 19:30:07,472 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.mortbay.jetty.servlet.DefaultServlet$NIOResourceCache@79e2192 2015-11-22 19:30:07,472 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.mortbay.jetty.servlet.DefaultServlet-1061138277 2015-11-22 19:30:07,472 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped ServletHandler@2f2eba34 2015-11-22 19:30:07,472 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped SecurityHandler@612fa7b 2015-11-22 19:30:07,472 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.mortbay.jetty.servlet.HashSessionManager@536027b9 2015-11-22 19:30:07,473 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped SessionHandler@5b5e6b93 2015-11-22 19:30:07,473 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping ErrorPageErrorHandler@945d8fb 2015-11-22 19:30:07,473 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped ErrorPageErrorHandler@945d8fb 2015-11-22 19:30:07,474 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 - guice as filter 2015-11-22 19:30:07,474 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 - Timeline Authentication Filter as filter 2015-11-22 19:30:07,474 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 - static_user_filter as filter 2015-11-22 19:30:07,474 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 - RMAuthenticationFilter as filter 2015-11-22 19:30:07,474 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 - safety as filter 2015-11-22 19:30:07,474 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 - NoCacheFilter as filter 2015-11-22 19:30:07,474 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 - NoCacheFilter as filter 2015-11-22 19:30:07,474 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 - (F=guice,[/*],[],15) as filterMapping 2015-11-22 19:30:07,475 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 - (F=static_user_filter,[/ws/*],[],15) as filterMapping 2015-11-22 19:30:07,475 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 - (F=RMAuthenticationFilter,[/ws/*],[],15) as filterMapping 2015-11-22 19:30:07,475 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 - (F=static_user_filter,[/applicationhistory/*],[],15) as filterMapping 2015-11-22 19:30:07,475 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 - (F=RMAuthenticationFilter,[/applicationhistory/*],[],15) as filterMapping 2015-11-22 19:30:07,475 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 - (F=static_user_filter,[/conf],[],15) as filterMapping 2015-11-22 19:30:07,475 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 - (F=RMAuthenticationFilter,[/conf],[],15) as filterMapping 2015-11-22 19:30:07,475 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 - (F=static_user_filter,[/jmx],[],15) as filterMapping 2015-11-22 19:30:07,475 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 - (F=RMAuthenticationFilter,[/jmx],[],15) as filterMapping 2015-11-22 19:30:07,476 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 - (F=static_user_filter,[/metrics],[],15) as filterMapping 2015-11-22 19:30:07,476 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 - (F=RMAuthenticationFilter,[/metrics],[],15) as filterMapping 2015-11-22 19:30:07,476 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 - (F=static_user_filter,[/logLevel],[],15) as filterMapping 2015-11-22 19:30:07,476 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 - (F=RMAuthenticationFilter,[/logLevel],[],15) as filterMapping 2015-11-22 19:30:07,476 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 - (F=static_user_filter,[/stacks],[],15) as filterMapping 2015-11-22 19:30:07,476 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 - (F=RMAuthenticationFilter,[/stacks],[],15) as filterMapping 2015-11-22 19:30:07,476 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 - (F=Timeline Authentication Filter,[/*],[],15) as filterMapping 2015-11-22 19:30:07,476 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 - (F=static_user_filter,[*.html, *.jsp],[],15) as filterMapping 2015-11-22 19:30:07,476 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 - (F=RMAuthenticationFilter,[*.html, *.jsp],[],15) as filterMapping 2015-11-22 19:30:07,477 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 - (F=safety,[/*],[],15) as filterMapping 2015-11-22 19:30:07,477 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 - (F=NoCacheFilter,[/*],[],15) as filterMapping 2015-11-22 19:30:07,477 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 - (F=NoCacheFilter,[/*],[],15) as filterMapping 2015-11-22 19:30:07,477 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap=null 2015-11-22 19:30:07,477 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=null 2015-11-22 19:30:07,477 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:30:07,477 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/jmx=jmx, /conf=conf, /stacks=stacks, /logLevel=logLevel, /metrics=metrics, /=org.mortbay.jetty.servlet.DefaultServlet-1061138277} 2015-11-22 19:30:07,477 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={logLevel=logLevel, jmx=jmx, stacks=stacks, conf=conf, metrics=metrics, org.mortbay.jetty.servlet.DefaultServlet-1061138277=org.mortbay.jetty.servlet.DefaultServlet-1061138277} 2015-11-22 19:30:07,477 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 - conf as servlet 2015-11-22 19:30:07,478 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 - jmx as servlet 2015-11-22 19:30:07,478 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 - metrics as servlet 2015-11-22 19:30:07,478 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 - logLevel as servlet 2015-11-22 19:30:07,478 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 - stacks as servlet 2015-11-22 19:30:07,478 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 - org.mortbay.jetty.servlet.DefaultServlet-1061138277 as servlet 2015-11-22 19:30:07,478 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 - (S=conf,[/conf]) as servletMapping 2015-11-22 19:30:07,478 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 - (S=jmx,[/jmx]) as servletMapping 2015-11-22 19:30:07,478 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 - (S=metrics,[/metrics]) as servletMapping 2015-11-22 19:30:07,478 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 - (S=logLevel,[/logLevel]) as servletMapping 2015-11-22 19:30:07,479 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 - (S=stacks,[/stacks]) as servletMapping 2015-11-22 19:30:07,479 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@2f2eba34 - (S=org.mortbay.jetty.servlet.DefaultServlet-1061138277,[/]) as servletMapping 2015-11-22 19:30:07,479 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap=null 2015-11-22 19:30:07,479 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=null 2015-11-22 19:30:07,479 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:30:07,479 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap=null 2015-11-22 19:30:07,479 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap=null 2015-11-22 19:30:07,479 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.mortbay.jetty.webapp.WebAppContext@733f6120{/,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes/webapps/applicationhistory} 2015-11-22 19:30:07,480 INFO [main] mortbay.log (Slf4jLog.java:info(67)) - Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:0 2015-11-22 19:30:07,480 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:0 2015-11-22 19:30:07,480 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping Server@1d16906a 2015-11-22 19:30:07,480 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping ContextHandlerCollection@5f10502a 2015-11-22 19:30:07,480 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping org.mortbay.jetty.servlet.Context@4515f214{/static,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes/webapps/static/} 2015-11-22 19:30:07,480 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping ServletHandler@74e55156 2015-11-22 19:30:07,480 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.mortbay.jetty.servlet.DefaultServlet-978534330 2015-11-22 19:30:07,480 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped ServletHandler@74e55156 2015-11-22 19:30:07,481 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping ErrorHandler@f815e6b 2015-11-22 19:30:07,481 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped ErrorHandler@f815e6b 2015-11-22 19:30:07,481 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.mortbay.jetty.servlet.Context@4515f214{/static,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes/webapps/static/} 2015-11-22 19:30:07,481 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping org.mortbay.jetty.servlet.Context@18153a8a{/logs,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/log} 2015-11-22 19:30:07,481 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping ServletHandler@303fd8e8 2015-11-22 19:30:07,481 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.apache.hadoop.http.AdminAuthorizedServlet-1655733433 2015-11-22 19:30:07,482 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped ServletHandler@303fd8e8 2015-11-22 19:30:07,482 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping ErrorHandler@7dd52434 2015-11-22 19:30:07,482 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped ErrorHandler@7dd52434 2015-11-22 19:30:07,482 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.mortbay.jetty.servlet.Context@18153a8a{/logs,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/log} 2015-11-22 19:30:07,482 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped ContextHandlerCollection@5f10502a 2015-11-22 19:30:07,482 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.mortbay.jetty.servlet.HashSessionIdManager@1fe1b8b1 2015-11-22 19:30:07,583 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.mortbay.thread.QueuedThreadPool@eca50fb 2015-11-22 19:30:07,583 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped Server@1d16906a 2015-11-22 19:30:07,585 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:shutdown(588)) - refCount=3 2015-11-22 19:30:07,585 DEBUG [main] service.CompositeService (CompositeService.java:serviceStop(129)) - org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer: stopping services, size=5 2015-11-22 19:30:07,585 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #4: Service org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryManagerImpl in state org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryManagerImpl: STARTED 2015-11-22 19:30:07,585 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryManagerImpl entered state STOPPED 2015-11-22 19:30:07,585 INFO [main] applicationhistoryservice.ApplicationHistoryManagerImpl (ApplicationHistoryManagerImpl.java:serviceStop(78)) - Stopping ApplicationHistory 2015-11-22 19:30:07,585 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.applicationhistoryservice.MemoryApplicationHistoryStore entered state STOPPED 2015-11-22 19:30:07,586 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #3: Service ApplicationHistoryClientService in state ApplicationHistoryClientService: STARTED 2015-11-22 19:30:07,586 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: ApplicationHistoryClientService entered state STOPPED 2015-11-22 19:30:07,587 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 56823 2015-11-22 19:30:07,587 DEBUG [IPC Server handler 0 on 56823] ipc.Server (Server.java:run(2290)) - IPC Server handler 0 on 56823: exiting 2015-11-22 19:30:07,587 DEBUG [IPC Server handler 5 on 56823] ipc.Server (Server.java:run(2290)) - IPC Server handler 5 on 56823: exiting 2015-11-22 19:30:07,588 DEBUG [IPC Server handler 6 on 56823] ipc.Server (Server.java:run(2290)) - IPC Server handler 6 on 56823: exiting 2015-11-22 19:30:07,588 DEBUG [IPC Server handler 8 on 56823] ipc.Server (Server.java:run(2290)) - IPC Server handler 8 on 56823: exiting 2015-11-22 19:30:07,587 DEBUG [IPC Server handler 2 on 56823] ipc.Server (Server.java:run(2290)) - IPC Server handler 2 on 56823: exiting 2015-11-22 19:30:07,587 DEBUG [IPC Server handler 4 on 56823] ipc.Server (Server.java:run(2290)) - IPC Server handler 4 on 56823: exiting 2015-11-22 19:30:07,589 DEBUG [IPC Server Responder] ipc.Server (Server.java:doRunLoop(997)) - Checking for old call responses. 2015-11-22 19:30:07,589 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:30:07,587 DEBUG [IPC Server handler 1 on 56823] ipc.Server (Server.java:run(2290)) - IPC Server handler 1 on 56823: exiting 2015-11-22 19:30:07,589 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #2: Service org.apache.hadoop.yarn.server.timeline.TimelineDataManager in state org.apache.hadoop.yarn.server.timeline.TimelineDataManager: STARTED 2015-11-22 19:30:07,590 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.timeline.TimelineDataManager entered state STOPPED 2015-11-22 19:30:07,589 INFO [IPC Server listener on 56823] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 56823 2015-11-22 19:30:07,588 DEBUG [IPC Server handler 9 on 56823] ipc.Server (Server.java:run(2290)) - IPC Server handler 9 on 56823: exiting 2015-11-22 19:30:07,588 DEBUG [IPC Server handler 3 on 56823] ipc.Server (Server.java:run(2290)) - IPC Server handler 3 on 56823: exiting 2015-11-22 19:30:07,588 DEBUG [IPC Server handler 7 on 56823] ipc.Server (Server.java:run(2290)) - IPC Server handler 7 on 56823: exiting 2015-11-22 19:30:07,590 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #1: Service org.apache.hadoop.yarn.server.timeline.security.TimelineDelegationTokenSecretManagerService in state org.apache.hadoop.yarn.server.timeline.security.TimelineDelegationTokenSecretManagerService: STARTED 2015-11-22 19:30:07,592 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.timeline.security.TimelineDelegationTokenSecretManagerService entered state STOPPED 2015-11-22 19:30:07,592 DEBUG [main] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:stopThreads(619)) - Stopping expired delegation token remover thread 2015-11-22 19:30:07,592 ERROR [Thread[Thread-4104,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(670)) - ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted 2015-11-22 19:30:07,592 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #0: Service org.apache.hadoop.yarn.server.timeline.MemoryTimelineStore in state org.apache.hadoop.yarn.server.timeline.MemoryTimelineStore: STARTED 2015-11-22 19:30:07,592 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.timeline.MemoryTimelineStore entered state STOPPED 2015-11-22 19:30:07,592 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #1: Service org.apache.hadoop.yarn.server.MiniYARNCluster$NodeManagerWrapper_0 in state org.apache.hadoop.yarn.server.MiniYARNCluster$NodeManagerWrapper_0: STARTED 2015-11-22 19:30:07,592 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.MiniYARNCluster$NodeManagerWrapper_0 entered state STOPPED 2015-11-22 19:30:07,592 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: NodeManager entered state STOPPED 2015-11-22 19:30:07,593 DEBUG [main] service.CompositeService (CompositeService.java:serviceStop(129)) - NodeManager: stopping services, size=7 2015-11-22 19:30:07,593 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #6: Service org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl in state org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: STARTED 2015-11-22 19:30:07,593 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl entered state STOPPED 2015-11-22 19:30:07,593 INFO [main] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:unRegisterNM(268)) - Successfully Unregistered the Node ip-172-31-20-42.ap-northeast-1.compute.internal:35735 with ResourceManager. 2015-11-22 19:30:07,593 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #5: Service Dispatcher in state Dispatcher: STARTED 2015-11-22 19:30:07,593 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: Dispatcher entered state STOPPED 2015-11-22 19:30:07,594 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #4: Service org.apache.hadoop.yarn.server.nodemanager.webapp.WebServer in state org.apache.hadoop.yarn.server.nodemanager.webapp.WebServer: STARTED 2015-11-22 19:30:07,594 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.nodemanager.webapp.WebServer entered state STOPPED 2015-11-22 19:30:07,594 DEBUG [main] webapp.WebServer (WebServer.java:serviceStop(99)) - Stopping webapp 2015-11-22 19:30:07,596 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.mortbay.jetty.nio.SelectChannelConnector$1@26525736 2015-11-22 19:30:07,596 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping org.mortbay.jetty.webapp.WebAppContext@1f7a38df{/,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes/webapps/node} 2015-11-22 19:30:07,597 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping SessionHandler@3368da8e 2015-11-22 19:30:07,597 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping SecurityHandler@6ebffb2d 2015-11-22 19:30:07,597 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping ServletHandler@7232d296 2015-11-22 19:30:07,597 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped guice 2015-11-22 19:30:07,597 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped Timeline Authentication Filter 2015-11-22 19:30:07,597 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped static_user_filter 2015-11-22 19:30:07,597 DEBUG [main] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:stopThreads(619)) - Stopping expired delegation token remover thread 2015-11-22 19:30:07,598 ERROR [Thread[Thread-4097,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(670)) - ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted 2015-11-22 19:30:07,598 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped RMAuthenticationFilter 2015-11-22 19:30:07,598 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped safety 2015-11-22 19:30:07,598 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped NoCacheFilter 2015-11-22 19:30:07,598 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped NoCacheFilter 2015-11-22 19:30:07,598 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped conf 2015-11-22 19:30:07,598 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped jmx 2015-11-22 19:30:07,598 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped metrics 2015-11-22 19:30:07,598 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped logLevel 2015-11-22 19:30:07,598 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped stacks 2015-11-22 19:30:07,599 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.mortbay.jetty.servlet.DefaultServlet$NIOResourceCache@740a7a1d 2015-11-22 19:30:07,599 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.mortbay.jetty.servlet.DefaultServlet-608661613 2015-11-22 19:30:07,599 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped ServletHandler@7232d296 2015-11-22 19:30:07,599 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped SecurityHandler@6ebffb2d 2015-11-22 19:30:07,599 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.mortbay.jetty.servlet.HashSessionManager@7d1aa0fc 2015-11-22 19:30:07,599 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped SessionHandler@3368da8e 2015-11-22 19:30:07,599 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping ErrorPageErrorHandler@19d87bab 2015-11-22 19:30:07,599 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped ErrorPageErrorHandler@19d87bab 2015-11-22 19:30:07,600 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 - guice as filter 2015-11-22 19:30:07,600 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 - Timeline Authentication Filter as filter 2015-11-22 19:30:07,600 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 - static_user_filter as filter 2015-11-22 19:30:07,600 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 - RMAuthenticationFilter as filter 2015-11-22 19:30:07,600 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 - safety as filter 2015-11-22 19:30:07,600 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 - NoCacheFilter as filter 2015-11-22 19:30:07,600 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 - NoCacheFilter as filter 2015-11-22 19:30:07,601 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 - (F=guice,[/*],[],15) as filterMapping 2015-11-22 19:30:07,601 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 - (F=static_user_filter,[/ws/*],[],15) as filterMapping 2015-11-22 19:30:07,601 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 - (F=RMAuthenticationFilter,[/ws/*],[],15) as filterMapping 2015-11-22 19:30:07,601 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 - (F=static_user_filter,[/node/*],[],15) as filterMapping 2015-11-22 19:30:07,601 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 - (F=RMAuthenticationFilter,[/node/*],[],15) as filterMapping 2015-11-22 19:30:07,601 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 - (F=static_user_filter,[/conf],[],15) as filterMapping 2015-11-22 19:30:07,601 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 - (F=RMAuthenticationFilter,[/conf],[],15) as filterMapping 2015-11-22 19:30:07,601 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 - (F=static_user_filter,[/jmx],[],15) as filterMapping 2015-11-22 19:30:07,601 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 - (F=RMAuthenticationFilter,[/jmx],[],15) as filterMapping 2015-11-22 19:30:07,602 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 - (F=static_user_filter,[/metrics],[],15) as filterMapping 2015-11-22 19:30:07,602 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 - (F=RMAuthenticationFilter,[/metrics],[],15) as filterMapping 2015-11-22 19:30:07,602 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 - (F=static_user_filter,[/logLevel],[],15) as filterMapping 2015-11-22 19:30:07,602 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 - (F=RMAuthenticationFilter,[/logLevel],[],15) as filterMapping 2015-11-22 19:30:07,602 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 - (F=static_user_filter,[/stacks],[],15) as filterMapping 2015-11-22 19:30:07,602 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 - (F=RMAuthenticationFilter,[/stacks],[],15) as filterMapping 2015-11-22 19:30:07,602 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 - (F=Timeline Authentication Filter,[/*],[],15) as filterMapping 2015-11-22 19:30:07,602 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 - (F=static_user_filter,[*.html, *.jsp],[],15) as filterMapping 2015-11-22 19:30:07,603 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 - (F=RMAuthenticationFilter,[*.html, *.jsp],[],15) as filterMapping 2015-11-22 19:30:07,603 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 - (F=safety,[/*],[],15) as filterMapping 2015-11-22 19:30:07,603 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 - (F=NoCacheFilter,[/*],[],15) as filterMapping 2015-11-22 19:30:07,603 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 - (F=NoCacheFilter,[/*],[],15) as filterMapping 2015-11-22 19:30:07,603 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap=null 2015-11-22 19:30:07,603 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=null 2015-11-22 19:30:07,603 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:30:07,603 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/jmx=jmx, /conf=conf, /stacks=stacks, /logLevel=logLevel, /metrics=metrics, /=org.mortbay.jetty.servlet.DefaultServlet-608661613} 2015-11-22 19:30:07,604 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={org.mortbay.jetty.servlet.DefaultServlet-608661613=org.mortbay.jetty.servlet.DefaultServlet-608661613, logLevel=logLevel, jmx=jmx, stacks=stacks, conf=conf, metrics=metrics} 2015-11-22 19:30:07,604 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 - conf as servlet 2015-11-22 19:30:07,604 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 - jmx as servlet 2015-11-22 19:30:07,604 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 - metrics as servlet 2015-11-22 19:30:07,604 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 - logLevel as servlet 2015-11-22 19:30:07,604 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 - stacks as servlet 2015-11-22 19:30:07,604 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 - org.mortbay.jetty.servlet.DefaultServlet-608661613 as servlet 2015-11-22 19:30:07,604 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 - (S=conf,[/conf]) as servletMapping 2015-11-22 19:30:07,604 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 - (S=jmx,[/jmx]) as servletMapping 2015-11-22 19:30:07,605 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 - (S=metrics,[/metrics]) as servletMapping 2015-11-22 19:30:07,605 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 - (S=logLevel,[/logLevel]) as servletMapping 2015-11-22 19:30:07,605 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 - (S=stacks,[/stacks]) as servletMapping 2015-11-22 19:30:07,605 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@7232d296 - (S=org.mortbay.jetty.servlet.DefaultServlet-608661613,[/]) as servletMapping 2015-11-22 19:30:07,605 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap=null 2015-11-22 19:30:07,605 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=null 2015-11-22 19:30:07,605 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:30:07,605 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap=null 2015-11-22 19:30:07,606 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap=null 2015-11-22 19:30:07,606 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.mortbay.jetty.webapp.WebAppContext@1f7a38df{/,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes/webapps/node} 2015-11-22 19:30:07,606 INFO [main] mortbay.log (Slf4jLog.java:info(67)) - Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:0 2015-11-22 19:30:07,606 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:0 2015-11-22 19:30:07,606 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping Server@23e7fedb 2015-11-22 19:30:07,606 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping ContextHandlerCollection@3c3c36d2 2015-11-22 19:30:07,606 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping org.mortbay.jetty.servlet.Context@f8cf43d{/static,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes/webapps/static/} 2015-11-22 19:30:07,607 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping ServletHandler@7b3ede0f 2015-11-22 19:30:07,607 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.mortbay.jetty.servlet.DefaultServlet-851547541 2015-11-22 19:30:07,607 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped ServletHandler@7b3ede0f 2015-11-22 19:30:07,607 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping ErrorHandler@48efb6c1 2015-11-22 19:30:07,607 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped ErrorHandler@48efb6c1 2015-11-22 19:30:07,607 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.mortbay.jetty.servlet.Context@f8cf43d{/static,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes/webapps/static/} 2015-11-22 19:30:07,608 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping org.mortbay.jetty.servlet.Context@82f9d2e{/logs,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/log} 2015-11-22 19:30:07,608 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping ServletHandler@7de68a0f 2015-11-22 19:30:07,608 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.apache.hadoop.http.AdminAuthorizedServlet-1439946767 2015-11-22 19:30:07,608 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped ServletHandler@7de68a0f 2015-11-22 19:30:07,608 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping ErrorHandler@101e5de1 2015-11-22 19:30:07,608 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped ErrorHandler@101e5de1 2015-11-22 19:30:07,609 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.mortbay.jetty.servlet.Context@82f9d2e{/logs,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/log} 2015-11-22 19:30:07,609 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped ContextHandlerCollection@3c3c36d2 2015-11-22 19:30:07,609 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.mortbay.jetty.servlet.HashSessionIdManager@4f1268d0 2015-11-22 19:30:07,709 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.mortbay.thread.QueuedThreadPool@7d421cea 2015-11-22 19:30:07,709 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped Server@23e7fedb 2015-11-22 19:30:07,710 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #3: Service org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl in state org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl: STARTED 2015-11-22 19:30:07,710 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl entered state STOPPED 2015-11-22 19:30:07,710 INFO [main] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:cleanUpApplicationsOnNMShutDown(556)) - Applications still running : [application_1448220594280_0001] 2015-11-22 19:30:07,710 INFO [main] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:cleanUpApplicationsOnNMShutDown(572)) - Waiting for Applications to be Finished 2015-11-22 19:30:07,710 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationFinishEvent.EventType: FINISH_APPLICATION 2015-11-22 19:30:07,710 DEBUG [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(452)) - Processing application_1448220594280_0001 of type FINISH_APPLICATION 2015-11-22 19:30:07,710 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(463)) - Application application_1448220594280_0001 transitioned from RUNNING to APPLICATION_RESOURCES_CLEANINGUP 2015-11-22 19:30:07,710 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.ApplicationLocalizationEvent.EventType: DESTROY_APPLICATION_RESOURCES 2015-11-22 19:30:07,711 DEBUG [DeletionService #0] nodemanager.DeletionService (DeletionService.java:run(266)) - FileDeletionTask : user : ubuntu subDir : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220594280_0001 baseDir : null 2015-11-22 19:30:07,711 DEBUG [DeletionService #0] nodemanager.DeletionService (DeletionService.java:run(292)) - Deleting path: [/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220594280_0001] as user: [ubuntu] 2015-11-22 19:30:07,711 INFO [DeletionService #0] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(480)) - Deleting absolute path : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/usercache/ubuntu/appcache/application_1448220594280_0001 2015-11-22 19:30:07,711 DEBUG [DeletionService #1] nodemanager.DeletionService (DeletionService.java:run(266)) - FileDeletionTask : user : null subDir : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/application_1448220594280_0001 baseDir : null 2015-11-22 19:30:07,711 DEBUG [DeletionService #1] nodemanager.DeletionService (DeletionService.java:run(271)) - NM deleting absolute path : /home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/TestDistributedShell/TestDistributedShell-localDir-nm-0_0/nmPrivate/application_1448220594280_0001 2015-11-22 19:30:07,711 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServicesEvent.EventType: APPLICATION_STOP 2015-11-22 19:30:07,711 INFO [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(196)) - Got event APPLICATION_STOP for appId application_1448220594280_0001 2015-11-22 19:30:07,711 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationEvent.EventType: APPLICATION_RESOURCES_CLEANEDUP 2015-11-22 19:30:07,711 DEBUG [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(452)) - Processing application_1448220594280_0001 of type APPLICATION_RESOURCES_CLEANEDUP 2015-11-22 19:30:07,711 DEBUG [AsyncDispatcher event handler] security.NMTokenSecretManagerInNM (NMTokenSecretManagerInNM.java:appFinished(191)) - Removing application attempts NMToken keys for application application_1448220594280_0001 2015-11-22 19:30:07,712 INFO [AsyncDispatcher event handler] application.ApplicationImpl (ApplicationImpl.java:handle(463)) - Application application_1448220594280_0001 transitioned from APPLICATION_RESOURCES_CLEANINGUP to FINISHED 2015-11-22 19:30:07,712 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(174)) - Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.event.LogHandlerAppFinishedEvent.EventType: APPLICATION_FINISHED 2015-11-22 19:30:07,712 INFO [AsyncDispatcher event handler] loghandler.NonAggregatingLogHandler (NonAggregatingLogHandler.java:handle(167)) - Scheduling Log Deletion for application: application_1448220594280_0001, with delay of 10800 seconds 2015-11-22 19:30:11,042 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:35735 from appattempt_1448220594280_0001_000001] ipc.Client (Client.java:close(1217)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:35735 from appattempt_1448220594280_0001_000001: closed 2015-11-22 19:30:11,042 DEBUG [Socket Reader #1 for port 35735] ipc.Server (Server.java:close(2894)) - Socket Reader #1 for port 35735: disconnecting client 172.31.20.42:47740. Number of active connections: 0 2015-11-22 19:30:11,042 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:35735 from appattempt_1448220594280_0001_000001] ipc.Client (Client.java:run(1018)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:35735 from appattempt_1448220594280_0001_000001: stopped, remaining connections 2 2015-11-22 19:30:11,711 INFO [main] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:cleanUpApplicationsOnNMShutDown(589)) - Done waiting for Applications to be Finished. Still alive: [application_1448220594280_0001] 2015-11-22 19:30:11,711 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 35735 2015-11-22 19:30:11,711 DEBUG [IPC Server handler 0 on 35735] ipc.Server (Server.java:run(2290)) - IPC Server handler 0 on 35735: exiting 2015-11-22 19:30:11,712 DEBUG [IPC Server handler 2 on 35735] ipc.Server (Server.java:run(2290)) - IPC Server handler 2 on 35735: exiting 2015-11-22 19:30:11,712 DEBUG [IPC Server handler 1 on 35735] ipc.Server (Server.java:run(2290)) - IPC Server handler 1 on 35735: exiting 2015-11-22 19:30:11,712 DEBUG [IPC Server handler 3 on 35735] ipc.Server (Server.java:run(2290)) - IPC Server handler 3 on 35735: exiting 2015-11-22 19:30:11,713 DEBUG [IPC Server handler 4 on 35735] ipc.Server (Server.java:run(2290)) - IPC Server handler 4 on 35735: exiting 2015-11-22 19:30:11,713 DEBUG [IPC Server handler 5 on 35735] ipc.Server (Server.java:run(2290)) - IPC Server handler 5 on 35735: exiting 2015-11-22 19:30:11,713 DEBUG [IPC Server handler 6 on 35735] ipc.Server (Server.java:run(2290)) - IPC Server handler 6 on 35735: exiting 2015-11-22 19:30:11,713 DEBUG [IPC Server handler 7 on 35735] ipc.Server (Server.java:run(2290)) - IPC Server handler 7 on 35735: exiting 2015-11-22 19:30:11,716 DEBUG [IPC Server handler 9 on 35735] ipc.Server (Server.java:run(2290)) - IPC Server handler 9 on 35735: exiting 2015-11-22 19:30:11,716 DEBUG [IPC Server handler 8 on 35735] ipc.Server (Server.java:run(2290)) - IPC Server handler 8 on 35735: exiting 2015-11-22 19:30:11,718 DEBUG [IPC Server handler 14 on 35735] ipc.Server (Server.java:run(2290)) - IPC Server handler 14 on 35735: exiting 2015-11-22 19:30:11,718 DEBUG [IPC Server handler 15 on 35735] ipc.Server (Server.java:run(2290)) - IPC Server handler 15 on 35735: exiting 2015-11-22 19:30:11,718 DEBUG [main] service.CompositeService (CompositeService.java:serviceStop(129)) - org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl: stopping services, size=7 2015-11-22 19:30:11,718 DEBUG [IPC Server handler 18 on 35735] ipc.Server (Server.java:run(2290)) - IPC Server handler 18 on 35735: exiting 2015-11-22 19:30:11,718 DEBUG [IPC Server handler 13 on 35735] ipc.Server (Server.java:run(2290)) - IPC Server handler 13 on 35735: exiting 2015-11-22 19:30:11,718 DEBUG [IPC Server Responder] ipc.Server (Server.java:doRunLoop(997)) - Checking for old call responses. 2015-11-22 19:30:11,719 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #6: Service org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.sharedcache.SharedCacheUploadService in state org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.sharedcache.SharedCacheUploadService: STARTED 2015-11-22 19:30:11,718 INFO [IPC Server listener on 35735] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 35735 2015-11-22 19:30:11,718 DEBUG [IPC Server handler 12 on 35735] ipc.Server (Server.java:run(2290)) - IPC Server handler 12 on 35735: exiting 2015-11-22 19:30:11,716 DEBUG [IPC Server handler 11 on 35735] ipc.Server (Server.java:run(2290)) - IPC Server handler 11 on 35735: exiting 2015-11-22 19:30:11,716 DEBUG [IPC Server handler 10 on 35735] ipc.Server (Server.java:run(2290)) - IPC Server handler 10 on 35735: exiting 2015-11-22 19:30:11,719 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.sharedcache.SharedCacheUploadService entered state STOPPED 2015-11-22 19:30:11,719 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:30:11,718 DEBUG [IPC Server handler 19 on 35735] ipc.Server (Server.java:run(2290)) - IPC Server handler 19 on 35735: exiting 2015-11-22 19:30:11,718 DEBUG [IPC Server handler 17 on 35735] ipc.Server (Server.java:run(2290)) - IPC Server handler 17 on 35735: exiting 2015-11-22 19:30:11,718 DEBUG [IPC Server handler 16 on 35735] ipc.Server (Server.java:run(2290)) - IPC Server handler 16 on 35735: exiting 2015-11-22 19:30:11,719 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #5: Service org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.NonAggregatingLogHandler in state org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.NonAggregatingLogHandler: STARTED 2015-11-22 19:30:11,720 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.NonAggregatingLogHandler entered state STOPPED 2015-11-22 19:30:12,121 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:40868 from ubuntu] ipc.Client (Client.java:close(1217)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:40868 from ubuntu: closed 2015-11-22 19:30:12,122 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:40868 from ubuntu] ipc.Client (Client.java:run(1018)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:40868 from ubuntu: stopped, remaining connections 1 2015-11-22 19:30:12,121 DEBUG [Socket Reader #1 for port 40868] ipc.Server (Server.java:close(2894)) - Socket Reader #1 for port 40868: disconnecting client 172.31.20.42:49300. Number of active connections: 0 2015-11-22 19:30:14,294 DEBUG [IPC Server idle connection scanner for port 50902] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 50902: task running 2015-11-22 19:30:14,308 DEBUG [IPC Server idle connection scanner for port 53664] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 53664: task running 2015-11-22 19:30:14,323 DEBUG [IPC Server idle connection scanner for port 38037] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 38037: task running 2015-11-22 19:30:14,533 DEBUG [IPC Server idle connection scanner for port 38415] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 38415: task running 2015-11-22 19:30:15,810 DEBUG [IPC Server idle connection scanner for port 40868] ipc.Server (Server.java:run(2955)) - IPC Server idle connection scanner for port 40868: task running 2015-11-22 19:30:17,464 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:38037 from ubuntu] ipc.Client (Client.java:close(1217)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:38037 from ubuntu: closed 2015-11-22 19:30:17,464 DEBUG [IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:38037 from ubuntu] ipc.Client (Client.java:run(1018)) - IPC Client (273924) connection to ip-172-31-20-42.ap-northeast-1.compute.internal/172.31.20.42:38037 from ubuntu: stopped, remaining connections 0 2015-11-22 19:30:17,464 DEBUG [Socket Reader #1 for port 38037] ipc.Server (Server.java:close(2894)) - Socket Reader #1 for port 38037: disconnecting client 172.31.20.42:35310. Number of active connections: 0 2015-11-22 19:30:21,720 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #4: Service Dispatcher in state Dispatcher: STARTED 2015-11-22 19:30:21,721 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: Dispatcher entered state STOPPED 2015-11-22 19:30:21,722 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #3: Service containers-monitor in state containers-monitor: STARTED 2015-11-22 19:30:21,722 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: containers-monitor entered state STOPPED 2015-11-22 19:30:21,722 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #2: Service org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices in state org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices: STARTED 2015-11-22 19:30:21,722 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices entered state STOPPED 2015-11-22 19:30:21,722 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #1: Service containers-launcher in state containers-launcher: STARTED 2015-11-22 19:30:21,722 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: containers-launcher entered state STOPPED 2015-11-22 19:30:21,722 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #0: Service org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService in state org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: STARTED 2015-11-22 19:30:21,723 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService entered state STOPPED 2015-11-22 19:30:21,723 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 40868 2015-11-22 19:30:21,723 DEBUG [IPC Server handler 0 on 40868] ipc.Server (Server.java:run(2290)) - IPC Server handler 0 on 40868: exiting 2015-11-22 19:30:21,723 DEBUG [IPC Server handler 1 on 40868] ipc.Server (Server.java:run(2290)) - IPC Server handler 1 on 40868: exiting 2015-11-22 19:30:21,723 DEBUG [IPC Server handler 4 on 40868] ipc.Server (Server.java:run(2290)) - IPC Server handler 4 on 40868: exiting 2015-11-22 19:30:21,723 DEBUG [IPC Server handler 3 on 40868] ipc.Server (Server.java:run(2290)) - IPC Server handler 3 on 40868: exiting 2015-11-22 19:30:21,723 DEBUG [IPC Server handler 2 on 40868] ipc.Server (Server.java:run(2290)) - IPC Server handler 2 on 40868: exiting 2015-11-22 19:30:21,724 INFO [IPC Server listener on 40868] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 40868 2015-11-22 19:30:21,724 DEBUG [IPC Server Responder] ipc.Server (Server.java:doRunLoop(997)) - Checking for old call responses. 2015-11-22 19:30:21,724 DEBUG [main] service.CompositeService (CompositeService.java:serviceStop(129)) - org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: stopping services, size=1 2015-11-22 19:30:21,724 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:30:21,725 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #0: Service org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerTracker in state org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerTracker: STARTED 2015-11-22 19:30:21,725 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerTracker entered state STOPPED 2015-11-22 19:30:21,725 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #2: Service org.apache.hadoop.yarn.server.nodemanager.NodeResourceMonitorImpl in state org.apache.hadoop.yarn.server.nodemanager.NodeResourceMonitorImpl: STARTED 2015-11-22 19:30:21,725 INFO [Public Localizer] localizer.ResourceLocalizationService (ResourceLocalizationService.java:run(908)) - Public cache exiting 2015-11-22 19:30:21,725 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.nodemanager.NodeResourceMonitorImpl entered state STOPPED 2015-11-22 19:30:21,726 WARN [Node Resource Monitor] nodemanager.NodeResourceMonitorImpl (NodeResourceMonitorImpl.java:run(154)) - org.apache.hadoop.yarn.server.nodemanager.NodeResourceMonitorImpl is interrupted. Exiting. 2015-11-22 19:30:21,726 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #1: Service org.apache.hadoop.yarn.server.nodemanager.NodeHealthCheckerService in state org.apache.hadoop.yarn.server.nodemanager.NodeHealthCheckerService: STARTED 2015-11-22 19:30:21,726 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.nodemanager.NodeHealthCheckerService entered state STOPPED 2015-11-22 19:30:21,727 DEBUG [main] service.CompositeService (CompositeService.java:serviceStop(129)) - org.apache.hadoop.yarn.server.nodemanager.NodeHealthCheckerService: stopping services, size=1 2015-11-22 19:30:21,727 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #0: Service org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService in state org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService: STARTED 2015-11-22 19:30:21,727 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService entered state STOPPED 2015-11-22 19:30:21,727 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #0: Service org.apache.hadoop.yarn.server.nodemanager.DeletionService in state org.apache.hadoop.yarn.server.nodemanager.DeletionService: STARTED 2015-11-22 19:30:21,727 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.nodemanager.DeletionService entered state STOPPED 2015-11-22 19:30:21,727 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:shutdown(588)) - refCount=2 2015-11-22 19:30:21,728 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.nodemanager.recovery.NMNullStateStoreService entered state STOPPED 2015-11-22 19:30:21,729 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #0: Service org.apache.hadoop.yarn.server.MiniYARNCluster$ResourceManagerWrapper_0 in state org.apache.hadoop.yarn.server.MiniYARNCluster$ResourceManagerWrapper_0: STARTED 2015-11-22 19:30:21,729 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.MiniYARNCluster$ResourceManagerWrapper_0 entered state STOPPED 2015-11-22 19:30:21,729 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: ResourceManager entered state STOPPED 2015-11-22 19:30:21,730 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.mortbay.jetty.nio.SelectChannelConnector$1@1b6186d1 2015-11-22 19:30:21,730 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping org.mortbay.jetty.webapp.WebAppContext@5b1e5299{/,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes/webapps/cluster} 2015-11-22 19:30:21,731 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping SessionHandler@6f2a8d0b 2015-11-22 19:30:21,731 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping SecurityHandler@48150f6a 2015-11-22 19:30:21,731 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping ServletHandler@38de1a90 2015-11-22 19:30:21,731 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped guice 2015-11-22 19:30:21,731 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped Timeline Authentication Filter 2015-11-22 19:30:21,731 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped static_user_filter 2015-11-22 19:30:21,732 DEBUG [main] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:stopThreads(619)) - Stopping expired delegation token remover thread 2015-11-22 19:30:21,732 ERROR [Thread[Thread-4047,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(670)) - ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted 2015-11-22 19:30:21,732 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped RMAuthenticationFilter 2015-11-22 19:30:21,732 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped safety 2015-11-22 19:30:21,732 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped NoCacheFilter 2015-11-22 19:30:21,732 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped NoCacheFilter 2015-11-22 19:30:21,732 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped proxy 2015-11-22 19:30:21,732 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped conf 2015-11-22 19:30:21,733 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped jmx 2015-11-22 19:30:21,733 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped metrics 2015-11-22 19:30:21,733 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped logLevel 2015-11-22 19:30:21,733 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped stacks 2015-11-22 19:30:21,733 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.mortbay.jetty.servlet.DefaultServlet$NIOResourceCache@290906b6 2015-11-22 19:30:21,733 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.mortbay.jetty.servlet.DefaultServlet-929987349 2015-11-22 19:30:21,733 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped ServletHandler@38de1a90 2015-11-22 19:30:21,733 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped SecurityHandler@48150f6a 2015-11-22 19:30:21,734 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.mortbay.jetty.servlet.HashSessionManager@468980ed 2015-11-22 19:30:21,734 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped SessionHandler@6f2a8d0b 2015-11-22 19:30:21,734 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping ErrorPageErrorHandler@e4b0836 2015-11-22 19:30:21,734 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped ErrorPageErrorHandler@e4b0836 2015-11-22 19:30:21,734 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 - guice as filter 2015-11-22 19:30:21,734 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 - Timeline Authentication Filter as filter 2015-11-22 19:30:21,735 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 - static_user_filter as filter 2015-11-22 19:30:21,735 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 - RMAuthenticationFilter as filter 2015-11-22 19:30:21,735 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 - safety as filter 2015-11-22 19:30:21,735 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 - NoCacheFilter as filter 2015-11-22 19:30:21,735 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 - NoCacheFilter as filter 2015-11-22 19:30:21,735 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 - (F=guice,[/*],[],15) as filterMapping 2015-11-22 19:30:21,735 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 - (F=static_user_filter,[/proxy/*],[],15) as filterMapping 2015-11-22 19:30:21,735 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 - (F=RMAuthenticationFilter,[/proxy/*],[],15) as filterMapping 2015-11-22 19:30:21,735 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 - (F=static_user_filter,[/ws/*],[],15) as filterMapping 2015-11-22 19:30:21,736 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 - (F=RMAuthenticationFilter,[/ws/*],[],15) as filterMapping 2015-11-22 19:30:21,736 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 - (F=static_user_filter,[/cluster/*],[],15) as filterMapping 2015-11-22 19:30:21,736 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 - (F=RMAuthenticationFilter,[/cluster/*],[],15) as filterMapping 2015-11-22 19:30:21,736 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 - (F=static_user_filter,[/conf],[],15) as filterMapping 2015-11-22 19:30:21,736 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 - (F=RMAuthenticationFilter,[/conf],[],15) as filterMapping 2015-11-22 19:30:21,736 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 - (F=static_user_filter,[/jmx],[],15) as filterMapping 2015-11-22 19:30:21,736 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 - (F=RMAuthenticationFilter,[/jmx],[],15) as filterMapping 2015-11-22 19:30:21,736 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 - (F=static_user_filter,[/metrics],[],15) as filterMapping 2015-11-22 19:30:21,737 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 - (F=RMAuthenticationFilter,[/metrics],[],15) as filterMapping 2015-11-22 19:30:21,737 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 - (F=static_user_filter,[/logLevel],[],15) as filterMapping 2015-11-22 19:30:21,737 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 - (F=RMAuthenticationFilter,[/logLevel],[],15) as filterMapping 2015-11-22 19:30:21,737 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 - (F=static_user_filter,[/stacks],[],15) as filterMapping 2015-11-22 19:30:21,737 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 - (F=RMAuthenticationFilter,[/stacks],[],15) as filterMapping 2015-11-22 19:30:21,737 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 - (F=Timeline Authentication Filter,[/*],[],15) as filterMapping 2015-11-22 19:30:21,737 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 - (F=static_user_filter,[*.html, *.jsp],[],15) as filterMapping 2015-11-22 19:30:21,737 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 - (F=RMAuthenticationFilter,[*.html, *.jsp],[],15) as filterMapping 2015-11-22 19:30:21,737 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 - (F=safety,[/*],[],15) as filterMapping 2015-11-22 19:30:21,738 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 - (F=NoCacheFilter,[/*],[],15) as filterMapping 2015-11-22 19:30:21,738 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 - (F=NoCacheFilter,[/*],[],15) as filterMapping 2015-11-22 19:30:21,738 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap=null 2015-11-22 19:30:21,738 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=null 2015-11-22 19:30:21,738 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:30:21,738 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap={/jmx=jmx, /proxy/*=proxy, /conf=conf, /stacks=stacks, /logLevel=logLevel, /metrics=metrics, /=org.mortbay.jetty.servlet.DefaultServlet-929987349} 2015-11-22 19:30:21,738 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap={proxy=proxy, logLevel=logLevel, jmx=jmx, org.mortbay.jetty.servlet.DefaultServlet-929987349=org.mortbay.jetty.servlet.DefaultServlet-929987349, stacks=stacks, conf=conf, metrics=metrics} 2015-11-22 19:30:21,738 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 - proxy as servlet 2015-11-22 19:30:21,739 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 - conf as servlet 2015-11-22 19:30:21,739 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 - jmx as servlet 2015-11-22 19:30:21,739 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 - metrics as servlet 2015-11-22 19:30:21,739 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 - logLevel as servlet 2015-11-22 19:30:21,739 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 - stacks as servlet 2015-11-22 19:30:21,739 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 - org.mortbay.jetty.servlet.DefaultServlet-929987349 as servlet 2015-11-22 19:30:21,739 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 - (S=proxy,[/proxy/*]) as servletMapping 2015-11-22 19:30:21,739 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 - (S=conf,[/conf]) as servletMapping 2015-11-22 19:30:21,739 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 - (S=jmx,[/jmx]) as servletMapping 2015-11-22 19:30:21,740 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 - (S=metrics,[/metrics]) as servletMapping 2015-11-22 19:30:21,740 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 - (S=logLevel,[/logLevel]) as servletMapping 2015-11-22 19:30:21,740 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 - (S=stacks,[/stacks]) as servletMapping 2015-11-22 19:30:21,740 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - Container ServletHandler@38de1a90 - (S=org.mortbay.jetty.servlet.DefaultServlet-929987349,[/]) as servletMapping 2015-11-22 19:30:21,740 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - filterNameMap=null 2015-11-22 19:30:21,740 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - pathFilters=null 2015-11-22 19:30:21,740 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - servletFilterMap=null 2015-11-22 19:30:21,740 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - servletPathMap=null 2015-11-22 19:30:21,741 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - servletNameMap=null 2015-11-22 19:30:21,741 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.mortbay.jetty.webapp.WebAppContext@5b1e5299{/,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes/webapps/cluster} 2015-11-22 19:30:21,741 INFO [main] mortbay.log (Slf4jLog.java:info(67)) - Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:0 2015-11-22 19:30:21,741 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped HttpServer2$SelectChannelConnectorWithSafeStartup@ip-172-31-20-42.ap-northeast-1.compute.internal:0 2015-11-22 19:30:21,741 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping Server@7ff1e948 2015-11-22 19:30:21,741 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping ContextHandlerCollection@3b039370 2015-11-22 19:30:21,741 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping org.mortbay.jetty.servlet.Context@9647707{/static,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes/webapps/static/} 2015-11-22 19:30:21,742 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping ServletHandler@4ea00d89 2015-11-22 19:30:21,742 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.mortbay.jetty.servlet.DefaultServlet-2126150723 2015-11-22 19:30:21,742 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped ServletHandler@4ea00d89 2015-11-22 19:30:21,742 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping ErrorHandler@3c22152a 2015-11-22 19:30:21,742 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped ErrorHandler@3c22152a 2015-11-22 19:30:21,742 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.mortbay.jetty.servlet.Context@9647707{/static,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes/webapps/static/} 2015-11-22 19:30:21,743 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping org.mortbay.jetty.servlet.Context@6ec538ca{/logs,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/log} 2015-11-22 19:30:21,743 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping ServletHandler@553cb2cc 2015-11-22 19:30:21,743 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.apache.hadoop.http.AdminAuthorizedServlet-1597562478 2015-11-22 19:30:21,743 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped ServletHandler@553cb2cc 2015-11-22 19:30:21,743 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopping ErrorHandler@4557bc67 2015-11-22 19:30:21,743 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped ErrorHandler@4557bc67 2015-11-22 19:30:21,744 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.mortbay.jetty.servlet.Context@6ec538ca{/logs,file:/home/ubuntu/hadoop-dev/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/log} 2015-11-22 19:30:21,744 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped ContextHandlerCollection@3b039370 2015-11-22 19:30:21,744 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.mortbay.jetty.servlet.HashSessionIdManager@122752d1 2015-11-22 19:30:21,845 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped org.mortbay.thread.QueuedThreadPool@7e23a01a 2015-11-22 19:30:21,845 DEBUG [main] mortbay.log (Slf4jLog.java:debug(40)) - stopped Server@7ff1e948 2015-11-22 19:30:21,845 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.ClientRMService entered state STOPPED 2015-11-22 19:30:21,845 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 38037 2015-11-22 19:30:21,845 DEBUG [IPC Server handler 0 on 38037] ipc.Server (Server.java:run(2290)) - IPC Server handler 0 on 38037: exiting 2015-11-22 19:30:21,845 DEBUG [IPC Server handler 2 on 38037] ipc.Server (Server.java:run(2290)) - IPC Server handler 2 on 38037: exiting 2015-11-22 19:30:21,845 DEBUG [IPC Server handler 6 on 38037] ipc.Server (Server.java:run(2290)) - IPC Server handler 6 on 38037: exiting 2015-11-22 19:30:21,845 DEBUG [IPC Server handler 9 on 38037] ipc.Server (Server.java:run(2290)) - IPC Server handler 9 on 38037: exiting 2015-11-22 19:30:21,845 DEBUG [IPC Server handler 11 on 38037] ipc.Server (Server.java:run(2290)) - IPC Server handler 11 on 38037: exiting 2015-11-22 19:30:21,845 DEBUG [IPC Server handler 5 on 38037] ipc.Server (Server.java:run(2290)) - IPC Server handler 5 on 38037: exiting 2015-11-22 19:30:21,845 DEBUG [IPC Server handler 12 on 38037] ipc.Server (Server.java:run(2290)) - IPC Server handler 12 on 38037: exiting 2015-11-22 19:30:21,846 DEBUG [IPC Server handler 16 on 38037] ipc.Server (Server.java:run(2290)) - IPC Server handler 16 on 38037: exiting 2015-11-22 19:30:21,845 DEBUG [IPC Server handler 1 on 38037] ipc.Server (Server.java:run(2290)) - IPC Server handler 1 on 38037: exiting 2015-11-22 19:30:21,846 DEBUG [IPC Server handler 18 on 38037] ipc.Server (Server.java:run(2290)) - IPC Server handler 18 on 38037: exiting 2015-11-22 19:30:21,846 DEBUG [IPC Server handler 13 on 38037] ipc.Server (Server.java:run(2290)) - IPC Server handler 13 on 38037: exiting 2015-11-22 19:30:21,845 DEBUG [IPC Server handler 10 on 38037] ipc.Server (Server.java:run(2290)) - IPC Server handler 10 on 38037: exiting 2015-11-22 19:30:21,845 DEBUG [IPC Server handler 8 on 38037] ipc.Server (Server.java:run(2290)) - IPC Server handler 8 on 38037: exiting 2015-11-22 19:30:21,845 DEBUG [IPC Server handler 7 on 38037] ipc.Server (Server.java:run(2290)) - IPC Server handler 7 on 38037: exiting 2015-11-22 19:30:21,845 DEBUG [IPC Server handler 4 on 38037] ipc.Server (Server.java:run(2290)) - IPC Server handler 4 on 38037: exiting 2015-11-22 19:30:21,845 DEBUG [IPC Server handler 3 on 38037] ipc.Server (Server.java:run(2290)) - IPC Server handler 3 on 38037: exiting 2015-11-22 19:30:21,847 DEBUG [IPC Server handler 28 on 38037] ipc.Server (Server.java:run(2290)) - IPC Server handler 28 on 38037: exiting 2015-11-22 19:30:21,847 DEBUG [IPC Server handler 26 on 38037] ipc.Server (Server.java:run(2290)) - IPC Server handler 26 on 38037: exiting 2015-11-22 19:30:21,847 DEBUG [IPC Server Responder] ipc.Server (Server.java:doRunLoop(997)) - Checking for old call responses. 2015-11-22 19:30:21,847 DEBUG [IPC Server handler 31 on 38037] ipc.Server (Server.java:run(2290)) - IPC Server handler 31 on 38037: exiting 2015-11-22 19:30:21,847 DEBUG [IPC Server handler 14 on 38037] ipc.Server (Server.java:run(2290)) - IPC Server handler 14 on 38037: exiting 2015-11-22 19:30:21,848 DEBUG [IPC Server handler 37 on 38037] ipc.Server (Server.java:run(2290)) - IPC Server handler 37 on 38037: exiting 2015-11-22 19:30:21,847 DEBUG [IPC Server handler 23 on 38037] ipc.Server (Server.java:run(2290)) - IPC Server handler 23 on 38037: exiting 2015-11-22 19:30:21,848 DEBUG [IPC Server handler 33 on 38037] ipc.Server (Server.java:run(2290)) - IPC Server handler 33 on 38037: exiting 2015-11-22 19:30:21,847 DEBUG [IPC Server handler 25 on 38037] ipc.Server (Server.java:run(2290)) - IPC Server handler 25 on 38037: exiting 2015-11-22 19:30:21,848 DEBUG [IPC Server handler 41 on 38037] ipc.Server (Server.java:run(2290)) - IPC Server handler 41 on 38037: exiting 2015-11-22 19:30:21,848 DEBUG [main] service.CompositeService (CompositeService.java:serviceStop(129)) - ResourceManager: stopping services, size=4 2015-11-22 19:30:21,848 DEBUG [IPC Server handler 42 on 38037] ipc.Server (Server.java:run(2290)) - IPC Server handler 42 on 38037: exiting 2015-11-22 19:30:21,847 INFO [IPC Server listener on 38037] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 38037 2015-11-22 19:30:21,847 DEBUG [IPC Server handler 22 on 38037] ipc.Server (Server.java:run(2290)) - IPC Server handler 22 on 38037: exiting 2015-11-22 19:30:21,847 DEBUG [IPC Server handler 20 on 38037] ipc.Server (Server.java:run(2290)) - IPC Server handler 20 on 38037: exiting 2015-11-22 19:30:21,847 DEBUG [IPC Server handler 19 on 38037] ipc.Server (Server.java:run(2290)) - IPC Server handler 19 on 38037: exiting 2015-11-22 19:30:21,847 DEBUG [IPC Server handler 15 on 38037] ipc.Server (Server.java:run(2290)) - IPC Server handler 15 on 38037: exiting 2015-11-22 19:30:21,849 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #3: Service org.apache.hadoop.yarn.server.resourcemanager.metrics.SystemMetricsPublisher in state org.apache.hadoop.yarn.server.resourcemanager.metrics.SystemMetricsPublisher: STARTED 2015-11-22 19:30:21,848 DEBUG [IPC Server handler 49 on 38037] ipc.Server (Server.java:run(2290)) - IPC Server handler 49 on 38037: exiting 2015-11-22 19:30:21,848 DEBUG [IPC Server handler 46 on 38037] ipc.Server (Server.java:run(2290)) - IPC Server handler 46 on 38037: exiting 2015-11-22 19:30:21,848 DEBUG [IPC Server handler 47 on 38037] ipc.Server (Server.java:run(2290)) - IPC Server handler 47 on 38037: exiting 2015-11-22 19:30:21,848 DEBUG [IPC Server handler 48 on 38037] ipc.Server (Server.java:run(2290)) - IPC Server handler 48 on 38037: exiting 2015-11-22 19:30:21,848 DEBUG [IPC Server handler 45 on 38037] ipc.Server (Server.java:run(2290)) - IPC Server handler 45 on 38037: exiting 2015-11-22 19:30:21,848 DEBUG [IPC Server handler 43 on 38037] ipc.Server (Server.java:run(2290)) - IPC Server handler 43 on 38037: exiting 2015-11-22 19:30:21,848 DEBUG [IPC Server handler 44 on 38037] ipc.Server (Server.java:run(2290)) - IPC Server handler 44 on 38037: exiting 2015-11-22 19:30:21,848 DEBUG [IPC Server handler 32 on 38037] ipc.Server (Server.java:run(2290)) - IPC Server handler 32 on 38037: exiting 2015-11-22 19:30:21,848 DEBUG [IPC Server handler 39 on 38037] ipc.Server (Server.java:run(2290)) - IPC Server handler 39 on 38037: exiting 2015-11-22 19:30:21,848 DEBUG [IPC Server handler 40 on 38037] ipc.Server (Server.java:run(2290)) - IPC Server handler 40 on 38037: exiting 2015-11-22 19:30:21,848 DEBUG [IPC Server handler 38 on 38037] ipc.Server (Server.java:run(2290)) - IPC Server handler 38 on 38037: exiting 2015-11-22 19:30:21,848 DEBUG [IPC Server handler 35 on 38037] ipc.Server (Server.java:run(2290)) - IPC Server handler 35 on 38037: exiting 2015-11-22 19:30:21,848 DEBUG [IPC Server handler 34 on 38037] ipc.Server (Server.java:run(2290)) - IPC Server handler 34 on 38037: exiting 2015-11-22 19:30:21,848 DEBUG [IPC Server handler 36 on 38037] ipc.Server (Server.java:run(2290)) - IPC Server handler 36 on 38037: exiting 2015-11-22 19:30:21,847 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:30:21,847 DEBUG [IPC Server handler 30 on 38037] ipc.Server (Server.java:run(2290)) - IPC Server handler 30 on 38037: exiting 2015-11-22 19:30:21,847 DEBUG [IPC Server handler 29 on 38037] ipc.Server (Server.java:run(2290)) - IPC Server handler 29 on 38037: exiting 2015-11-22 19:30:21,847 DEBUG [IPC Server handler 27 on 38037] ipc.Server (Server.java:run(2290)) - IPC Server handler 27 on 38037: exiting 2015-11-22 19:30:21,847 DEBUG [IPC Server handler 21 on 38037] ipc.Server (Server.java:run(2290)) - IPC Server handler 21 on 38037: exiting 2015-11-22 19:30:21,847 DEBUG [IPC Server handler 17 on 38037] ipc.Server (Server.java:run(2290)) - IPC Server handler 17 on 38037: exiting 2015-11-22 19:30:21,847 DEBUG [IPC Server handler 24 on 38037] ipc.Server (Server.java:run(2290)) - IPC Server handler 24 on 38037: exiting 2015-11-22 19:30:21,849 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.metrics.SystemMetricsPublisher entered state STOPPED 2015-11-22 19:30:21,851 DEBUG [main] service.CompositeService (CompositeService.java:serviceStop(129)) - org.apache.hadoop.yarn.server.resourcemanager.metrics.SystemMetricsPublisher: stopping services, size=0 2015-11-22 19:30:21,852 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #2: Service org.apache.hadoop.yarn.server.resourcemanager.ahs.RMApplicationHistoryWriter in state org.apache.hadoop.yarn.server.resourcemanager.ahs.RMApplicationHistoryWriter: STARTED 2015-11-22 19:30:21,852 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.ahs.RMApplicationHistoryWriter entered state STOPPED 2015-11-22 19:30:21,852 DEBUG [main] service.CompositeService (CompositeService.java:serviceStop(129)) - org.apache.hadoop.yarn.server.resourcemanager.ahs.RMApplicationHistoryWriter: stopping services, size=0 2015-11-22 19:30:21,852 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #1: Service org.apache.hadoop.yarn.server.resourcemanager.AdminService in state org.apache.hadoop.yarn.server.resourcemanager.AdminService: STARTED 2015-11-22 19:30:21,852 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.AdminService entered state STOPPED 2015-11-22 19:30:21,852 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 38415 2015-11-22 19:30:21,853 DEBUG [IPC Server handler 0 on 38415] ipc.Server (Server.java:run(2290)) - IPC Server handler 0 on 38415: exiting 2015-11-22 19:30:21,853 DEBUG [IPC Server Responder] ipc.Server (Server.java:doRunLoop(997)) - Checking for old call responses. 2015-11-22 19:30:21,853 DEBUG [main] service.CompositeService (CompositeService.java:serviceStop(129)) - org.apache.hadoop.yarn.server.resourcemanager.AdminService: stopping services, size=0 2015-11-22 19:30:21,853 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:30:21,853 INFO [IPC Server listener on 38415] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 38415 2015-11-22 19:30:21,854 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #0: Service Dispatcher in state Dispatcher: STARTED 2015-11-22 19:30:21,854 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: Dispatcher entered state STOPPED 2015-11-22 19:30:21,854 INFO [main] resourcemanager.ResourceManager (ResourceManager.java:transitionToStandby(1051)) - Transitioning to standby state 2015-11-22 19:30:21,854 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: RMActiveServices entered state STOPPED 2015-11-22 19:30:21,854 DEBUG [main] service.CompositeService (CompositeService.java:serviceStop(129)) - RMActiveServices: stopping services, size=13 2015-11-22 19:30:21,855 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #12: Service org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher in state org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher: STARTED 2015-11-22 19:30:21,855 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher entered state STOPPED 2015-11-22 19:30:21,855 WARN [ApplicationMaster Launcher] amlauncher.ApplicationMasterLauncher (ApplicationMasterLauncher.java:run(122)) - org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher$LauncherThread interrupted. Returning. 2015-11-22 19:30:21,855 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #11: Service org.apache.hadoop.yarn.server.resourcemanager.ClientRMService in state org.apache.hadoop.yarn.server.resourcemanager.ClientRMService: STOPPED 2015-11-22 19:30:21,855 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #10: Service org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService in state org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService: STARTED 2015-11-22 19:30:21,856 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService entered state STOPPED 2015-11-22 19:30:21,856 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 53664 2015-11-22 19:30:21,856 DEBUG [IPC Server handler 0 on 53664] ipc.Server (Server.java:run(2290)) - IPC Server handler 0 on 53664: exiting 2015-11-22 19:30:21,856 DEBUG [IPC Server handler 1 on 53664] ipc.Server (Server.java:run(2290)) - IPC Server handler 1 on 53664: exiting 2015-11-22 19:30:21,856 DEBUG [IPC Server handler 3 on 53664] ipc.Server (Server.java:run(2290)) - IPC Server handler 3 on 53664: exiting 2015-11-22 19:30:21,856 DEBUG [IPC Server handler 5 on 53664] ipc.Server (Server.java:run(2290)) - IPC Server handler 5 on 53664: exiting 2015-11-22 19:30:21,856 DEBUG [IPC Server handler 4 on 53664] ipc.Server (Server.java:run(2290)) - IPC Server handler 4 on 53664: exiting 2015-11-22 19:30:21,857 INFO [IPC Server listener on 53664] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 53664 2015-11-22 19:30:21,857 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #9: Service org.apache.hadoop.yarn.server.resourcemanager.ResourceTrackerService in state org.apache.hadoop.yarn.server.resourcemanager.ResourceTrackerService: STARTED 2015-11-22 19:30:21,857 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.ResourceTrackerService entered state STOPPED 2015-11-22 19:30:21,857 DEBUG [IPC Server handler 9 on 53664] ipc.Server (Server.java:run(2290)) - IPC Server handler 9 on 53664: exiting 2015-11-22 19:30:21,857 DEBUG [IPC Server handler 18 on 53664] ipc.Server (Server.java:run(2290)) - IPC Server handler 18 on 53664: exiting 2015-11-22 19:30:21,857 DEBUG [IPC Server handler 16 on 53664] ipc.Server (Server.java:run(2290)) - IPC Server handler 16 on 53664: exiting 2015-11-22 19:30:21,857 DEBUG [IPC Server handler 27 on 53664] ipc.Server (Server.java:run(2290)) - IPC Server handler 27 on 53664: exiting 2015-11-22 19:30:21,857 INFO [main] ipc.Server (Server.java:stop(2611)) - Stopping server on 50902 2015-11-22 19:30:21,857 DEBUG [IPC Server handler 10 on 53664] ipc.Server (Server.java:run(2290)) - IPC Server handler 10 on 53664: exiting 2015-11-22 19:30:21,858 DEBUG [IPC Server handler 0 on 50902] ipc.Server (Server.java:run(2290)) - IPC Server handler 0 on 50902: exiting 2015-11-22 19:30:21,857 DEBUG [IPC Server handler 8 on 53664] ipc.Server (Server.java:run(2290)) - IPC Server handler 8 on 53664: exiting 2015-11-22 19:30:21,858 DEBUG [IPC Server handler 4 on 50902] ipc.Server (Server.java:run(2290)) - IPC Server handler 4 on 50902: exiting 2015-11-22 19:30:21,857 DEBUG [IPC Server handler 22 on 53664] ipc.Server (Server.java:run(2290)) - IPC Server handler 22 on 53664: exiting 2015-11-22 19:30:21,858 DEBUG [IPC Server handler 35 on 53664] ipc.Server (Server.java:run(2290)) - IPC Server handler 35 on 53664: exiting 2015-11-22 19:30:21,857 DEBUG [IPC Server handler 21 on 53664] ipc.Server (Server.java:run(2290)) - IPC Server handler 21 on 53664: exiting 2015-11-22 19:30:21,857 DEBUG [IPC Server handler 19 on 53664] ipc.Server (Server.java:run(2290)) - IPC Server handler 19 on 53664: exiting 2015-11-22 19:30:21,858 DEBUG [IPC Server handler 6 on 50902] ipc.Server (Server.java:run(2290)) - IPC Server handler 6 on 50902: exiting 2015-11-22 19:30:21,857 DEBUG [IPC Server handler 17 on 53664] ipc.Server (Server.java:run(2290)) - IPC Server handler 17 on 53664: exiting 2015-11-22 19:30:21,858 DEBUG [IPC Server handler 23 on 53664] ipc.Server (Server.java:run(2290)) - IPC Server handler 23 on 53664: exiting 2015-11-22 19:30:21,858 DEBUG [IPC Server handler 8 on 50902] ipc.Server (Server.java:run(2290)) - IPC Server handler 8 on 50902: exiting 2015-11-22 19:30:21,857 DEBUG [IPC Server handler 15 on 53664] ipc.Server (Server.java:run(2290)) - IPC Server handler 15 on 53664: exiting 2015-11-22 19:30:21,857 DEBUG [IPC Server handler 13 on 53664] ipc.Server (Server.java:run(2290)) - IPC Server handler 13 on 53664: exiting 2015-11-22 19:30:21,857 DEBUG [IPC Server handler 11 on 53664] ipc.Server (Server.java:run(2290)) - IPC Server handler 11 on 53664: exiting 2015-11-22 19:30:21,859 DEBUG [IPC Server handler 44 on 53664] ipc.Server (Server.java:run(2290)) - IPC Server handler 44 on 53664: exiting 2015-11-22 19:30:21,857 DEBUG [IPC Server handler 12 on 53664] ipc.Server (Server.java:run(2290)) - IPC Server handler 12 on 53664: exiting 2015-11-22 19:30:21,860 DEBUG [IPC Server handler 28 on 53664] ipc.Server (Server.java:run(2290)) - IPC Server handler 28 on 53664: exiting 2015-11-22 19:30:21,857 DEBUG [IPC Server Responder] ipc.Server (Server.java:doRunLoop(997)) - Checking for old call responses. 2015-11-22 19:30:21,860 DEBUG [IPC Server handler 45 on 53664] ipc.Server (Server.java:run(2290)) - IPC Server handler 45 on 53664: exiting 2015-11-22 19:30:21,860 DEBUG [IPC Server handler 33 on 53664] ipc.Server (Server.java:run(2290)) - IPC Server handler 33 on 53664: exiting 2015-11-22 19:30:21,860 DEBUG [IPC Server Responder] ipc.Server (Server.java:doRunLoop(997)) - Checking for old call responses. 2015-11-22 19:30:21,860 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:30:21,860 DEBUG [IPC Server handler 42 on 53664] ipc.Server (Server.java:run(2290)) - IPC Server handler 42 on 53664: exiting 2015-11-22 19:30:21,860 DEBUG [IPC Server handler 23 on 50902] ipc.Server (Server.java:run(2290)) - IPC Server handler 23 on 50902: exiting 2015-11-22 19:30:21,860 DEBUG [IPC Server handler 46 on 53664] ipc.Server (Server.java:run(2290)) - IPC Server handler 46 on 53664: exiting 2015-11-22 19:30:21,860 DEBUG [IPC Server handler 21 on 50902] ipc.Server (Server.java:run(2290)) - IPC Server handler 21 on 50902: exiting 2015-11-22 19:30:21,860 DEBUG [IPC Server handler 20 on 50902] ipc.Server (Server.java:run(2290)) - IPC Server handler 20 on 50902: exiting 2015-11-22 19:30:21,860 DEBUG [IPC Server handler 17 on 50902] ipc.Server (Server.java:run(2290)) - IPC Server handler 17 on 50902: exiting 2015-11-22 19:30:21,860 DEBUG [IPC Server handler 16 on 50902] ipc.Server (Server.java:run(2290)) - IPC Server handler 16 on 50902: exiting 2015-11-22 19:30:21,860 DEBUG [IPC Server handler 14 on 50902] ipc.Server (Server.java:run(2290)) - IPC Server handler 14 on 50902: exiting 2015-11-22 19:30:21,860 DEBUG [IPC Server handler 15 on 50902] ipc.Server (Server.java:run(2290)) - IPC Server handler 15 on 50902: exiting 2015-11-22 19:30:21,859 DEBUG [IPC Server handler 12 on 50902] ipc.Server (Server.java:run(2290)) - IPC Server handler 12 on 50902: exiting 2015-11-22 19:30:21,859 DEBUG [IPC Server handler 19 on 50902] ipc.Server (Server.java:run(2290)) - IPC Server handler 19 on 50902: exiting 2015-11-22 19:30:21,859 DEBUG [IPC Server handler 10 on 50902] ipc.Server (Server.java:run(2290)) - IPC Server handler 10 on 50902: exiting 2015-11-22 19:30:21,864 DEBUG [IPC Server handler 48 on 53664] ipc.Server (Server.java:run(2290)) - IPC Server handler 48 on 53664: exiting 2015-11-22 19:30:21,864 DEBUG [IPC Server handler 24 on 50902] ipc.Server (Server.java:run(2290)) - IPC Server handler 24 on 50902: exiting 2015-11-22 19:30:21,864 DEBUG [IPC Server handler 26 on 50902] ipc.Server (Server.java:run(2290)) - IPC Server handler 26 on 50902: exiting 2015-11-22 19:30:21,864 DEBUG [IPC Server handler 29 on 50902] ipc.Server (Server.java:run(2290)) - IPC Server handler 29 on 50902: exiting 2015-11-22 19:30:21,864 DEBUG [IPC Server handler 32 on 50902] ipc.Server (Server.java:run(2290)) - IPC Server handler 32 on 50902: exiting 2015-11-22 19:30:21,865 DEBUG [IPC Server handler 38 on 50902] ipc.Server (Server.java:run(2290)) - IPC Server handler 38 on 50902: exiting 2015-11-22 19:30:21,859 DEBUG [IPC Server handler 13 on 50902] ipc.Server (Server.java:run(2290)) - IPC Server handler 13 on 50902: exiting 2015-11-22 19:30:21,865 DEBUG [IPC Server handler 39 on 50902] ipc.Server (Server.java:run(2290)) - IPC Server handler 39 on 50902: exiting 2015-11-22 19:30:21,865 DEBUG [IPC Server handler 43 on 50902] ipc.Server (Server.java:run(2290)) - IPC Server handler 43 on 50902: exiting 2015-11-22 19:30:21,865 DEBUG [IPC Server handler 44 on 50902] ipc.Server (Server.java:run(2290)) - IPC Server handler 44 on 50902: exiting 2015-11-22 19:30:21,865 DEBUG [IPC Server handler 46 on 50902] ipc.Server (Server.java:run(2290)) - IPC Server handler 46 on 50902: exiting 2015-11-22 19:30:21,865 DEBUG [IPC Server handler 49 on 50902] ipc.Server (Server.java:run(2290)) - IPC Server handler 49 on 50902: exiting 2015-11-22 19:30:21,859 DEBUG [IPC Server handler 7 on 50902] ipc.Server (Server.java:run(2290)) - IPC Server handler 7 on 50902: exiting 2015-11-22 19:30:21,859 DEBUG [IPC Server handler 18 on 50902] ipc.Server (Server.java:run(2290)) - IPC Server handler 18 on 50902: exiting 2015-11-22 19:30:21,859 DEBUG [IPC Server handler 9 on 50902] ipc.Server (Server.java:run(2290)) - IPC Server handler 9 on 50902: exiting 2015-11-22 19:30:21,859 DEBUG [IPC Server handler 11 on 50902] ipc.Server (Server.java:run(2290)) - IPC Server handler 11 on 50902: exiting 2015-11-22 19:30:21,858 DEBUG [IPC Server handler 26 on 53664] ipc.Server (Server.java:run(2290)) - IPC Server handler 26 on 53664: exiting 2015-11-22 19:30:21,858 DEBUG [IPC Server handler 25 on 53664] ipc.Server (Server.java:run(2290)) - IPC Server handler 25 on 53664: exiting 2015-11-22 19:30:21,858 DEBUG [IPC Server handler 24 on 53664] ipc.Server (Server.java:run(2290)) - IPC Server handler 24 on 53664: exiting 2015-11-22 19:30:21,858 DEBUG [IPC Server handler 38 on 53664] ipc.Server (Server.java:run(2290)) - IPC Server handler 38 on 53664: exiting 2015-11-22 19:30:21,858 DEBUG [IPC Server handler 36 on 53664] ipc.Server (Server.java:run(2290)) - IPC Server handler 36 on 53664: exiting 2015-11-22 19:30:21,858 DEBUG [IPC Server handler 5 on 50902] ipc.Server (Server.java:run(2290)) - IPC Server handler 5 on 50902: exiting 2015-11-22 19:30:21,858 DEBUG [IPC Server handler 20 on 53664] ipc.Server (Server.java:run(2290)) - IPC Server handler 20 on 53664: exiting 2015-11-22 19:30:21,858 DEBUG [IPC Server handler 3 on 50902] ipc.Server (Server.java:run(2290)) - IPC Server handler 3 on 50902: exiting 2015-11-22 19:30:21,858 DEBUG [IPC Server handler 2 on 50902] ipc.Server (Server.java:run(2290)) - IPC Server handler 2 on 50902: exiting 2015-11-22 19:30:21,858 DEBUG [IPC Server handler 1 on 50902] ipc.Server (Server.java:run(2290)) - IPC Server handler 1 on 50902: exiting 2015-11-22 19:30:21,858 DEBUG [IPC Server handler 14 on 53664] ipc.Server (Server.java:run(2290)) - IPC Server handler 14 on 53664: exiting 2015-11-22 19:30:21,858 DEBUG [IPC Server handler 34 on 53664] ipc.Server (Server.java:run(2290)) - IPC Server handler 34 on 53664: exiting 2015-11-22 19:30:21,857 DEBUG [IPC Server handler 7 on 53664] ipc.Server (Server.java:run(2290)) - IPC Server handler 7 on 53664: exiting 2015-11-22 19:30:21,857 DEBUG [IPC Server handler 31 on 53664] ipc.Server (Server.java:run(2290)) - IPC Server handler 31 on 53664: exiting 2015-11-22 19:30:21,857 DEBUG [IPC Server handler 6 on 53664] ipc.Server (Server.java:run(2290)) - IPC Server handler 6 on 53664: exiting 2015-11-22 19:30:21,857 DEBUG [IPC Server handler 30 on 53664] ipc.Server (Server.java:run(2290)) - IPC Server handler 30 on 53664: exiting 2015-11-22 19:30:21,857 DEBUG [IPC Server handler 2 on 53664] ipc.Server (Server.java:run(2290)) - IPC Server handler 2 on 53664: exiting 2015-11-22 19:30:21,865 DEBUG [IPC Server handler 47 on 50902] ipc.Server (Server.java:run(2290)) - IPC Server handler 47 on 50902: exiting 2015-11-22 19:30:21,865 DEBUG [IPC Server handler 45 on 50902] ipc.Server (Server.java:run(2290)) - IPC Server handler 45 on 50902: exiting 2015-11-22 19:30:21,865 DEBUG [IPC Server handler 48 on 50902] ipc.Server (Server.java:run(2290)) - IPC Server handler 48 on 50902: exiting 2015-11-22 19:30:21,865 INFO [IPC Server listener on 50902] ipc.Server (Server.java:run(826)) - Stopping IPC Server listener on 50902 2015-11-22 19:30:21,865 DEBUG [IPC Server handler 42 on 50902] ipc.Server (Server.java:run(2290)) - IPC Server handler 42 on 50902: exiting 2015-11-22 19:30:21,865 DEBUG [IPC Server handler 41 on 50902] ipc.Server (Server.java:run(2290)) - IPC Server handler 41 on 50902: exiting 2015-11-22 19:30:21,865 DEBUG [IPC Server handler 37 on 50902] ipc.Server (Server.java:run(2290)) - IPC Server handler 37 on 50902: exiting 2015-11-22 19:30:21,865 DEBUG [IPC Server handler 36 on 50902] ipc.Server (Server.java:run(2290)) - IPC Server handler 36 on 50902: exiting 2015-11-22 19:30:21,865 DEBUG [IPC Server handler 40 on 50902] ipc.Server (Server.java:run(2290)) - IPC Server handler 40 on 50902: exiting 2015-11-22 19:30:21,865 DEBUG [IPC Server handler 28 on 50902] ipc.Server (Server.java:run(2290)) - IPC Server handler 28 on 50902: exiting 2015-11-22 19:30:21,864 DEBUG [IPC Server handler 35 on 50902] ipc.Server (Server.java:run(2290)) - IPC Server handler 35 on 50902: exiting 2015-11-22 19:30:21,864 DEBUG [IPC Server handler 34 on 50902] ipc.Server (Server.java:run(2290)) - IPC Server handler 34 on 50902: exiting 2015-11-22 19:30:21,864 DEBUG [IPC Server handler 33 on 50902] ipc.Server (Server.java:run(2290)) - IPC Server handler 33 on 50902: exiting 2015-11-22 19:30:21,864 DEBUG [IPC Server handler 31 on 50902] ipc.Server (Server.java:run(2290)) - IPC Server handler 31 on 50902: exiting 2015-11-22 19:30:21,864 DEBUG [IPC Server handler 30 on 50902] ipc.Server (Server.java:run(2290)) - IPC Server handler 30 on 50902: exiting 2015-11-22 19:30:21,864 DEBUG [IPC Server handler 27 on 50902] ipc.Server (Server.java:run(2290)) - IPC Server handler 27 on 50902: exiting 2015-11-22 19:30:21,864 DEBUG [IPC Server handler 25 on 50902] ipc.Server (Server.java:run(2290)) - IPC Server handler 25 on 50902: exiting 2015-11-22 19:30:21,864 DEBUG [IPC Server handler 22 on 50902] ipc.Server (Server.java:run(2290)) - IPC Server handler 22 on 50902: exiting 2015-11-22 19:30:21,864 DEBUG [IPC Server handler 43 on 53664] ipc.Server (Server.java:run(2290)) - IPC Server handler 43 on 53664: exiting 2015-11-22 19:30:21,864 DEBUG [IPC Server handler 40 on 53664] ipc.Server (Server.java:run(2290)) - IPC Server handler 40 on 53664: exiting 2015-11-22 19:30:21,860 DEBUG [IPC Server handler 41 on 53664] ipc.Server (Server.java:run(2290)) - IPC Server handler 41 on 53664: exiting 2015-11-22 19:30:21,860 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #8: Service NMLivelinessMonitor in state NMLivelinessMonitor: STARTED 2015-11-22 19:30:21,860 DEBUG [IPC Server handler 39 on 53664] ipc.Server (Server.java:run(2290)) - IPC Server handler 39 on 53664: exiting 2015-11-22 19:30:21,860 DEBUG [IPC Server handler 49 on 53664] ipc.Server (Server.java:run(2290)) - IPC Server handler 49 on 53664: exiting 2015-11-22 19:30:21,860 DEBUG [IPC Server handler 37 on 53664] ipc.Server (Server.java:run(2290)) - IPC Server handler 37 on 53664: exiting 2015-11-22 19:30:21,860 DEBUG [IPC Server handler 47 on 53664] ipc.Server (Server.java:run(2290)) - IPC Server handler 47 on 53664: exiting 2015-11-22 19:30:21,860 DEBUG [IPC Server handler 32 on 53664] ipc.Server (Server.java:run(2290)) - IPC Server handler 32 on 53664: exiting 2015-11-22 19:30:21,860 DEBUG [IPC Server handler 29 on 53664] ipc.Server (Server.java:run(2290)) - IPC Server handler 29 on 53664: exiting 2015-11-22 19:30:21,860 INFO [IPC Server Responder] ipc.Server (Server.java:run(959)) - Stopping IPC Server Responder 2015-11-22 19:30:21,869 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: NMLivelinessMonitor entered state STOPPED 2015-11-22 19:30:21,869 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #7: Service org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher in state org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher: STARTED 2015-11-22 19:30:21,870 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher entered state STOPPED 2015-11-22 19:30:21,869 INFO [Ping Checker] util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(135)) - NMLivelinessMonitor thread interrupted 2015-11-22 19:30:21,870 ERROR [ResourceManager Event Processor] resourcemanager.ResourceManager (ResourceManager.java:run(691)) - Returning, interrupted : java.lang.InterruptedException 2015-11-22 19:30:21,870 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #6: Service org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler in state org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler: STARTED 2015-11-22 19:30:21,871 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler entered state STOPPED 2015-11-22 19:30:21,871 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #5: Service org.apache.hadoop.yarn.server.resourcemanager.NodesListManager in state org.apache.hadoop.yarn.server.resourcemanager.NodesListManager: STARTED 2015-11-22 19:30:21,871 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.NodesListManager entered state STOPPED 2015-11-22 19:30:21,871 DEBUG [main] service.CompositeService (CompositeService.java:serviceStop(129)) - org.apache.hadoop.yarn.server.resourcemanager.NodesListManager: stopping services, size=0 2015-11-22 19:30:21,871 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #4: Service org.apache.hadoop.yarn.nodelabels.CommonNodeLabelsManager in state org.apache.hadoop.yarn.nodelabels.CommonNodeLabelsManager: STARTED 2015-11-22 19:30:21,871 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.nodelabels.CommonNodeLabelsManager entered state STOPPED 2015-11-22 19:30:21,871 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: Dispatcher entered state STOPPED 2015-11-22 19:30:21,871 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:serviceStop(141)) - AsyncDispatcher is draining to stop, igonring any new events. 2015-11-22 19:30:21,872 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #3: Service AMLivelinessMonitor in state AMLivelinessMonitor: STARTED 2015-11-22 19:30:21,872 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: AMLivelinessMonitor entered state STOPPED 2015-11-22 19:30:21,872 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #2: Service AMLivelinessMonitor in state AMLivelinessMonitor: STARTED 2015-11-22 19:30:21,872 INFO [Ping Checker] util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(135)) - AMLivelinessMonitor thread interrupted 2015-11-22 19:30:21,872 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: AMLivelinessMonitor entered state STOPPED 2015-11-22 19:30:21,872 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #1: Service org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.ContainerAllocationExpirer in state org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.ContainerAllocationExpirer: STARTED 2015-11-22 19:30:21,873 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.ContainerAllocationExpirer entered state STOPPED 2015-11-22 19:30:21,873 INFO [Ping Checker] util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(135)) - AMLivelinessMonitor thread interrupted 2015-11-22 19:30:21,873 INFO [Ping Checker] util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(135)) - org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.ContainerAllocationExpirer thread interrupted 2015-11-22 19:30:21,873 DEBUG [main] service.CompositeService (CompositeService.java:stop(151)) - Stopping service #0: Service org.apache.hadoop.yarn.server.resourcemanager.RMSecretManagerService in state org.apache.hadoop.yarn.server.resourcemanager.RMSecretManagerService: STARTED 2015-11-22 19:30:21,873 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.RMSecretManagerService entered state STOPPED 2015-11-22 19:30:21,873 DEBUG [main] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:stopThreads(619)) - Stopping expired delegation token remover thread 2015-11-22 19:30:21,873 ERROR [Thread[Thread-3878,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(670)) - ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted 2015-11-22 19:30:21,874 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:shutdown(588)) - refCount=1 2015-11-22 19:30:21,874 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stop(211)) - Stopping ApplicationHistoryServer metrics system... 2015-11-22 19:30:21,874 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stopSources(458)) - Stopping metrics source JvmMetrics: class=class org.apache.hadoop.metrics2.source.JvmMetrics 2015-11-22 19:30:21,874 DEBUG [main] util.MBeans (MBeans.java:unregister(103)) - Unregistering Hadoop:service=ResourceManager,name=JvmMetrics 2015-11-22 19:30:21,874 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stopSources(458)) - Stopping metrics source NodeManagerMetrics: class=class org.apache.hadoop.metrics2.lib.MetricsSourceBuilder$1 2015-11-22 19:30:21,874 DEBUG [main] util.MBeans (MBeans.java:unregister(103)) - Unregistering Hadoop:service=ResourceManager,name=NodeManagerMetrics 2015-11-22 19:30:21,874 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stopSources(458)) - Stopping metrics source ClusterMetrics: class=class org.apache.hadoop.metrics2.lib.MetricsSourceBuilder$1 2015-11-22 19:30:21,875 DEBUG [main] util.MBeans (MBeans.java:unregister(103)) - Unregistering Hadoop:service=ResourceManager,name=ClusterMetrics 2015-11-22 19:30:21,875 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stopSources(458)) - Stopping metrics source QueueMetrics,q0=root: class=class org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CSQueueMetrics 2015-11-22 19:30:21,875 DEBUG [main] util.MBeans (MBeans.java:unregister(103)) - Unregistering Hadoop:service=ResourceManager,name=QueueMetrics,q0=root 2015-11-22 19:30:21,875 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stopSources(458)) - Stopping metrics source QueueMetrics,q0=root,q1=default: class=class org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CSQueueMetrics 2015-11-22 19:30:21,875 DEBUG [main] util.MBeans (MBeans.java:unregister(103)) - Unregistering Hadoop:service=ResourceManager,name=QueueMetrics,q0=root,q1=default 2015-11-22 19:30:21,875 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stopSources(458)) - Stopping metrics source TimelineDataManagerMetrics: class=class org.apache.hadoop.metrics2.lib.MetricsSourceBuilder$1 2015-11-22 19:30:21,875 DEBUG [main] util.MBeans (MBeans.java:unregister(103)) - Unregistering Hadoop:service=ApplicationHistoryServer,name=TimelineDataManagerMetrics 2015-11-22 19:30:21,875 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stopSources(458)) - Stopping metrics source RpcActivityForPort50902: class=class org.apache.hadoop.metrics2.lib.MetricsSourceBuilder$1 2015-11-22 19:30:21,876 DEBUG [main] util.MBeans (MBeans.java:unregister(103)) - Unregistering Hadoop:service=ApplicationHistoryServer,name=RpcActivityForPort50902 2015-11-22 19:30:21,876 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stopSources(458)) - Stopping metrics source RpcDetailedActivityForPort50902: class=class org.apache.hadoop.metrics2.lib.MetricsSourceBuilder$1 2015-11-22 19:30:21,876 DEBUG [main] util.MBeans (MBeans.java:unregister(103)) - Unregistering Hadoop:service=ApplicationHistoryServer,name=RpcDetailedActivityForPort50902 2015-11-22 19:30:21,876 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stopSources(458)) - Stopping metrics source RpcActivityForPort53664: class=class org.apache.hadoop.metrics2.lib.MetricsSourceBuilder$1 2015-11-22 19:30:21,876 DEBUG [main] util.MBeans (MBeans.java:unregister(103)) - Unregistering Hadoop:service=ApplicationHistoryServer,name=RpcActivityForPort53664 2015-11-22 19:30:21,877 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stopSources(458)) - Stopping metrics source RpcDetailedActivityForPort53664: class=class org.apache.hadoop.metrics2.lib.MetricsSourceBuilder$1 2015-11-22 19:30:21,877 DEBUG [main] util.MBeans (MBeans.java:unregister(103)) - Unregistering Hadoop:service=ApplicationHistoryServer,name=RpcDetailedActivityForPort53664 2015-11-22 19:30:21,877 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stopSources(458)) - Stopping metrics source RpcActivityForPort38037: class=class org.apache.hadoop.metrics2.lib.MetricsSourceBuilder$1 2015-11-22 19:30:21,877 DEBUG [main] util.MBeans (MBeans.java:unregister(103)) - Unregistering Hadoop:service=ApplicationHistoryServer,name=RpcActivityForPort38037 2015-11-22 19:30:21,878 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stopSources(458)) - Stopping metrics source RpcDetailedActivityForPort38037: class=class org.apache.hadoop.metrics2.lib.MetricsSourceBuilder$1 2015-11-22 19:30:21,878 DEBUG [main] util.MBeans (MBeans.java:unregister(103)) - Unregistering Hadoop:service=ApplicationHistoryServer,name=RpcDetailedActivityForPort38037 2015-11-22 19:30:21,878 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stopSources(458)) - Stopping metrics source RpcActivityForPort38415: class=class org.apache.hadoop.metrics2.lib.MetricsSourceBuilder$1 2015-11-22 19:30:21,878 DEBUG [main] util.MBeans (MBeans.java:unregister(103)) - Unregistering Hadoop:service=ApplicationHistoryServer,name=RpcActivityForPort38415 2015-11-22 19:30:21,878 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stopSources(458)) - Stopping metrics source RpcDetailedActivityForPort38415: class=class org.apache.hadoop.metrics2.lib.MetricsSourceBuilder$1 2015-11-22 19:30:21,878 DEBUG [main] util.MBeans (MBeans.java:unregister(103)) - Unregistering Hadoop:service=ApplicationHistoryServer,name=RpcDetailedActivityForPort38415 2015-11-22 19:30:21,878 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stopSources(458)) - Stopping metrics source RpcActivityForPort35735: class=class org.apache.hadoop.metrics2.lib.MetricsSourceBuilder$1 2015-11-22 19:30:21,879 DEBUG [main] util.MBeans (MBeans.java:unregister(103)) - Unregistering Hadoop:service=ApplicationHistoryServer,name=RpcActivityForPort35735 2015-11-22 19:30:21,879 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stopSources(458)) - Stopping metrics source RpcDetailedActivityForPort35735: class=class org.apache.hadoop.metrics2.lib.MetricsSourceBuilder$1 2015-11-22 19:30:21,879 DEBUG [main] util.MBeans (MBeans.java:unregister(103)) - Unregistering Hadoop:service=ApplicationHistoryServer,name=RpcDetailedActivityForPort35735 2015-11-22 19:30:21,879 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stopSources(458)) - Stopping metrics source RpcActivityForPort40868: class=class org.apache.hadoop.metrics2.lib.MetricsSourceBuilder$1 2015-11-22 19:30:21,879 DEBUG [main] util.MBeans (MBeans.java:unregister(103)) - Unregistering Hadoop:service=ApplicationHistoryServer,name=RpcActivityForPort40868 2015-11-22 19:30:21,879 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stopSources(458)) - Stopping metrics source RpcDetailedActivityForPort40868: class=class org.apache.hadoop.metrics2.lib.MetricsSourceBuilder$1 2015-11-22 19:30:21,879 DEBUG [main] util.MBeans (MBeans.java:unregister(103)) - Unregistering Hadoop:service=ApplicationHistoryServer,name=RpcDetailedActivityForPort40868 2015-11-22 19:30:21,879 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stopSources(458)) - Stopping metrics source RpcActivityForPort56823: class=class org.apache.hadoop.metrics2.lib.MetricsSourceBuilder$1 2015-11-22 19:30:21,879 DEBUG [main] util.MBeans (MBeans.java:unregister(103)) - Unregistering Hadoop:service=ApplicationHistoryServer,name=RpcActivityForPort56823 2015-11-22 19:30:21,879 DEBUG [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stopSources(458)) - Stopping metrics source RpcDetailedActivityForPort56823: class=class org.apache.hadoop.metrics2.lib.MetricsSourceBuilder$1 2015-11-22 19:30:21,880 DEBUG [main] util.MBeans (MBeans.java:unregister(103)) - Unregistering Hadoop:service=ApplicationHistoryServer,name=RpcDetailedActivityForPort56823 2015-11-22 19:30:21,880 DEBUG [main] util.MBeans (MBeans.java:unregister(103)) - Unregistering Hadoop:service=ResourceManager,name=MetricsSystem,sub=Stats 2015-11-22 19:30:21,880 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stop(217)) - ApplicationHistoryServer metrics system stopped. 2015-11-22 19:30:21,880 DEBUG [main] util.MBeans (MBeans.java:unregister(103)) - Unregistering Hadoop:service=ResourceManager,name=MetricsSystem,sub=Control 2015-11-22 19:30:21,880 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:shutdown(608)) - ApplicationHistoryServer metrics system shutdown complete. 2015-11-22 19:30:21,880 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore entered state STOPPED 2015-11-22 19:30:21,880 DEBUG [main] service.AbstractService (AbstractService.java:enterState(452)) - Service: Dispatcher entered state STOPPED 2015-11-22 19:30:21,880 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:serviceStop(141)) - AsyncDispatcher is draining to stop, igonring any new events. 2015-11-22 19:30:21,881 INFO [main] resourcemanager.ResourceManager (ResourceManager.java:transitionToStandby(1058)) - Transitioned to standby state 2015-11-22 19:30:21,881 DEBUG [main] security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1684)) - PrivilegedAction as:ubuntu (auth:SIMPLE) from:org.apache.hadoop.fs.FileContext.getAbstractFileSystem(FileContext.java:339)