2016-08-20 21:57:07,982 DEBUG [IPC Server handler 0 on 37347] ipc.Server: IPC Server handler 0 on 37347: org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 127.0.0.1:46672 Call#16 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2016-08-20 21:57:07,983 DEBUG [IPC Server handler 0 on 37347] security.UserGroupInformation: PrivilegedAction as:appattempt_1471710419543_0001_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2419) 2016-08-20 21:57:07,984 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType: STATUS_UPDATE 2016-08-20 21:57:07,984 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl: Processing event for appattempt_1471710419543_0001_000001 of type STATUS_UPDATE 2016-08-20 21:57:07,986 DEBUG [IPC Server handler 0 on 37347] rmcontainer.RMContainerImpl: Processing container_1471710419543_0001_01_000004 of type RELEASED 2016-08-20 21:57:07,987 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeCleanContainerEvent.EventType: CLEANUP_CONTAINER 2016-08-20 21:57:07,987 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl: Processing localhost:36489 of type CLEANUP_CONTAINER 2016-08-20 21:57:07,987 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptContainerFinishedEvent.EventType: CONTAINER_FINISHED 2016-08-20 21:57:07,987 INFO [IPC Server handler 0 on 37347] rmcontainer.RMContainerImpl: container_1471710419543_0001_01_000004 Container Transitioned from RUNNING to RELEASED 2016-08-20 21:57:07,987 DEBUG [AsyncDispatcher event handler] attempt.RMAppAttemptImpl: Processing event for appattempt_1471710419543_0001_000001 of type CONTAINER_FINISHED 2016-08-20 21:57:07,987 INFO [IPC Server handler 0 on 37347] resourcemanager.RMAuditLogger: USER=root IP=127.0.0.1 OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1471710419543_0001 CONTAINERID=container_1471710419543_0001_01_000004 RESOURCE= 2016-08-20 21:57:07,989 DEBUG [IPC Server handler 0 on 37347] scheduler.SchedulerNode: Released container container_1471710419543_0001_01_000004 of capacity on host localhost:36489, which currently has 2 containers, used and available, release resources=true 2016-08-20 21:57:07,991 DEBUG [IPC Server handler 0 on 37347] capacity.LeafQueue: User limit computation for root in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: consumedRatio: 0.0 currentCapacity: activeUsers: 0 clusterCapacity: resourceByLabel: usageratio: 0.25 Partition: 2016-08-20 21:57:07,991 DEBUG [IPC Server handler 0 on 37347] capacity.LeafQueue: default used= numContainers=3 user=root user-resources= 2016-08-20 21:57:07,991 DEBUG [IPC Server handler 0 on 37347] capacity.ParentQueue: completedContainer root: numChildQueue= 1, capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.25, numApps=1, numContainers=3, cluster= 2016-08-20 21:57:07,992 DEBUG [IPC Server handler 0 on 37347] capacity.ParentQueue: Re-sorting completed queue: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.25, absoluteUsedCapacity=0.25, numApps=1, numContainers=3 2016-08-20 21:57:07,994 WARN [IPC Server handler 0 on 37347] scheduler.AbstractYarnScheduler: Error happens when checking increase request, Ignoring.. exception= org.apache.hadoop.yarn.exceptions.InvalidResourceRequestException: Failed to get rmContainer for increase request, with container-id=container_1471710419543_0001_01_000004 at org.apache.hadoop.yarn.server.resourcemanager.scheduler.AbstractYarnScheduler.createSchedContainerChangeRequest(AbstractYarnScheduler.java:768) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.AbstractYarnScheduler.createSchedContainerChangeRequests(AbstractYarnScheduler.java:785) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.updateIncreaseRequests(CapacityScheduler.java:934) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.allocate(CapacityScheduler.java:968) at org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService.allocate(ApplicationMasterService.java:525) at org.apache.hadoop.yarn.api.impl.pb.service.ApplicationMasterProtocolPBServiceImpl.allocate(ApplicationMasterProtocolPBServiceImpl.java:60) at org.apache.hadoop.yarn.proto.ApplicationMasterProtocol$ApplicationMasterProtocolService$2.callBlockingMethod(ApplicationMasterProtocol.java:99) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:663) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:989) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2423) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2419) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1790) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2419) 2016-08-20 21:57:08,000 DEBUG [IPC Server handler 0 on 37347] scheduler.AppSchedulingInfo: Added increase request:container_1471710419543_0001_01_000003 delta= 2016-08-20 21:57:08,002 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl: Node's health-status : true, 2016-08-20 21:57:08,003 DEBUG [IPC Server handler 0 on 37347] scheduler.AbstractYarnScheduler: Processing decrease request:, node=localhost:36489> 2016-08-20 21:57:08,003 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl: Sending out 3 container statuses: [ContainerStatus: [ContainerId: container_1471710419543_0001_01_000002, ExecutionType: GUARANTEED, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ], ContainerStatus: [ContainerId: container_1471710419543_0001_01_000003, ExecutionType: GUARANTEED, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ], ContainerStatus: [ContainerId: container_1471710419543_0001_01_000004, ExecutionType: GUARANTEED, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2016-08-20 21:57:08,005 DEBUG [IPC Server handler 0 on 37347] capacity.LeafQueue: User limit computation for root in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: consumedRatio: 0.0 currentCapacity: activeUsers: 0 clusterCapacity: resourceByLabel: usageratio: 0.20833333 Partition: 2016-08-20 21:57:08,005 DEBUG [IPC Server handler 0 on 37347] capacity.LeafQueue: default used= numContainers=3 user=root user-resources= 2016-08-20 21:57:08,005 DEBUG [IPC Server handler 0 on 37347] scheduler.AppSchedulingInfo: Decrease container : applicationId=application_1471710419543_0001 container=container_1471710419543_0001_01_000002 host=localhost:36489 user=root resource= 2016-08-20 21:57:08,006 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2016-08-20 21:57:08,007 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl: Processing localhost:36489 of type STATUS_UPDATE 2016-08-20 21:57:08,006 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.CMgrCompletedContainersEvent.EventType: FINISH_CONTAINERS 2016-08-20 21:57:08,012 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl: Node's health-status : true, 2016-08-20 21:57:08,012 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2016-08-20 21:57:08,012 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl: Sending out 0 container statuses: [] 2016-08-20 21:57:08,012 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler: nodeUpdate: localhost:36489 clusterResources: 2016-08-20 21:57:08,012 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler: Node being looked for scheduling localhost:36489 availableResource: 2016-08-20 21:57:08,012 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler: Trying to schedule on node: localhost, available: 2016-08-20 21:57:08,012 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2016-08-20 21:57:08,012 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue: Trying to assign containers to child-queue of root 2016-08-20 21:57:08,012 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl: Processing localhost:46239 of type STATUS_UPDATE 2016-08-20 21:57:08,013 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2016-08-20 21:57:08,014 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerKillEvent.EventType: KILL_CONTAINER 2016-08-20 21:57:08,014 DEBUG [IPC Server handler 0 on 37347] rmcontainer.RMContainerImpl: Processing container_1471710419543_0001_01_000002 of type CHANGE_RESOURCE 2016-08-20 21:57:08,014 DEBUG [AsyncDispatcher event handler] container.ContainerImpl: Processing container_1471710419543_0001_01_000004 of type KILL_CONTAINER 2016-08-20 21:57:08,014 INFO [AsyncDispatcher event handler] container.ContainerImpl: Container container_1471710419543_0001_01_000004 transitioned from RUNNING to KILLING 2016-08-20 21:57:08,014 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncherEvent.EventType: CLEANUP_CONTAINER 2016-08-20 21:57:08,015 INFO [AsyncDispatcher event handler] launcher.ContainerLaunch: Cleaning up container container_1471710419543_0001_01_000004 2016-08-20 21:57:08,015 DEBUG [AsyncDispatcher event handler] launcher.ContainerLaunch: Marking container container_1471710419543_0001_01_000004 as inactive 2016-08-20 21:57:08,015 DEBUG [AsyncDispatcher event handler] launcher.ContainerLaunch: Getting pid for container container_1471710419543_0001_01_000004 to kill from pid file /opt/hadooptrunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/target/org.apache.hadoop.yarn.client.api.impl.TestAMRMClient/org.apache.hadoop.yarn.client.api.impl.TestAMRMClient-localDir-nm-0_0/nmPrivate/application_1471710419543_0001/container_1471710419543_0001_01_000004/container_1471710419543_0001_01_000004.pid 2016-08-20 21:57:08,015 DEBUG [AsyncDispatcher event handler] launcher.ContainerLaunch: Accessing pid for container container_1471710419543_0001_01_000004 from pid file /opt/hadooptrunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/target/org.apache.hadoop.yarn.client.api.impl.TestAMRMClient/org.apache.hadoop.yarn.client.api.impl.TestAMRMClient-localDir-nm-0_0/nmPrivate/application_1471710419543_0001/container_1471710419543_0001_01_000004/container_1471710419543_0001_01_000004.pid 2016-08-20 21:57:08,016 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl: Node's health-status : true, 2016-08-20 21:57:08,017 DEBUG [AsyncDispatcher event handler] util.ProcessIdFileReader: Accessing pid from pid file /opt/hadooptrunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/target/org.apache.hadoop.yarn.client.api.impl.TestAMRMClient/org.apache.hadoop.yarn.client.api.impl.TestAMRMClient-localDir-nm-0_0/nmPrivate/application_1471710419543_0001/container_1471710419543_0001_01_000004/container_1471710419543_0001_01_000004.pid 2016-08-20 21:57:08,017 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl: Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1471710419543_0001_01_000001, ExecutionType: GUARANTEED, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2016-08-20 21:57:08,017 DEBUG [AsyncDispatcher event handler] util.ProcessIdFileReader: Got pid 3843 from path /opt/hadooptrunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/target/org.apache.hadoop.yarn.client.api.impl.TestAMRMClient/org.apache.hadoop.yarn.client.api.impl.TestAMRMClient-localDir-nm-0_0/nmPrivate/application_1471710419543_0001/container_1471710419543_0001_01_000004/container_1471710419543_0001_01_000004.pid 2016-08-20 21:57:08,017 DEBUG [AsyncDispatcher event handler] launcher.ContainerLaunch: Got pid 3843 for container container_1471710419543_0001_01_000004 2016-08-20 21:57:08,017 DEBUG [AsyncDispatcher event handler] launcher.ContainerLaunch: Sending signal to pid 3843 as user root for container container_1471710419543_0001_01_000004 2016-08-20 21:57:08,017 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2016-08-20 21:57:08,017 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl: Processing localhost:43931 of type STATUS_UPDATE 2016-08-20 21:57:08,019 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2016-08-20 21:57:08,025 DEBUG [AsyncDispatcher event handler] nodemanager.DefaultContainerExecutor: Sending signal 15 to pid 3843 as user root 2016-08-20 21:57:08,026 DEBUG [IPC Server handler 0 on 37347] scheduler.SchedulerNode: Decreased container container_1471710419543_0001_01_000002 of capacity on host localhost:36489, which has 2 containers, used and available after allocation 2016-08-20 21:57:08,026 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue: printChildQueues - queue: root child-queues: root.defaultusedCapacity=(0.20833333), label=(*) 2016-08-20 21:57:08,026 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue: Trying to assign to queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.20833333, absoluteUsedCapacity=0.20833333, numApps=1, numContainers=3 2016-08-20 21:57:08,026 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue: assignContainers: node=localhost #applications=1 2016-08-20 21:57:08,027 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue: User limit computation for root in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: consumedRatio: 0.0 currentCapacity: activeUsers: 0 clusterCapacity: resourceByLabel: usageratio: 0.20833333 Partition: 2016-08-20 21:57:08,027 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue: Headroom calculation for user root: userLimit= queueMaxAvailRes= consumed= headroom= 2016-08-20 21:57:08,027 DEBUG [SchedulerEventDispatcher:Event Processor] fica.FiCaSchedulerApp: pre-assignContainers for application application_1471710419543_0001 2016-08-20 21:57:08,027 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue: User limit computation for root in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: consumedRatio: 0.0 currentCapacity: activeUsers: 0 clusterCapacity: resourceByLabel: usageratio: 0.20833333 Partition: 2016-08-20 21:57:08,027 DEBUG [SchedulerEventDispatcher:Event Processor] scheduler.SchedulerApplicationAttempt: showRequests: application=application_1471710419543_0001 headRoom= currentConsumption=2560 2016-08-20 21:57:08,027 DEBUG [SchedulerEventDispatcher:Event Processor] scheduler.SchedulerApplicationAttempt: showRequests: application=application_1471710419543_0001 request={AllocationRequestId: 0, Priority: 0, Capability: , # Containers: 0, Location: *, Relax Locality: true, Execution Type Request: {Execution Type: GUARANTEED, Enforce Execution Type: false}, Node Label Expression: } 2016-08-20 21:57:08,027 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue: User limit computation for root in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: consumedRatio: 0.0 currentCapacity: activeUsers: 0 clusterCapacity: resourceByLabel: usageratio: 0.20833333 Partition: 2016-08-20 21:57:08,027 DEBUG [SchedulerEventDispatcher:Event Processor] scheduler.SchedulerApplicationAttempt: showRequests: application=application_1471710419543_0001 headRoom= currentConsumption=2560 2016-08-20 21:57:08,027 DEBUG [SchedulerEventDispatcher:Event Processor] scheduler.SchedulerApplicationAttempt: showRequests: application=application_1471710419543_0001 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 0, Location: *, Relax Locality: true, Execution Type Request: {Execution Type: GUARANTEED, Enforce Execution Type: false}, Node Label Expression: } 2016-08-20 21:57:08,028 DEBUG [SchedulerEventDispatcher:Event Processor] allocator.IncreaseContainerAllocator: Looking at increase request for application=appattempt_1471710419543_0001_000001 priority=0 2016-08-20 21:57:08,028 DEBUG [SchedulerEventDispatcher:Event Processor] allocator.IncreaseContainerAllocator: There's no increase request for appattempt_1471710419543_0001_000001 priority=0 2016-08-20 21:57:08,028 DEBUG [SchedulerEventDispatcher:Event Processor] allocator.IncreaseContainerAllocator: Looking at increase request for application=appattempt_1471710419543_0001_000001 priority=1 2016-08-20 21:57:08,028 DEBUG [SchedulerEventDispatcher:Event Processor] allocator.IncreaseContainerAllocator: Looking at increase request=, node=localhost:36489> 2016-08-20 21:57:08,033 DEBUG [SchedulerEventDispatcher:Event Processor] scheduler.SchedulerNode: Increased container container_1471710419543_0001_01_000003 of capacity on host localhost:36489, which has 2 containers, used and available after allocation 2016-08-20 21:57:08,033 DEBUG [SchedulerEventDispatcher:Event Processor] scheduler.AppSchedulingInfo: allocated increase request : applicationId=application_1471710419543_0001 container=container_1471710419543_0001_01_000003 host=localhost:36489 user=root resource= 2016-08-20 21:57:08,033 DEBUG [SchedulerEventDispatcher:Event Processor] scheduler.AppSchedulingInfo: remove increase request:, node=localhost:36489> 2016-08-20 21:57:08,033 DEBUG [SchedulerEventDispatcher:Event Processor] rmcontainer.RMContainerImpl: Processing container_1471710419543_0001_01_000003 of type CHANGE_RESOURCE 2016-08-20 21:57:08,034 INFO [SchedulerEventDispatcher:Event Processor] allocator.IncreaseContainerAllocator: Approved increase container request:, node=localhost:36489> fromReservation=false 2016-08-20 21:57:08,034 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue: post-assignContainers for application application_1471710419543_0001 2016-08-20 21:57:08,034 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue: User limit computation for root in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: consumedRatio: 0.0 currentCapacity: activeUsers: 0 clusterCapacity: resourceByLabel: usageratio: 0.20833333 Partition: 2016-08-20 21:57:08,034 DEBUG [SchedulerEventDispatcher:Event Processor] scheduler.SchedulerApplicationAttempt: showRequests: application=application_1471710419543_0001 headRoom= currentConsumption=3584 2016-08-20 21:57:08,034 DEBUG [SchedulerEventDispatcher:Event Processor] scheduler.SchedulerApplicationAttempt: showRequests: application=application_1471710419543_0001 request={AllocationRequestId: 0, Priority: 0, Capability: , # Containers: 0, Location: *, Relax Locality: true, Execution Type Request: {Execution Type: GUARANTEED, Enforce Execution Type: false}, Node Label Expression: } 2016-08-20 21:57:08,037 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue: User limit computation for root in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: consumedRatio: 0.0 currentCapacity: activeUsers: 0 clusterCapacity: resourceByLabel: usageratio: 0.20833333 Partition: 2016-08-20 21:57:08,037 DEBUG [SchedulerEventDispatcher:Event Processor] scheduler.SchedulerApplicationAttempt: showRequests: application=application_1471710419543_0001 headRoom= currentConsumption=3584 2016-08-20 21:57:08,037 DEBUG [SchedulerEventDispatcher:Event Processor] scheduler.SchedulerApplicationAttempt: showRequests: application=application_1471710419543_0001 request={AllocationRequestId: 0, Priority: 1, Capability: , # Containers: 0, Location: *, Relax Locality: true, Execution Type Request: {Execution Type: GUARANTEED, Enforce Execution Type: false}, Node Label Expression: } 2016-08-20 21:57:08,038 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue: User limit computation for root in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: consumedRatio: 0.0 currentCapacity: activeUsers: 0 clusterCapacity: resourceByLabel: usageratio: 0.29166666 Partition: 2016-08-20 21:57:08,038 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue: User limit computation for root in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: consumedRatio: 0.0 currentCapacity: activeUsers: 0 clusterCapacity: resourceByLabel: usageratio: 0.29166666 Partition: 2016-08-20 21:57:08,038 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue: User limit computation for root in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: consumedRatio: 0.0 currentCapacity: activeUsers: 0 clusterCapacity: resourceByLabel: usageratio: 0.29166666 Partition: 2016-08-20 21:57:08,038 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue: default user=root used= numContainers=3 headroom = user-resources= 2016-08-20 21:57:08,038 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue: Assigned to queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.29166666, absoluteUsedCapacity=0.29166666, numApps=1, numContainers=3 --> , NODE_LOCAL 2016-08-20 21:57:08,038 INFO [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue: Re-sorting assigned queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.29166666, absoluteUsedCapacity=0.29166666, numApps=1, numContainers=3 2016-08-20 21:57:08,039 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue: printChildQueues - queue: root child-queues: root.defaultusedCapacity=(0.29166666), label=(*) 2016-08-20 21:57:08,039 INFO [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue: assignedContainer queue=root usedCapacity=0.33333334 absoluteUsedCapacity=0.33333334 used= cluster= 2016-08-20 21:57:08,039 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue: ParentQ=root assignedSoFarInThisIteration= usedCapacity=0.33333334 absoluteUsedCapacity=0.33333334 2016-08-20 21:57:08,039 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue: Trying to assign containers to child-queue of root 2016-08-20 21:57:08,040 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue: printChildQueues - queue: root child-queues: root.defaultusedCapacity=(0.29166666), label=(*) 2016-08-20 21:57:08,040 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue: Trying to assign to queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.29166666, absoluteUsedCapacity=0.29166666, numApps=1, numContainers=3 2016-08-20 21:57:08,040 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue: assignContainers: node=localhost #applications=1 2016-08-20 21:57:08,040 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.LeafQueue: Skip this queue=root.default, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-20 21:57:08,041 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue: Assigned to queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.29166666, absoluteUsedCapacity=0.29166666, numApps=1, numContainers=3 --> , NODE_LOCAL 2016-08-20 21:57:08,041 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler: nodeUpdate: localhost:46239 clusterResources: 2016-08-20 21:57:08,041 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler: Node being looked for scheduling localhost:46239 availableResource: 2016-08-20 21:57:08,041 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler: Trying to schedule on node: localhost, available: 2016-08-20 21:57:08,041 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue: Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-20 21:57:08,041 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler: nodeUpdate: localhost:43931 clusterResources: 2016-08-20 21:57:08,041 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler: Node being looked for scheduling localhost:43931 availableResource: 2016-08-20 21:57:08,041 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler: Trying to schedule on node: localhost, available: 2016-08-20 21:57:08,041 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue: Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-20 21:57:08,041 DEBUG [IPC Server handler 0 on 37347] capacity.ParentQueue: completedContainer root: numChildQueue= 1, capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.29166666, numApps=1, numContainers=2, cluster= 2016-08-20 21:57:08,050 INFO [IPC Server handler 0 on 37347] capacity.LeafQueue: Application attempt appattempt_1471710419543_0001_000001 decreased container:container_1471710419543_0001_01_000002 from to 2016-08-20 21:57:08,052 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeDecreaseContainerEvent.EventType: DECREASE_CONTAINER 2016-08-20 21:57:08,052 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl: Processing localhost:36489 of type DECREASE_CONTAINER 2016-08-20 21:57:08,053 DEBUG [IPC Server handler 0 on 37347] security.BaseContainerTokenSecretManager: Creating password for container_1471710419543_0001_01_000003 for user container_1471710419543_0001_01_000003 (auth:SIMPLE) to be run on NM localhost:36489 2016-08-20 21:57:08,055 DEBUG [IPC Server handler 0 on 37347] security.ContainerTokenIdentifier: Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 1 cluster_timestamp: 1471710419543 } attemptId: 1 } id: 3 } nmHostAddr: "localhost:36489" appSubmitter: "root" resource { memory: 2048 virtual_cores: 1 } expiryTimeStamp: 1471711028053 masterKeyId: -991580041 rmIdentifier: 1471710419543 priority { priority: 1 } creationTime: 1471710427234 nodeLabelExpression: "" containerType: TASK executionType: GUARANTEED 2016-08-20 21:57:08,060 DEBUG [AsyncDispatcher event handler] launcher.ContainerLaunch: Sent signal SIGTERM to pid 3843 as user root for container container_1471710419543_0001_01_000004, result=success 2016-08-20 21:57:08,061 DEBUG [IPC Server handler 0 on 37347] security.ContainerTokenIdentifier: Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 1 cluster_timestamp: 1471710419543 } attemptId: 1 } id: 3 } nmHostAddr: "localhost:36489" appSubmitter: "root" resource { memory: 2048 virtual_cores: 1 } expiryTimeStamp: 1471711028053 masterKeyId: -991580041 rmIdentifier: 1471710419543 priority { priority: 1 } creationTime: 1471710427234 nodeLabelExpression: "" containerType: TASK executionType: GUARANTEED 2016-08-20 21:57:08,068 DEBUG [IPC Server handler 0 on 37347] rmcontainer.RMContainerImpl: Processing container_1471710419543_0001_01_000003 of type ACQUIRE_UPDATED_CONTAINER 2016-08-20 21:57:08,068 DEBUG [IPC Server handler 0 on 37347] security.BaseContainerTokenSecretManager: Creating password for container_1471710419543_0001_01_000002 for user container_1471710419543_0001_01_000002 (auth:SIMPLE) to be run on NM localhost:36489 2016-08-20 21:57:08,069 DEBUG [IPC Server handler 0 on 37347] security.ContainerTokenIdentifier: Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 1 cluster_timestamp: 1471710419543 } attemptId: 1 } id: 2 } nmHostAddr: "localhost:36489" appSubmitter: "root" resource { memory: 512 virtual_cores: 1 } expiryTimeStamp: 1471711028068 masterKeyId: -991580041 rmIdentifier: 1471710419543 priority { priority: 1 } creationTime: 1471710427212 nodeLabelExpression: "" containerType: TASK executionType: GUARANTEED 2016-08-20 21:57:08,062 DEBUG [AsyncDispatcher event handler] security.UserGroupInformation: PrivilegedAction as:root (auth:SIMPLE) from:org.apache.hadoop.fs.FileContext.getAbstractFileSystem(FileContext.java:327) 2016-08-20 21:57:08,101 WARN [ContainersLauncher #2] nodemanager.DefaultContainerExecutor: Exit code from container container_1471710419543_0001_01_000004 is : 143 2016-08-20 21:57:08,106 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl: Node's health-status : true, 2016-08-20 21:57:08,106 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl: Sending out 3 container statuses: [ContainerStatus: [ContainerId: container_1471710419543_0001_01_000002, ExecutionType: GUARANTEED, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ], ContainerStatus: [ContainerId: container_1471710419543_0001_01_000003, ExecutionType: GUARANTEED, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ], ContainerStatus: [ContainerId: container_1471710419543_0001_01_000004, ExecutionType: GUARANTEED, State: RUNNING, Capability: , Diagnostics: Container Killed by ResourceManager , ExitStatus: -106, ]] 2016-08-20 21:57:08,106 DEBUG [ContainersLauncher #2] container.ContainerImpl: Processing container_1471710419543_0001_01_000004 of type UPDATE_DIAGNOSTICS_MSG 2016-08-20 21:57:08,108 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2016-08-20 21:57:08,110 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl: Processing localhost:36489 of type STATUS_UPDATE 2016-08-20 21:57:08,107 DEBUG [ContainersLauncher #2] launcher.ContainerLaunch: Container container_1471710419543_0001_01_000004 completed with exit code 143 2016-08-20 21:57:08,108 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.CMgrDecreaseContainersResourceEvent.EventType: DECREASE_CONTAINERS_RESOURCE 2016-08-20 21:57:08,112 DEBUG [ContainersLauncher #2] concurrent.ExecutorHelper: afterExecute in thread: ContainersLauncher #2, runnable type: java.util.concurrent.FutureTask 2016-08-20 21:57:08,113 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl: Node's health-status : true, 2016-08-20 21:57:08,113 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl: Sending out 0 container statuses: [] 2016-08-20 21:57:08,114 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerExitEvent.EventType: CONTAINER_KILLED_ON_REQUEST 2016-08-20 21:57:08,119 DEBUG [IPC Server handler 0 on 37347] security.ContainerTokenIdentifier: Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 1 cluster_timestamp: 1471710419543 } attemptId: 1 } id: 2 } nmHostAddr: "localhost:36489" appSubmitter: "root" resource { memory: 512 virtual_cores: 1 } expiryTimeStamp: 1471711028068 masterKeyId: -991580041 rmIdentifier: 1471710419543 priority { priority: 1 } creationTime: 1471710427212 nodeLabelExpression: "" containerType: TASK executionType: GUARANTEED 2016-08-20 21:57:08,117 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl: Node's health-status : true, 2016-08-20 21:57:08,116 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2016-08-20 21:57:08,120 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl: Processing localhost:46239 of type STATUS_UPDATE 2016-08-20 21:57:08,121 DEBUG [Node Status Updater] nodemanager.NodeStatusUpdaterImpl: Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1471710419543_0001_01_000001, ExecutionType: GUARANTEED, State: RUNNING, Capability: , Diagnostics: , ExitStatus: -1000, ]] 2016-08-20 21:57:08,121 DEBUG [AsyncDispatcher event handler] container.ContainerImpl: Processing container_1471710419543_0001_01_000004 of type CONTAINER_KILLED_ON_REQUEST 2016-08-20 21:57:08,121 DEBUG [IPC Server handler 0 on 37347] rmcontainer.RMContainerImpl: Processing container_1471710419543_0001_01_000002 of type ACQUIRE_UPDATED_CONTAINER 2016-08-20 21:57:08,121 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2016-08-20 21:57:08,121 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2016-08-20 21:57:08,121 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler: nodeUpdate: localhost:36489 clusterResources: 2016-08-20 21:57:08,121 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler: Node being looked for scheduling localhost:36489 availableResource: 2016-08-20 21:57:08,121 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler: Trying to schedule on node: localhost, available: 2016-08-20 21:57:08,121 DEBUG [IPC Server handler 0 on 37347] capacity.LeafQueue: User limit computation for root in queue default userLimitPercent=100 userLimitFactor=1.0 required: consumed: user-limit-resource: queueCapacity: qconsumed: consumedRatio: 0.0 currentCapacity: activeUsers: 0 clusterCapacity: resourceByLabel: usageratio: 0.29166666 Partition: 2016-08-20 21:57:08,121 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType: STATUS_UPDATE 2016-08-20 21:57:08,122 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl: Processing localhost:43931 of type STATUS_UPDATE 2016-08-20 21:57:08,121 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue: Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-20 21:57:08,130 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler: nodeUpdate: localhost:46239 clusterResources: 2016-08-20 21:57:08,130 INFO [AsyncDispatcher event handler] container.ContainerImpl: Container container_1471710419543_0001_01_000004 transitioned from KILLING to CONTAINER_CLEANEDUP_AFTER_KILL 2016-08-20 21:57:08,131 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.ContainerLocalizationCleanupEvent.EventType: CLEANUP_CONTAINER_RESOURCES 2016-08-20 21:57:08,125 DEBUG [IPC Server handler 0 on 37347] ipc.Server: Served: allocate queueTime= 1 procesingTime= 142 2016-08-20 21:57:08,123 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType: NODE_UPDATE 2016-08-20 21:57:08,130 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler: Node being looked for scheduling localhost:46239 availableResource: 2016-08-20 21:57:08,131 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler: Trying to schedule on node: localhost, available: 2016-08-20 21:57:08,131 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue: Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-20 21:57:08,131 DEBUG [IPC Server handler 0 on 37347] ipc.Server: IPC Server handler 0 on 37347: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 127.0.0.1:46672 Call#16 Retry#0 2016-08-20 21:57:08,131 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler: nodeUpdate: localhost:43931 clusterResources: 2016-08-20 21:57:08,131 DEBUG [IPC Server handler 0 on 37347] ipc.Server: IPC Server handler 0 on 37347: responding to org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from 127.0.0.1:46672 Call#16 Retry#0 Wrote 561 bytes. 2016-08-20 21:57:08,131 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler: Node being looked for scheduling localhost:43931 availableResource: 2016-08-20 21:57:08,132 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.CapacityScheduler: Trying to schedule on node: localhost, available: 2016-08-20 21:57:08,132 DEBUG [SchedulerEventDispatcher:Event Processor] capacity.ParentQueue: Skip this queue=root, because it doesn't need more resource, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY node-partition= 2016-08-20 21:57:08,132 DEBUG [IPC Client (1470959992) connection to localhost/127.0.0.1:37347 from root] ipc.Client: IPC Client (1470959992) connection to localhost/127.0.0.1:37347 from root got value #16 2016-08-20 21:57:08,132 DEBUG [Thread-346] ipc.ProtobufRpcEngine: Call: allocate took 151ms 2016-08-20 21:57:08,133 DEBUG [Thread-346] impl.AMRMClientImpl: RM has confirmed changed resource allocation for container container_1471710419543_0001_01_000003. Current resource allocation:. Remove pending change request: 2016-08-20 21:57:08,133 DEBUG [Thread-346] impl.AMRMClientImpl: RM has confirmed changed resource allocation for container container_1471710419543_0001_01_000002. Current resource allocation:. Remove pending change request: