2018-08-30 09:30:22,683 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Created MRAppMaster for application appattempt_1535621162012_0001_000001 2018-08-30 09:30:22,879 WARN [main] org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2018-08-30 09:30:22,953 INFO [main] org.apache.hadoop.security.SecurityUtil: Updating Configuration 2018-08-30 09:30:22,954 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Executing with tokens: 2018-08-30 09:30:23,200 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Kind: YARN_AM_RM_TOKEN, Service: , Ident: (appAttemptId { application_id { id: 1 cluster_timestamp: 1535621162012 } attemptId: 1 } keyId: -2120451857) 2018-08-30 09:30:23,214 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Using mapred newApiCommitter. 2018-08-30 09:30:23,215 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: OutputCommitter set in config null 2018-08-30 09:30:23,261 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: OutputCommitter is gobblin.runtime.mapreduce.GobblinOutputCommitter 2018-08-30 09:30:23,952 WARN [main] org.apache.hadoop.hdfs.shortcircuit.DomainSocketFactory: The short-circuit local reads feature cannot be used because libhadoop cannot be loaded. 2018-08-30 09:30:24,122 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.jobhistory.EventType for class org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler 2018-08-30 09:30:24,123 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.JobEventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobEventDispatcher 2018-08-30 09:30:24,123 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.TaskEventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskEventDispatcher 2018-08-30 09:30:24,124 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.TaskAttemptEventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskAttemptEventDispatcher 2018-08-30 09:30:24,124 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventType for class org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler 2018-08-30 09:30:24,125 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.speculate.Speculator$EventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$SpeculatorEventDispatcher 2018-08-30 09:30:24,125 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.rm.ContainerAllocator$EventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter 2018-08-30 09:30:24,126 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncher$EventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerLauncherRouter 2018-08-30 09:30:24,157 INFO [main] org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils: Default file system is set solely by core-default.xml therefore - ignoring 2018-08-30 09:30:24,182 INFO [main] org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils: Default file system is set solely by core-default.xml therefore - ignoring 2018-08-30 09:30:24,203 INFO [main] org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils: Default file system is set solely by core-default.xml therefore - ignoring 2018-08-30 09:30:24,219 INFO [main] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Emitting job history data to the timeline server is not enabled 2018-08-30 09:30:24,250 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.JobFinishEvent$Type for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobFinishEventHandler 2018-08-30 09:30:24,336 WARN [main] org.apache.hadoop.metrics2.impl.MetricsConfig: Cannot locate configuration: tried hadoop-metrics2-mrappmaster.properties,hadoop-metrics2.properties 2018-08-30 09:30:24,404 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s). 2018-08-30 09:30:24,404 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MRAppMaster metrics system started 2018-08-30 09:30:24,413 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Adding job token for job_1535621162012_0001 to jobTokenSecretManager 2018-08-30 09:30:24,559 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Not uberizing job_1535621162012_0001 because: not enabled; 2018-08-30 09:30:24,571 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Input size for job job_1535621162012_0001 = 0. Number of splits = 5 2018-08-30 09:30:24,571 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Number of reduces for job job_1535621162012_0001 = 0 2018-08-30 09:30:24,571 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1535621162012_0001Job Transitioned from NEW to INITED 2018-08-30 09:30:24,572 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: MRAppMaster launching normal, non-uberized, multi-container job job_1535621162012_0001. 2018-08-30 09:30:24,595 INFO [main] org.apache.hadoop.ipc.CallQueueManager: Using callQueue: class java.util.concurrent.LinkedBlockingQueue scheduler: class org.apache.hadoop.ipc.DefaultRpcScheduler 2018-08-30 09:30:24,603 INFO [Socket Reader #1 for port 33483] org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 33483 2018-08-30 09:30:24,645 INFO [main] org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl: Adding protocol org.apache.hadoop.mapreduce.v2.api.MRClientProtocolPB to the server 2018-08-30 09:30:24,645 INFO [IPC Server Responder] org.apache.hadoop.ipc.Server: IPC Server Responder: starting 2018-08-30 09:30:24,645 INFO [IPC Server listener on 33483] org.apache.hadoop.ipc.Server: IPC Server listener on 33483: starting 2018-08-30 09:30:24,647 INFO [main] org.apache.hadoop.mapreduce.v2.app.client.MRClientService: Instantiated MRClientService at mgmt-hadoop-dn-0.node.dc1.pnda.local/10.0.1.158:33483 2018-08-30 09:30:24,704 INFO [main] org.mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2018-08-30 09:30:24,712 INFO [main] org.apache.hadoop.security.authentication.server.AuthenticationFilter: Unable to initialize FileSignerSecretProvider, falling back to use random secrets. 2018-08-30 09:30:24,716 WARN [main] org.apache.hadoop.http.HttpRequestLog: Jetty request log can only be enabled using Log4j 2018-08-30 09:30:24,721 INFO [main] org.apache.hadoop.http.HttpServer2: Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter) 2018-08-30 09:30:24,726 INFO [main] org.apache.hadoop.http.HttpServer2: Added filter AM_PROXY_FILTER (class=org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter) to context mapreduce 2018-08-30 09:30:24,726 INFO [main] org.apache.hadoop.http.HttpServer2: Added filter AM_PROXY_FILTER (class=org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter) to context static 2018-08-30 09:30:24,761 INFO [main] org.apache.hadoop.http.HttpServer2: adding path spec: /mapreduce/* 2018-08-30 09:30:24,762 INFO [main] org.apache.hadoop.http.HttpServer2: adding path spec: /ws/* 2018-08-30 09:30:25,064 INFO [main] org.apache.hadoop.yarn.webapp.WebApps: Registered webapp guice modules 2018-08-30 09:30:25,066 INFO [main] org.apache.hadoop.http.HttpServer2: Jetty bound to port 42213 2018-08-30 09:30:25,066 INFO [main] org.mortbay.log: jetty-6.1.26 2018-08-30 09:30:25,098 INFO [main] org.mortbay.log: Extract jar:file:/mnt/hadoop/yarn/nm/filecache/10/mapreduce.tar.gz/hadoop/share/hadoop/yarn/hadoop-yarn-common-2.7.3.2.6.5.0-292.jar!/webapps/mapreduce to /mnt/hadoop/yarn/nm/usercache/pnda/appcache/application_1535621162012_0001/container_e02_1535621162012_0001_01_000001/tmp/Jetty_0_0_0_0_42213_mapreduce____45cq1v/webapp 2018-08-30 09:30:26,264 INFO [main] org.mortbay.log: Started HttpServer2$SelectChannelConnectorWithSafeStartup@0.0.0.0:42213 2018-08-30 09:30:26,264 INFO [main] org.apache.hadoop.yarn.webapp.WebApps: Web app mapreduce started at 42213 2018-08-30 09:30:26,269 INFO [main] org.apache.hadoop.ipc.CallQueueManager: Using callQueue: class java.util.concurrent.LinkedBlockingQueue scheduler: class org.apache.hadoop.ipc.DefaultRpcScheduler 2018-08-30 09:30:26,270 INFO [Socket Reader #1 for port 33433] org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 33433 2018-08-30 09:30:26,276 INFO [IPC Server Responder] org.apache.hadoop.ipc.Server: IPC Server Responder: starting 2018-08-30 09:30:26,276 INFO [IPC Server listener on 33433] org.apache.hadoop.ipc.Server: IPC Server listener on 33433: starting 2018-08-30 09:30:26,308 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: nodeBlacklistingEnabled:true 2018-08-30 09:30:26,308 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: maxTaskFailuresPerNode is 3 2018-08-30 09:30:26,308 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: blacklistDisablePercent is 33 2018-08-30 09:30:26,358 INFO [main] org.apache.hadoop.yarn.client.RMProxy: Connecting to ResourceManager at mgmt-hadoop-mgr-1.node.dc1.pnda.local/10.0.0.17:8030 2018-08-30 09:30:26,462 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: maxContainerCapability: 2018-08-30 09:30:26,462 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: queue: default 2018-08-30 09:30:26,466 INFO [main] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Upper limit on the thread pool size is 500 2018-08-30 09:30:26,466 INFO [main] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: The thread pool initial size is 10 2018-08-30 09:30:26,469 INFO [main] org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0 2018-08-30 09:30:26,474 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1535621162012_0001Job Transitioned from INITED to SETUP 2018-08-30 09:30:26,475 INFO [CommitterEvent Processor #0] org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler: Processing the event EventType: JOB_SETUP 2018-08-30 09:30:26,476 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1535621162012_0001Job Transitioned from SETUP to RUNNING 2018-08-30 09:30:26,490 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1535621162012_0001_m_000000 Task Transitioned from NEW to SCHEDULED 2018-08-30 09:30:26,490 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1535621162012_0001_m_000001 Task Transitioned from NEW to SCHEDULED 2018-08-30 09:30:26,490 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1535621162012_0001_m_000002 Task Transitioned from NEW to SCHEDULED 2018-08-30 09:30:26,490 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1535621162012_0001_m_000003 Task Transitioned from NEW to SCHEDULED 2018-08-30 09:30:26,491 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1535621162012_0001_m_000004 Task Transitioned from NEW to SCHEDULED 2018-08-30 09:30:26,491 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1535621162012_0001_m_000000_0 TaskAttempt Transitioned from NEW to UNASSIGNED 2018-08-30 09:30:26,492 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1535621162012_0001_m_000001_0 TaskAttempt Transitioned from NEW to UNASSIGNED 2018-08-30 09:30:26,492 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1535621162012_0001_m_000002_0 TaskAttempt Transitioned from NEW to UNASSIGNED 2018-08-30 09:30:26,492 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1535621162012_0001_m_000003_0 TaskAttempt Transitioned from NEW to UNASSIGNED 2018-08-30 09:30:26,492 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1535621162012_0001_m_000004_0 TaskAttempt Transitioned from NEW to UNASSIGNED 2018-08-30 09:30:26,493 INFO [Thread-53] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: mapResourceRequest: 2018-08-30 09:30:26,580 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Event Writer setup for JobId: job_1535621162012_0001, File: hdfs://mgmt-hadoop-mgr-1.node.dc1.pnda.local:8020/user/pnda/.staging/job_1535621162012_0001/job_1535621162012_0001_1.jhist 2018-08-30 09:30:27,465 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Before Scheduling: PendingReds:0 ScheduledMaps:5 ScheduledReds:0 AssignedMaps:0 AssignedReds:0 CompletedMaps:0 CompletedReds:0 ContAlloc:0 ContRel:0 HostLocal:0 RackLocal:0 2018-08-30 09:30:27,508 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: getResources() for application_1535621162012_0001: ask=1 release= 0 newContainers=0 finishedContainers=0 resourcelimit= knownNMs=1 2018-08-30 09:30:28,521 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Got allocated containers 1 2018-08-30 09:30:28,538 INFO [RMCommunicator Allocator] org.apache.hadoop.yarn.util.RackResolver: Resolved mgmt-hadoop-dn-0.node.dc1.pnda.local to /default-rack 2018-08-30 09:30:28,540 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned container container_e02_1535621162012_0001_01_000002 to attempt_1535621162012_0001_m_000000_0 2018-08-30 09:30:28,541 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: After Scheduling: PendingReds:0 ScheduledMaps:4 ScheduledReds:0 AssignedMaps:1 AssignedReds:0 CompletedMaps:0 CompletedReds:0 ContAlloc:1 ContRel:0 HostLocal:0 RackLocal:0 2018-08-30 09:30:28,589 INFO [AsyncDispatcher event handler] org.apache.hadoop.yarn.util.RackResolver: Resolved mgmt-hadoop-dn-0.node.dc1.pnda.local to /default-rack 2018-08-30 09:30:28,604 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: The job-jar file on the remote FS is hdfs://mgmt-hadoop-mgr-1.node.dc1.pnda.local:8020/user/pnda/.staging/job_1535621162012_0001/job.jar 2018-08-30 09:30:28,607 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: The job-conf file on the remote FS is /user/pnda/.staging/job_1535621162012_0001/job.xml 2018-08-30 09:30:28,842 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Adding #0 tokens and #1 secret keys for NM use for launching container 2018-08-30 09:30:28,842 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Size of containertokens_dob is 1 2018-08-30 09:30:28,842 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Putting shuffle token in serviceData 2018-08-30 09:30:29,388 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1535621162012_0001_m_000000_0 TaskAttempt Transitioned from UNASSIGNED to ASSIGNED 2018-08-30 09:30:29,393 INFO [ContainerLauncher #0] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_LAUNCH for container container_e02_1535621162012_0001_01_000002 taskAttempt attempt_1535621162012_0001_m_000000_0 2018-08-30 09:30:29,394 INFO [ContainerLauncher #0] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Launching attempt_1535621162012_0001_m_000000_0 2018-08-30 09:30:29,395 INFO [ContainerLauncher #0] org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy: Opening proxy : mgmt-hadoop-dn-0.node.dc1.pnda.local:45454 2018-08-30 09:30:29,479 INFO [ContainerLauncher #0] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Shuffle port returned by ContainerManager for attempt_1535621162012_0001_m_000000_0 : 13562 2018-08-30 09:30:29,481 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: TaskAttempt: [attempt_1535621162012_0001_m_000000_0] using containerId: [container_e02_1535621162012_0001_01_000002 on NM: [mgmt-hadoop-dn-0.node.dc1.pnda.local:45454] 2018-08-30 09:30:29,483 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1535621162012_0001_m_000000_0 TaskAttempt Transitioned from ASSIGNED to RUNNING 2018-08-30 09:30:29,483 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1535621162012_0001_m_000000 Task Transitioned from SCHEDULED to RUNNING 2018-08-30 09:30:29,546 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: getResources() for application_1535621162012_0001: ask=1 release= 0 newContainers=1 finishedContainers=0 resourcelimit= knownNMs=1 2018-08-30 09:30:29,546 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Got allocated containers 1 2018-08-30 09:30:29,546 INFO [RMCommunicator Allocator] org.apache.hadoop.yarn.util.RackResolver: Resolved mgmt-hadoop-dn-0.node.dc1.pnda.local to /default-rack 2018-08-30 09:30:29,546 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned container container_e02_1535621162012_0001_01_000003 to attempt_1535621162012_0001_m_000001_0 2018-08-30 09:30:29,547 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: After Scheduling: PendingReds:0 ScheduledMaps:3 ScheduledReds:0 AssignedMaps:2 AssignedReds:0 CompletedMaps:0 CompletedReds:0 ContAlloc:2 ContRel:0 HostLocal:0 RackLocal:0 2018-08-30 09:30:29,547 INFO [AsyncDispatcher event handler] org.apache.hadoop.yarn.util.RackResolver: Resolved mgmt-hadoop-dn-0.node.dc1.pnda.local to /default-rack 2018-08-30 09:30:29,547 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1535621162012_0001_m_000001_0 TaskAttempt Transitioned from UNASSIGNED to ASSIGNED 2018-08-30 09:30:29,553 INFO [ContainerLauncher #1] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_LAUNCH for container container_e02_1535621162012_0001_01_000003 taskAttempt attempt_1535621162012_0001_m_000001_0 2018-08-30 09:30:29,553 INFO [ContainerLauncher #1] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Launching attempt_1535621162012_0001_m_000001_0 2018-08-30 09:30:29,553 INFO [ContainerLauncher #1] org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy: Opening proxy : mgmt-hadoop-dn-0.node.dc1.pnda.local:45454 2018-08-30 09:30:29,600 INFO [ContainerLauncher #1] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Shuffle port returned by ContainerManager for attempt_1535621162012_0001_m_000001_0 : 13562 2018-08-30 09:30:29,600 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: TaskAttempt: [attempt_1535621162012_0001_m_000001_0] using containerId: [container_e02_1535621162012_0001_01_000003 on NM: [mgmt-hadoop-dn-0.node.dc1.pnda.local:45454] 2018-08-30 09:30:29,601 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1535621162012_0001_m_000001_0 TaskAttempt Transitioned from ASSIGNED to RUNNING 2018-08-30 09:30:29,601 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1535621162012_0001_m_000001 Task Transitioned from SCHEDULED to RUNNING 2018-08-30 09:30:30,553 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: getResources() for application_1535621162012_0001: ask=1 release= 0 newContainers=1 finishedContainers=0 resourcelimit= knownNMs=1 2018-08-30 09:30:30,553 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Got allocated containers 1 2018-08-30 09:30:30,553 INFO [RMCommunicator Allocator] org.apache.hadoop.yarn.util.RackResolver: Resolved mgmt-hadoop-dn-0.node.dc1.pnda.local to /default-rack 2018-08-30 09:30:30,553 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned container container_e02_1535621162012_0001_01_000004 to attempt_1535621162012_0001_m_000002_0 2018-08-30 09:30:30,553 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: After Scheduling: PendingReds:0 ScheduledMaps:2 ScheduledReds:0 AssignedMaps:3 AssignedReds:0 CompletedMaps:0 CompletedReds:0 ContAlloc:3 ContRel:0 HostLocal:0 RackLocal:0 2018-08-30 09:30:30,554 INFO [AsyncDispatcher event handler] org.apache.hadoop.yarn.util.RackResolver: Resolved mgmt-hadoop-dn-0.node.dc1.pnda.local to /default-rack 2018-08-30 09:30:30,554 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1535621162012_0001_m_000002_0 TaskAttempt Transitioned from UNASSIGNED to ASSIGNED 2018-08-30 09:30:30,555 INFO [ContainerLauncher #2] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_LAUNCH for container container_e02_1535621162012_0001_01_000004 taskAttempt attempt_1535621162012_0001_m_000002_0 2018-08-30 09:30:30,555 INFO [ContainerLauncher #2] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Launching attempt_1535621162012_0001_m_000002_0 2018-08-30 09:30:30,555 INFO [ContainerLauncher #2] org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy: Opening proxy : mgmt-hadoop-dn-0.node.dc1.pnda.local:45454 2018-08-30 09:30:30,591 INFO [ContainerLauncher #2] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Shuffle port returned by ContainerManager for attempt_1535621162012_0001_m_000002_0 : 13562 2018-08-30 09:30:30,591 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: TaskAttempt: [attempt_1535621162012_0001_m_000002_0] using containerId: [container_e02_1535621162012_0001_01_000004 on NM: [mgmt-hadoop-dn-0.node.dc1.pnda.local:45454] 2018-08-30 09:30:30,591 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1535621162012_0001_m_000002_0 TaskAttempt Transitioned from ASSIGNED to RUNNING 2018-08-30 09:30:30,591 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1535621162012_0001_m_000002 Task Transitioned from SCHEDULED to RUNNING 2018-08-30 09:30:31,571 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: getResources() for application_1535621162012_0001: ask=1 release= 0 newContainers=1 finishedContainers=0 resourcelimit= knownNMs=1 2018-08-30 09:30:31,571 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Got allocated containers 1 2018-08-30 09:30:31,571 INFO [RMCommunicator Allocator] org.apache.hadoop.yarn.util.RackResolver: Resolved mgmt-hadoop-dn-0.node.dc1.pnda.local to /default-rack 2018-08-30 09:30:31,571 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned container container_e02_1535621162012_0001_01_000005 to attempt_1535621162012_0001_m_000003_0 2018-08-30 09:30:31,571 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: After Scheduling: PendingReds:0 ScheduledMaps:1 ScheduledReds:0 AssignedMaps:4 AssignedReds:0 CompletedMaps:0 CompletedReds:0 ContAlloc:4 ContRel:0 HostLocal:0 RackLocal:0 2018-08-30 09:30:31,572 INFO [AsyncDispatcher event handler] org.apache.hadoop.yarn.util.RackResolver: Resolved mgmt-hadoop-dn-0.node.dc1.pnda.local to /default-rack 2018-08-30 09:30:31,577 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1535621162012_0001_m_000003_0 TaskAttempt Transitioned from UNASSIGNED to ASSIGNED 2018-08-30 09:30:31,606 INFO [ContainerLauncher #3] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_LAUNCH for container container_e02_1535621162012_0001_01_000005 taskAttempt attempt_1535621162012_0001_m_000003_0 2018-08-30 09:30:31,606 INFO [ContainerLauncher #3] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Launching attempt_1535621162012_0001_m_000003_0 2018-08-30 09:30:31,606 INFO [ContainerLauncher #3] org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy: Opening proxy : mgmt-hadoop-dn-0.node.dc1.pnda.local:45454 2018-08-30 09:30:31,676 INFO [ContainerLauncher #3] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Shuffle port returned by ContainerManager for attempt_1535621162012_0001_m_000003_0 : 13562 2018-08-30 09:30:31,679 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: TaskAttempt: [attempt_1535621162012_0001_m_000003_0] using containerId: [container_e02_1535621162012_0001_01_000005 on NM: [mgmt-hadoop-dn-0.node.dc1.pnda.local:45454] 2018-08-30 09:30:31,679 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1535621162012_0001_m_000003_0 TaskAttempt Transitioned from ASSIGNED to RUNNING 2018-08-30 09:30:31,679 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1535621162012_0001_m_000003 Task Transitioned from SCHEDULED to RUNNING 2018-08-30 09:30:32,187 INFO [Socket Reader #1 for port 33433] SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for job_1535621162012_0001 (auth:SIMPLE) 2018-08-30 09:30:32,263 INFO [IPC Server handler 1 on 33433] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID : jvm_1535621162012_0001_m_2199023255554 asked for a task 2018-08-30 09:30:32,263 INFO [IPC Server handler 1 on 33433] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID: jvm_1535621162012_0001_m_2199023255554 given task: attempt_1535621162012_0001_m_000000_0 2018-08-30 09:30:32,460 INFO [Socket Reader #1 for port 33433] SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for job_1535621162012_0001 (auth:SIMPLE) 2018-08-30 09:30:32,552 INFO [IPC Server handler 1 on 33433] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID : jvm_1535621162012_0001_m_2199023255555 asked for a task 2018-08-30 09:30:32,552 INFO [IPC Server handler 1 on 33433] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID: jvm_1535621162012_0001_m_2199023255555 given task: attempt_1535621162012_0001_m_000001_0 2018-08-30 09:30:32,592 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: getResources() for application_1535621162012_0001: ask=1 release= 0 newContainers=1 finishedContainers=0 resourcelimit= knownNMs=1 2018-08-30 09:30:32,592 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Got allocated containers 1 2018-08-30 09:30:32,592 INFO [RMCommunicator Allocator] org.apache.hadoop.yarn.util.RackResolver: Resolved mgmt-hadoop-dn-0.node.dc1.pnda.local to /default-rack 2018-08-30 09:30:32,592 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned container container_e02_1535621162012_0001_01_000006 to attempt_1535621162012_0001_m_000004_0 2018-08-30 09:30:32,593 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: After Scheduling: PendingReds:0 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:5 AssignedReds:0 CompletedMaps:0 CompletedReds:0 ContAlloc:5 ContRel:0 HostLocal:0 RackLocal:0 2018-08-30 09:30:32,593 INFO [AsyncDispatcher event handler] org.apache.hadoop.yarn.util.RackResolver: Resolved mgmt-hadoop-dn-0.node.dc1.pnda.local to /default-rack 2018-08-30 09:30:32,593 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1535621162012_0001_m_000004_0 TaskAttempt Transitioned from UNASSIGNED to ASSIGNED 2018-08-30 09:30:32,609 INFO [ContainerLauncher #4] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_LAUNCH for container container_e02_1535621162012_0001_01_000006 taskAttempt attempt_1535621162012_0001_m_000004_0 2018-08-30 09:30:32,609 INFO [ContainerLauncher #4] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Launching attempt_1535621162012_0001_m_000004_0 2018-08-30 09:30:32,609 INFO [ContainerLauncher #4] org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy: Opening proxy : mgmt-hadoop-dn-0.node.dc1.pnda.local:45454 2018-08-30 09:30:32,650 INFO [ContainerLauncher #4] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Shuffle port returned by ContainerManager for attempt_1535621162012_0001_m_000004_0 : 13562 2018-08-30 09:30:32,650 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: TaskAttempt: [attempt_1535621162012_0001_m_000004_0] using containerId: [container_e02_1535621162012_0001_01_000006 on NM: [mgmt-hadoop-dn-0.node.dc1.pnda.local:45454] 2018-08-30 09:30:32,650 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1535621162012_0001_m_000004_0 TaskAttempt Transitioned from ASSIGNED to RUNNING 2018-08-30 09:30:32,651 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1535621162012_0001_m_000004 Task Transitioned from SCHEDULED to RUNNING 2018-08-30 09:30:33,605 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: getResources() for application_1535621162012_0001: ask=1 release= 0 newContainers=1 finishedContainers=0 resourcelimit= knownNMs=1 2018-08-30 09:30:33,606 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Got allocated containers 1 2018-08-30 09:30:33,606 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Cannot assign container Container: [ContainerId: container_e02_1535621162012_0001_01_000007, NodeId: mgmt-hadoop-dn-0.node.dc1.pnda.local:45454, NodeHttpAddress: mgmt-hadoop-dn-0.node.dc1.pnda.local:8042, Resource: , Priority: 20, Token: Token { kind: ContainerToken, service: 10.0.1.158:45454 }, ] for a map as either container memory less than required or no pending map tasks - maps.isEmpty=true 2018-08-30 09:30:33,606 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: After Scheduling: PendingReds:0 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:5 AssignedReds:0 CompletedMaps:0 CompletedReds:0 ContAlloc:6 ContRel:1 HostLocal:0 RackLocal:0 2018-08-30 09:30:34,625 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: getResources() for application_1535621162012_0001: ask=0 release= 1 newContainers=0 finishedContainers=1 resourcelimit= knownNMs=1 2018-08-30 09:30:34,626 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Received completed container container_e02_1535621162012_0001_01_000007 2018-08-30 09:30:34,626 ERROR [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Container complete event for unknown container id container_e02_1535621162012_0001_01_000007 2018-08-30 09:30:34,809 INFO [Socket Reader #1 for port 33433] SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for job_1535621162012_0001 (auth:SIMPLE) 2018-08-30 09:30:34,903 INFO [IPC Server handler 2 on 33433] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID : jvm_1535621162012_0001_m_2199023255556 asked for a task 2018-08-30 09:30:34,903 INFO [IPC Server handler 2 on 33433] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID: jvm_1535621162012_0001_m_2199023255556 given task: attempt_1535621162012_0001_m_000002_0 2018-08-30 09:30:37,725 INFO [Socket Reader #1 for port 33433] SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for job_1535621162012_0001 (auth:SIMPLE) 2018-08-30 09:30:37,856 INFO [IPC Server handler 3 on 33433] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID : jvm_1535621162012_0001_m_2199023255557 asked for a task 2018-08-30 09:30:37,856 INFO [IPC Server handler 3 on 33433] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID: jvm_1535621162012_0001_m_2199023255557 given task: attempt_1535621162012_0001_m_000003_0 2018-08-30 09:30:40,732 INFO [Socket Reader #1 for port 33433] SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for job_1535621162012_0001 (auth:SIMPLE) 2018-08-30 09:30:40,817 INFO [IPC Server handler 3 on 33433] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1535621162012_0001_m_000000_0 is : 1.0 2018-08-30 09:30:40,921 INFO [IPC Server handler 19 on 33433] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID : jvm_1535621162012_0001_m_2199023255558 asked for a task 2018-08-30 09:30:40,922 INFO [IPC Server handler 19 on 33433] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID: jvm_1535621162012_0001_m_2199023255558 given task: attempt_1535621162012_0001_m_000004_0 2018-08-30 09:30:41,378 INFO [IPC Server handler 1 on 33433] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1535621162012_0001_m_000001_0 is : 1.0 2018-08-30 09:30:41,502 INFO [IPC Server handler 2 on 33433] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1535621162012_0001_m_000002_0 is : 0.0 2018-08-30 09:30:41,661 INFO [IPC Server handler 3 on 33433] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1535621162012_0001_m_000002_0 is : 1.0 2018-08-30 09:30:41,679 INFO [IPC Server handler 19 on 33433] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Done acknowledgement from attempt_1535621162012_0001_m_000002_0 2018-08-30 09:30:41,681 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1535621162012_0001_m_000002_0 TaskAttempt Transitioned from RUNNING to SUCCESS_CONTAINER_CLEANUP 2018-08-30 09:30:41,712 INFO [ContainerLauncher #5] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_CLEANUP for container container_e02_1535621162012_0001_01_000004 taskAttempt attempt_1535621162012_0001_m_000002_0 2018-08-30 09:30:41,712 INFO [ContainerLauncher #5] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: KILLING attempt_1535621162012_0001_m_000002_0 2018-08-30 09:30:41,713 INFO [ContainerLauncher #5] org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy: Opening proxy : mgmt-hadoop-dn-0.node.dc1.pnda.local:45454 2018-08-30 09:30:41,846 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1535621162012_0001_m_000002_0 TaskAttempt Transitioned from SUCCESS_CONTAINER_CLEANUP to SUCCEEDED 2018-08-30 09:30:41,854 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: Task succeeded with attempt attempt_1535621162012_0001_m_000002_0 2018-08-30 09:30:41,855 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1535621162012_0001_m_000002 Task Transitioned from RUNNING to SUCCEEDED 2018-08-30 09:30:41,857 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Num completed Tasks: 1 2018-08-30 09:30:42,744 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Before Scheduling: PendingReds:0 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:5 AssignedReds:0 CompletedMaps:1 CompletedReds:0 ContAlloc:6 ContRel:1 HostLocal:0 RackLocal:0 2018-08-30 09:30:42,758 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Received completed container container_e02_1535621162012_0001_01_000004 2018-08-30 09:30:42,759 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: After Scheduling: PendingReds:0 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:4 AssignedReds:0 CompletedMaps:1 CompletedReds:0 ContAlloc:6 ContRel:1 HostLocal:0 RackLocal:0 2018-08-30 09:30:42,762 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Diagnostics report from attempt_1535621162012_0001_m_000002_0: Container killed by the ApplicationMaster. Container killed on request. Exit code is 143 Container exited with a non-zero exit code 143. 2018-08-30 09:30:43,026 INFO [IPC Server handler 23 on 33433] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1535621162012_0001_m_000000_0 is : 1.0 2018-08-30 09:30:43,070 INFO [IPC Server handler 24 on 33433] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1535621162012_0001_m_000000_0 is : 1.0 2018-08-30 09:30:43,079 INFO [IPC Server handler 20 on 33433] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Done acknowledgement from attempt_1535621162012_0001_m_000000_0 2018-08-30 09:30:43,083 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1535621162012_0001_m_000000_0 TaskAttempt Transitioned from RUNNING to SUCCESS_CONTAINER_CLEANUP 2018-08-30 09:30:43,089 INFO [ContainerLauncher #6] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_CLEANUP for container container_e02_1535621162012_0001_01_000002 taskAttempt attempt_1535621162012_0001_m_000000_0 2018-08-30 09:30:43,090 INFO [ContainerLauncher #6] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: KILLING attempt_1535621162012_0001_m_000000_0 2018-08-30 09:30:43,090 INFO [ContainerLauncher #6] org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy: Opening proxy : mgmt-hadoop-dn-0.node.dc1.pnda.local:45454 2018-08-30 09:30:43,173 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1535621162012_0001_m_000000_0 TaskAttempt Transitioned from SUCCESS_CONTAINER_CLEANUP to SUCCEEDED 2018-08-30 09:30:43,173 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: Task succeeded with attempt attempt_1535621162012_0001_m_000000_0 2018-08-30 09:30:43,173 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1535621162012_0001_m_000000 Task Transitioned from RUNNING to SUCCEEDED 2018-08-30 09:30:43,173 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Num completed Tasks: 2 2018-08-30 09:30:43,759 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Before Scheduling: PendingReds:0 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:4 AssignedReds:0 CompletedMaps:2 CompletedReds:0 ContAlloc:6 ContRel:1 HostLocal:0 RackLocal:0 2018-08-30 09:30:43,764 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Received completed container container_e02_1535621162012_0001_01_000002 2018-08-30 09:30:43,764 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: After Scheduling: PendingReds:0 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:3 AssignedReds:0 CompletedMaps:2 CompletedReds:0 ContAlloc:6 ContRel:1 HostLocal:0 RackLocal:0 2018-08-30 09:30:43,764 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Diagnostics report from attempt_1535621162012_0001_m_000000_0: Container killed by the ApplicationMaster. Container killed on request. Exit code is 143 Container exited with a non-zero exit code 143. 2018-08-30 09:30:44,707 INFO [IPC Server handler 23 on 33433] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1535621162012_0001_m_000001_0 is : 1.0 2018-08-30 09:30:44,830 INFO [IPC Server handler 24 on 33433] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1535621162012_0001_m_000001_0 is : 1.0 2018-08-30 09:30:44,860 INFO [IPC Server handler 20 on 33433] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Done acknowledgement from attempt_1535621162012_0001_m_000001_0 2018-08-30 09:30:44,861 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1535621162012_0001_m_000001_0 TaskAttempt Transitioned from RUNNING to SUCCESS_CONTAINER_CLEANUP 2018-08-30 09:30:44,872 INFO [ContainerLauncher #7] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_CLEANUP for container container_e02_1535621162012_0001_01_000003 taskAttempt attempt_1535621162012_0001_m_000001_0 2018-08-30 09:30:44,874 INFO [ContainerLauncher #7] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: KILLING attempt_1535621162012_0001_m_000001_0 2018-08-30 09:30:44,874 INFO [ContainerLauncher #7] org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy: Opening proxy : mgmt-hadoop-dn-0.node.dc1.pnda.local:45454 2018-08-30 09:30:44,938 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1535621162012_0001_m_000001_0 TaskAttempt Transitioned from SUCCESS_CONTAINER_CLEANUP to SUCCEEDED 2018-08-30 09:30:44,938 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: Task succeeded with attempt attempt_1535621162012_0001_m_000001_0 2018-08-30 09:30:44,938 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1535621162012_0001_m_000001 Task Transitioned from RUNNING to SUCCEEDED 2018-08-30 09:30:44,939 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Num completed Tasks: 3 2018-08-30 09:30:45,169 INFO [IPC Server handler 5 on 33433] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1535621162012_0001_m_000004_0 is : 0.0 2018-08-30 09:30:45,280 INFO [IPC Server handler 29 on 33433] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1535621162012_0001_m_000004_0 is : 1.0 2018-08-30 09:30:45,283 INFO [IPC Server handler 28 on 33433] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Done acknowledgement from attempt_1535621162012_0001_m_000004_0 2018-08-30 09:30:45,285 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1535621162012_0001_m_000004_0 TaskAttempt Transitioned from RUNNING to SUCCESS_CONTAINER_CLEANUP 2018-08-30 09:30:45,309 INFO [ContainerLauncher #8] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_CLEANUP for container container_e02_1535621162012_0001_01_000006 taskAttempt attempt_1535621162012_0001_m_000004_0 2018-08-30 09:30:45,309 INFO [ContainerLauncher #8] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: KILLING attempt_1535621162012_0001_m_000004_0 2018-08-30 09:30:45,309 INFO [ContainerLauncher #8] org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy: Opening proxy : mgmt-hadoop-dn-0.node.dc1.pnda.local:45454 2018-08-30 09:30:45,361 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1535621162012_0001_m_000004_0 TaskAttempt Transitioned from SUCCESS_CONTAINER_CLEANUP to SUCCEEDED 2018-08-30 09:30:45,361 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: Task succeeded with attempt attempt_1535621162012_0001_m_000004_0 2018-08-30 09:30:45,362 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1535621162012_0001_m_000004 Task Transitioned from RUNNING to SUCCEEDED 2018-08-30 09:30:45,362 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Num completed Tasks: 4 2018-08-30 09:30:45,777 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Before Scheduling: PendingReds:0 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:3 AssignedReds:0 CompletedMaps:4 CompletedReds:0 ContAlloc:6 ContRel:1 HostLocal:0 RackLocal:0 2018-08-30 09:30:45,780 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Received completed container container_e02_1535621162012_0001_01_000003 2018-08-30 09:30:45,780 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Received completed container container_e02_1535621162012_0001_01_000006 2018-08-30 09:30:45,780 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: After Scheduling: PendingReds:0 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:1 AssignedReds:0 CompletedMaps:4 CompletedReds:0 ContAlloc:6 ContRel:1 HostLocal:0 RackLocal:0 2018-08-30 09:30:45,780 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Diagnostics report from attempt_1535621162012_0001_m_000001_0: Container killed by the ApplicationMaster. Container killed on request. Exit code is 143 Container exited with a non-zero exit code 143. 2018-08-30 09:30:45,780 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Diagnostics report from attempt_1535621162012_0001_m_000004_0: Container killed by the ApplicationMaster. Container killed on request. Exit code is 143 Container exited with a non-zero exit code 143. 2018-08-30 09:30:46,094 INFO [IPC Server handler 4 on 33433] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1535621162012_0001_m_000003_0 is : 0.0 2018-08-30 09:30:46,127 INFO [IPC Server handler 5 on 33433] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1535621162012_0001_m_000003_0 is : 1.0 2018-08-30 09:30:46,131 INFO [IPC Server handler 29 on 33433] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Done acknowledgement from attempt_1535621162012_0001_m_000003_0 2018-08-30 09:30:46,133 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1535621162012_0001_m_000003_0 TaskAttempt Transitioned from RUNNING to SUCCESS_CONTAINER_CLEANUP 2018-08-30 09:30:46,133 INFO [ContainerLauncher #9] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_CLEANUP for container container_e02_1535621162012_0001_01_000005 taskAttempt attempt_1535621162012_0001_m_000003_0 2018-08-30 09:30:46,133 INFO [ContainerLauncher #9] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: KILLING attempt_1535621162012_0001_m_000003_0 2018-08-30 09:30:46,133 INFO [ContainerLauncher #9] org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy: Opening proxy : mgmt-hadoop-dn-0.node.dc1.pnda.local:45454 2018-08-30 09:30:46,143 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1535621162012_0001_m_000003_0 TaskAttempt Transitioned from SUCCESS_CONTAINER_CLEANUP to SUCCEEDED 2018-08-30 09:30:46,143 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: Task succeeded with attempt attempt_1535621162012_0001_m_000003_0 2018-08-30 09:30:46,143 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1535621162012_0001_m_000003 Task Transitioned from RUNNING to SUCCEEDED 2018-08-30 09:30:46,143 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Num completed Tasks: 5 2018-08-30 09:30:46,144 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1535621162012_0001Job Transitioned from RUNNING to COMMITTING 2018-08-30 09:30:46,147 INFO [CommitterEvent Processor #1] org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler: Processing the event EventType: JOB_COMMIT 2018-08-30 09:30:46,161 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Calling handler for JobFinishedEvent 2018-08-30 09:30:46,161 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1535621162012_0001Job Transitioned from COMMITTING to SUCCEEDED 2018-08-30 09:30:46,162 INFO [Thread-76] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: We are finishing cleanly so this is the last retry 2018-08-30 09:30:46,162 INFO [Thread-76] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Notify RMCommunicator isAMLastRetry: true 2018-08-30 09:30:46,162 INFO [Thread-76] org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: RMCommunicator notified that shouldUnregistered is: true 2018-08-30 09:30:46,162 INFO [Thread-76] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Notify JHEH isAMLastRetry: true 2018-08-30 09:30:46,162 INFO [Thread-76] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: JobHistoryEventHandler notified that forceJobCompletion is true 2018-08-30 09:30:46,162 INFO [Thread-76] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Calling stop for all the services 2018-08-30 09:30:46,163 INFO [Thread-76] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Stopping JobHistoryEventHandler. Size of the outstanding queue size is 0 2018-08-30 09:30:46,205 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copying hdfs://mgmt-hadoop-mgr-1.node.dc1.pnda.local:8020/user/pnda/.staging/job_1535621162012_0001/job_1535621162012_0001_1.jhist to hdfs://mgmt-hadoop-mgr-1.node.dc1.pnda.local:8020/mr-history/tmp/pnda/job_1535621162012_0001-1535621415173-pnda-Gobblin%2DPullFromKafkaMR-1535621446157-5-0-SUCCEEDED-default-1535621426470.jhist_tmp 2018-08-30 09:30:46,619 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copied to done location: hdfs://mgmt-hadoop-mgr-1.node.dc1.pnda.local:8020/mr-history/tmp/pnda/job_1535621162012_0001-1535621415173-pnda-Gobblin%2DPullFromKafkaMR-1535621446157-5-0-SUCCEEDED-default-1535621426470.jhist_tmp 2018-08-30 09:30:46,621 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copying hdfs://mgmt-hadoop-mgr-1.node.dc1.pnda.local:8020/user/pnda/.staging/job_1535621162012_0001/job_1535621162012_0001_1_conf.xml to hdfs://mgmt-hadoop-mgr-1.node.dc1.pnda.local:8020/mr-history/tmp/pnda/job_1535621162012_0001_conf.xml_tmp 2018-08-30 09:30:46,635 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copied to done location: hdfs://mgmt-hadoop-mgr-1.node.dc1.pnda.local:8020/mr-history/tmp/pnda/job_1535621162012_0001_conf.xml_tmp 2018-08-30 09:30:46,642 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to done: hdfs://mgmt-hadoop-mgr-1.node.dc1.pnda.local:8020/mr-history/tmp/pnda/job_1535621162012_0001.summary_tmp to hdfs://mgmt-hadoop-mgr-1.node.dc1.pnda.local:8020/mr-history/tmp/pnda/job_1535621162012_0001.summary 2018-08-30 09:30:46,643 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to done: hdfs://mgmt-hadoop-mgr-1.node.dc1.pnda.local:8020/mr-history/tmp/pnda/job_1535621162012_0001_conf.xml_tmp to hdfs://mgmt-hadoop-mgr-1.node.dc1.pnda.local:8020/mr-history/tmp/pnda/job_1535621162012_0001_conf.xml 2018-08-30 09:30:46,644 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to done: hdfs://mgmt-hadoop-mgr-1.node.dc1.pnda.local:8020/mr-history/tmp/pnda/job_1535621162012_0001-1535621415173-pnda-Gobblin%2DPullFromKafkaMR-1535621446157-5-0-SUCCEEDED-default-1535621426470.jhist_tmp to hdfs://mgmt-hadoop-mgr-1.node.dc1.pnda.local:8020/mr-history/tmp/pnda/job_1535621162012_0001-1535621415173-pnda-Gobblin%2DPullFromKafkaMR-1535621446157-5-0-SUCCEEDED-default-1535621426470.jhist 2018-08-30 09:30:46,645 INFO [Thread-76] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Stopped JobHistoryEventHandler. super.stop() 2018-08-30 09:30:46,648 INFO [Thread-76] org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: Setting job diagnostics to 2018-08-30 09:30:46,649 INFO [Thread-76] org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: History url is http://mgmt-hadoop-mgr-1.node.dc1.pnda.local:19888/jobhistory/job/job_1535621162012_0001 2018-08-30 09:30:46,665 INFO [Thread-76] org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: Waiting for application to be successfully unregistered. 2018-08-30 09:30:47,666 INFO [Thread-76] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Final Stats: PendingReds:0 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:1 AssignedReds:0 CompletedMaps:4 CompletedReds:0 ContAlloc:6 ContRel:1 HostLocal:0 RackLocal:0 2018-08-30 09:30:47,667 INFO [Thread-76] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Deleting staging directory hdfs://mgmt-hadoop-mgr-1.node.dc1.pnda.local:8020 /user/pnda/.staging/job_1535621162012_0001 2018-08-30 09:30:47,673 INFO [Thread-76] org.apache.hadoop.ipc.Server: Stopping server on 33433 2018-08-30 09:30:47,676 INFO [IPC Server Responder] org.apache.hadoop.ipc.Server: Stopping IPC Server Responder 2018-08-30 09:30:47,676 INFO [TaskHeartbeatHandler PingChecker] org.apache.hadoop.mapreduce.v2.app.TaskHeartbeatHandler: TaskHeartbeatHandler thread interrupted 2018-08-30 09:30:47,676 INFO [IPC Server listener on 33433] org.apache.hadoop.ipc.Server: Stopping IPC Server listener on 33433