2016-10-19 16:34:11,863 [INFO] [IPC Server handler 0 on 41231] |app.DAGAppMaster|: Running DAG: INSERT INTO TABLE test_table VA...'Oakland')(Stage-1), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=shfs3453_20161019163403_4260f79f-bb42-46a0-be1d-05c911211fbf } 2016-10-19 16:34:12,122 [INFO] [IPC Server handler 0 on 41231] |history.HistoryEventHandler|: [HISTORY][DAG:dag_1475850791417_0105_1][Event:DAG_SUBMITTED]: dagID=dag_1475850791417_0105_1, submitTime=1476887651803 2016-10-19 16:34:12,183 [INFO] [IPC Server handler 0 on 41231] |impl.VertexImpl|: Default container context for vertex_1475850791417_0105_1_00 [Map 1]=LocalResources: [[ name=tezlib, value=resource { scheme: "hdfs" host: "sandbox" port: -1 file: "/Products/TEZ/tez-0.8.4.tar.gz" } size: 46518370 timestamp: 1472807169156 type: ARCHIVE visibility: PUBLIC],[ name=hive-exec-2.1.0-a48a35a908c555af851f10ac610c0c368256944751b6d924b3731491eaf1d424.jar, value=resource { scheme: "hdfs" host: "sandbox" port: -1 file: "/user/shfs3453/.hiveJars/hive-exec-2.1.0-a48a35a908c555af851f10ac610c0c368256944751b6d924b3731491eaf1d424.jar" } size: 32316600 timestamp: 1467102497528 type: FILE visibility: PRIVATE],[ name=tez-conf.pb, value=resource { scheme: "hdfs" host: "sandbox" port: -1 file: "/tmp/hive/shfs3453/_tez_session_dir/5fbc35b1-b53e-4799-ac26-725aa2f48beb/.tez/application_1475850791417_0105/tez-conf.pb" } size: 81235 timestamp: 1476887644895 type: FILE visibility: APPLICATION],], environment: [[ SHELL=/bin/bash ],[ CLASSPATH=$PWD:$PWD/*:$PWD/tezlib/*:$PWD/tezlib/lib/*:$HADOOP_CONF_DIR: ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ LD_LIBRARY_PATH=$PWD:/opt/application/Hadoop/current/lib/native:$PWD:$PWD:/opt/application/Hadoop/current/lib/native:$PWD:/opt/application/Hadoop/current/lib/native/ ],], credentials(token kinds): [HDFS_DELEGATION_TOKEN,tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Xmx1900m -XX:+UseParallelGC -XX:ParallelGCThreads=4 -verbose:gc -XX:+PrintTenuringDistribution -XX:+PrintGCDetails -Xloggc:GC.log -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1475850791417_0105_1_00 [Map 1], Default Resources= 2016-10-19 16:34:12,183 [INFO] [IPC Server handler 0 on 41231] |impl.VertexImpl|: Setting 1 additional inputs for vertexvertex_1475850791417_0105_1_00 [Map 1] 2016-10-19 16:34:12,184 [INFO] [IPC Server handler 0 on 41231] |impl.VertexImpl|: Setting 1 additional outputs for vertex vertex_1475850791417_0105_1_00 [Map 1] 2016-10-19 16:34:12,185 [INFO] [IPC Server handler 0 on 41231] |impl.VertexImpl|: Using ExecutionContext from Vertex for Vertex Map 1 2016-10-19 16:34:12,186 [INFO] [IPC Server handler 0 on 41231] |impl.VertexImpl|: Running vertex: vertex_1475850791417_0105_1_00 [Map 1] : TaskScheduler=0:TezYarn, ContainerLauncher=0:TezYarn, TaskCommunicator=0:TezYarn 2016-10-19 16:34:12,186 [INFO] [IPC Server handler 0 on 41231] |impl.DAGImpl|: Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrder 2016-10-19 16:34:12,188 [INFO] [IPC Server handler 0 on 41231] |history.HistoryEventHandler|: [HISTORY][DAG:dag_1475850791417_0105_1][Event:DAG_INITIALIZED]: dagID=dag_1475850791417_0105_1, initTime=1476887652187 2016-10-19 16:34:12,188 [INFO] [IPC Server handler 0 on 41231] |impl.DAGImpl|: dag_1475850791417_0105_1 transitioned from NEW to INITED due to event DAG_INIT 2016-10-19 16:34:12,189 [INFO] [Dispatcher thread {Central}] |impl.DAGImpl|: Added additional resources : [[]] to classpath 2016-10-19 16:34:12,191 [INFO] [Dispatcher thread {Central}] |history.HistoryEventHandler|: [HISTORY][DAG:dag_1475850791417_0105_1][Event:DAG_STARTED]: dagID=dag_1475850791417_0105_1, startTime=1476887652189 2016-10-19 16:34:12,191 [INFO] [Dispatcher thread {Central}] |impl.DAGImpl|: dag_1475850791417_0105_1 transitioned from INITED to RUNNING due to event DAG_START 2016-10-19 16:34:12,191 [INFO] [Dispatcher thread {Central}] |impl.VertexImpl|: Root Inputs exist for Vertex: Map 1 : {values__tmp__table__25={InputName=values__tmp__table__25}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInputLegacy, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator, hasPayload=false}} 2016-10-19 16:34:12,191 [INFO] [Dispatcher thread {Central}] |impl.VertexImpl|: Starting root input initializer for input: values__tmp__table__25, with class: [org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator] 2016-10-19 16:34:12,191 [INFO] [Dispatcher thread {Central}] |impl.VertexImpl|: Setting vertexManager to RootInputVertexManager for vertex_1475850791417_0105_1_00 [Map 1] 2016-10-19 16:34:12,200 [INFO] [Dispatcher thread {Central}] |impl.VertexImpl|: Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1475850791417_0105_1_00 [Map 1] 2016-10-19 16:34:12,200 [INFO] [Dispatcher thread {Central}] |impl.VertexImpl|: Vertex will initialize from input initializer. vertex_1475850791417_0105_1_00 [Map 1] 2016-10-19 16:34:12,202 [INFO] [Dispatcher thread {Central}] |impl.VertexImpl|: Starting 1 inputInitializers for vertex vertex_1475850791417_0105_1_00 [Map 1] 2016-10-19 16:34:12,221 [INFO] [Dispatcher thread {Central}] |Configuration.deprecation|: mapred.committer.job.setup.cleanup.needed is deprecated. Instead, use mapreduce.job.committer.setup.cleanup.needed 2016-10-19 16:34:12,271 [INFO] [Dispatcher thread {Central}] |tez.HiveSplitGenerator|: SplitGenerator using llap affinitized locations: false 2016-10-19 16:34:12,272 [INFO] [Dispatcher thread {Central}] |tez.HiveSplitGenerator|: SplitLocationProvider: null 2016-10-19 16:34:12,282 [INFO] [Socket Reader #1 for port 41231] |ipc.Server|: Auth successful for shfs3453@SANDBOX.HADOOP (auth:SIMPLE) 2016-10-19 16:34:12,284 [INFO] [Socket Reader #1 for port 41231] |authorize.ServiceAuthorizationManager|: Authorization successful for shfs3453@SANDBOX.HADOOP (auth:TOKEN) for protocol=interface org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPB 2016-10-19 16:34:12,405 [INFO] [Dispatcher thread {Central}] |exec.Utilities|: PLAN PATH = hdfs://sandbox/tmp/hive/shfs3453/5fbc35b1-b53e-4799-ac26-725aa2f48beb/hive_2016-10-19_16-34-03_897_7672422288208573906-26/shfs3453/_tez_scratch_dir/34640bdb-4a0f-4fc9-811f-d558906171b3/map.xml 2016-10-19 16:34:12,414 [INFO] [Dispatcher thread {Central}] |exec.SerializationUtilities|: Deserializing MapWork using kryo 2016-10-19 16:34:12,501 [INFO] [InputInitializer {Map 1} #0] |dag.RootInputInitializerManager|: Starting InputInitializer for Input: values__tmp__table__25 on vertex vertex_1475850791417_0105_1_00 [Map 1] 2016-10-19 16:34:12,504 [INFO] [InputInitializer {Map 1} #0] |tez.HiveSplitGenerator|: GenerateConsistentSplitsInHive=true 2016-10-19 16:34:12,516 [INFO] [Dispatcher thread {Central}] |impl.VertexImpl|: vertex_1475850791417_0105_1_00 [Map 1] transitioned from NEW to INITIALIZING due to event V_INIT 2016-10-19 16:34:12,517 [INFO] [InputInitializer {Map 1} #0] |tez.HiveSplitGenerator|: The preferred split size is 52428800 2016-10-19 16:34:12,518 [INFO] [InputInitializer {Map 1} #0] |exec.Utilities|: PLAN PATH = hdfs://sandbox/tmp/hive/shfs3453/5fbc35b1-b53e-4799-ac26-725aa2f48beb/hive_2016-10-19_16-34-03_897_7672422288208573906-26/shfs3453/_tez_scratch_dir/34640bdb-4a0f-4fc9-811f-d558906171b3/map.xml 2016-10-19 16:34:12,520 [INFO] [InputInitializer {Map 1} #0] |exec.Utilities|: Processing alias values__tmp__table__25 2016-10-19 16:34:12,520 [INFO] [InputInitializer {Map 1} #0] |exec.Utilities|: Adding input file hdfs://sandbox/tmp/hive/shfs3453/5fbc35b1-b53e-4799-ac26-725aa2f48beb/_tmp_space.db/Values__Tmp__Table__25 2016-10-19 16:34:12,524 [INFO] [InputInitializer {Map 1} #0] |io.HiveInputFormat|: hive.io.file.read.all.columns = false 2016-10-19 16:34:12,524 [INFO] [InputInitializer {Map 1} #0] |io.HiveInputFormat|: hive.io.file.readcolumn.ids = 0,1,2 2016-10-19 16:34:12,525 [INFO] [InputInitializer {Map 1} #0] |io.HiveInputFormat|: hive.io.file.readcolumn.names = tmp_values_col1,tmp_values_col2,tmp_values_col3 2016-10-19 16:34:12,525 [INFO] [InputInitializer {Map 1} #0] |io.HiveInputFormat|: Generating splits for dirs: hdfs://sandbox/tmp/hive/shfs3453/5fbc35b1-b53e-4799-ac26-725aa2f48beb/_tmp_space.db/Values__Tmp__Table__25 2016-10-19 16:34:12,593 [INFO] [InputInitializer {Map 1} #0] |mapred.FileInputFormat|: Total input paths to process : 1 2016-10-19 16:34:12,599 [INFO] [InputInitializer {Map 1} #0] |io.HiveInputFormat|: number of splits 1 2016-10-19 16:34:12,599 [INFO] [InputInitializer {Map 1} #0] |tez.HiveSplitGenerator|: Number of input splits: 1. 5 available slots, 1.7 waves. Input format is: org.apache.hadoop.hive.ql.io.HiveInputFormat 2016-10-19 16:34:12,600 [INFO] [InputInitializer {Map 1} #0] |exec.Utilities|: PLAN PATH = hdfs://sandbox/tmp/hive/shfs3453/5fbc35b1-b53e-4799-ac26-725aa2f48beb/hive_2016-10-19_16-34-03_897_7672422288208573906-26/shfs3453/_tez_scratch_dir/34640bdb-4a0f-4fc9-811f-d558906171b3/map.xml 2016-10-19 16:34:12,600 [INFO] [InputInitializer {Map 1} #0] |exec.SerializationUtilities|: Deserializing MapWork using kryo 2016-10-19 16:34:12,607 [INFO] [InputInitializer {Map 1} #0] |tez.SplitGrouper|: # Src groups for split generation: 2 2016-10-19 16:34:12,609 [INFO] [InputInitializer {Map 1} #0] |tez.SplitGrouper|: Estimated number of tasks: 8 for bucket 1 2016-10-19 16:34:12,611 [INFO] [InputInitializer {Map 1} #0] |grouper.TezSplitGrouper|: Grouping splits in Tez 2016-10-19 16:34:12,612 [INFO] [InputInitializer {Map 1} #0] |grouper.TezSplitGrouper|: Desired splits: 1 too large. Desired splitLength: 2 Min splitLength: 52428800 New desired splits: 1 Final desired splits: 1 All splits have localhost: false Total length: 19 Original splits: 1 2016-10-19 16:34:12,612 [INFO] [InputInitializer {Map 1} #0] |grouper.TezSplitGrouper|: Using original number of splits: 1 desired splits: 1 2016-10-19 16:34:12,613 [INFO] [InputInitializer {Map 1} #0] |tez.SplitGrouper|: Original split count is 1 grouped split count is 1, for bucket: 1 2016-10-19 16:34:12,615 [INFO] [InputInitializer {Map 1} #0] |tez.HiveSplitGenerator|: Number of split groups: 1 2016-10-19 16:34:12,626 [INFO] [InputInitializer {Map 1} #0] |dag.RootInputInitializerManager|: Succeeded InputInitializer for Input: values__tmp__table__25 on vertex vertex_1475850791417_0105_1_00 [Map 1] 2016-10-19 16:34:12,630 [INFO] [App Shared Pool - #0] |impl.VertexImpl|: Cannot init vertex: vertex_1475850791417_0105_1_00 [Map 1] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2016-10-19 16:34:12,630 [INFO] [App Shared Pool - #0] |impl.VertexImpl|: Got updated RootInputsSpecs: {values__tmp__table__25=forAllWorkUnits=true, update=[1]} 2016-10-19 16:34:12,641 [INFO] [App Shared Pool - #0] |impl.VertexImpl|: Vertex vertex_1475850791417_0105_1_00 [Map 1] parallelism set to 1 2016-10-19 16:34:12,641 [INFO] [App Shared Pool - #0] |impl.VertexImpl|: Cannot init vertex: vertex_1475850791417_0105_1_00 [Map 1] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2016-10-19 16:34:12,657 [INFO] [Dispatcher thread {Central}] |impl.VertexImpl|: Setting up committers for vertex vertex_1475850791417_0105_1_00 [Map 1], numAdditionalOutputs=1 2016-10-19 16:34:12,659 [INFO] [Dispatcher thread {Central}] |history.HistoryEventHandler|: [HISTORY][DAG:dag_1475850791417_0105_1][Event:VERTEX_INITIALIZED]: vertexName=Map 1, vertexId=vertex_1475850791417_0105_1_00, initRequestedTime=1476887652191, initedTime=1476887652657, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, additionalInputsCount=1, initGeneratedEventsCount=1, servicePluginInfo=ServicePluginInfo {containerLauncherName=TezYarn, taskSchedulerName=TezYarn, taskCommunicatorName=TezYarn, containerLauncherClassName=org.apache.tez.dag.app.launcher.TezContainerLauncherImpl, taskSchedulerClassName=org.apache.tez.dag.app.rm.YarnTaskSchedulerService, taskCommunicatorClassName=org.apache.tez.dag.app.TezTaskCommunicatorImpl } 2016-10-19 16:34:12,659 [INFO] [Dispatcher thread {Central}] |impl.VertexImpl|: Triggering start event for vertex: vertex_1475850791417_0105_1_00 [Map 1] with distanceFromRoot: 0 2016-10-19 16:34:12,659 [INFO] [Dispatcher thread {Central}] |impl.VertexImpl|: vertex_1475850791417_0105_1_00 [Map 1] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION 2016-10-19 16:34:12,661 [INFO] [Dispatcher thread {Central}] |history.HistoryEventHandler|: [HISTORY][DAG:dag_1475850791417_0105_1][Event:VERTEX_STARTED]: vertexId=vertex_1475850791417_0105_1_00, startRequestedTime=1476887652521, startedTime=1476887652660 2016-10-19 16:34:12,663 [INFO] [Dispatcher thread {Central}] |history.HistoryEventHandler|: [HISTORY][DAG:dag_1475850791417_0105_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1475850791417_0105_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@9380d4f7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true 2016-10-19 16:34:12,663 [INFO] [Dispatcher thread {Central}] |impl.VertexImpl|: vertex_1475850791417_0105_1_00 [Map 1] transitioned from INITED to RUNNING due to event V_START 2016-10-19 16:34:12,664 [INFO] [App Shared Pool - #0] |impl.ImmediateStartVertexManager|: Starting 1 in Map 1 2016-10-19 16:34:12,665 [INFO] [App Shared Pool - #0] |impl.VertexImpl|: Routing pending task events for vertex: vertex_1475850791417_0105_1_00 [Map 1] 2016-10-19 16:34:12,671 [INFO] [Dispatcher thread {Central}] |history.HistoryEventHandler|: [HISTORY][DAG:dag_1475850791417_0105_1][Event:TASK_STARTED]: vertexName=Map 1, taskId=task_1475850791417_0105_1_00_000000, scheduledTime=1476887652670, launchTime=1476887652670 2016-10-19 16:34:12,719 [INFO] [Dispatcher thread {Central}] |util.RackResolver|: Resolved datanode03.bigdata.fr to /HZ_002A 2016-10-19 16:34:12,762 [INFO] [Dispatcher thread {Central}] |util.RackResolver|: Resolved datanode06.bigdata.fr to /HZ_002A 2016-10-19 16:34:12,803 [INFO] [Dispatcher thread {Central}] |util.RackResolver|: Resolved datanode05.bigdata.fr to /HZ_002A 2016-10-19 16:34:12,815 [INFO] [TaskSchedulerEventHandlerThread] |util.RackResolver|: Resolved datanode03.bigdata.fr to /HZ_002A 2016-10-19 16:34:12,816 [INFO] [TaskSchedulerEventHandlerThread] |util.RackResolver|: Resolved datanode06.bigdata.fr to /HZ_002A 2016-10-19 16:34:12,816 [INFO] [TaskSchedulerEventHandlerThread] |util.RackResolver|: Resolved datanode05.bigdata.fr to /HZ_002A 2016-10-19 16:34:12,825 [INFO] [TaskSchedulerEventHandlerThread] |rm.YarnTaskSchedulerService|: Allocation request for task: attempt_1475850791417_0105_1_00_000000_0 with request: Capability[]Priority[2] host: datanode03.bigdata.fr rack: null 2016-10-19 16:34:14,335 [INFO] [AMRM Heartbeater thread] |impl.AMRMClientImpl|: Received new token for : datanode06.bigdata.fr:60737 2016-10-19 16:34:14,348 [INFO] [AMRM Callback Handler Thread] |util.RackResolver|: Resolved datanode06.bigdata.fr to /HZ_002A 2016-10-19 16:34:14,353 [INFO] [DelayedContainerManager] |rm.YarnTaskSchedulerService|: Assigning container to task: containerId=container_1475850791417_0105_01_000002, task=attempt_1475850791417_0105_1_00_000000_0, containerHost=datanode06.bigdata.fr:60737, containerPriority= 2, containerResources=, localityMatchType=NodeLocal, matchedLocation=datanode06.bigdata.fr, honorLocalityFlags=true, reusedContainer=false, delayedContainers=0 2016-10-19 16:34:14,353 [INFO] [DelayedContainerManager] |util.RackResolver|: Resolved datanode03.bigdata.fr to /HZ_002A 2016-10-19 16:34:14,353 [INFO] [DelayedContainerManager] |util.RackResolver|: Resolved datanode06.bigdata.fr to /HZ_002A 2016-10-19 16:34:14,353 [INFO] [DelayedContainerManager] |util.RackResolver|: Resolved datanode05.bigdata.fr to /HZ_002A 2016-10-19 16:34:14,354 [INFO] [DelayedContainerManager] |util.RackResolver|: Resolved datanode06.bigdata.fr to /HZ_002A 2016-10-19 16:34:14,376 [INFO] [TaskSchedulerAppCallbackExecutor #0] |node.PerSourceNodeTracker|: Adding new node datanode06.bigdata.fr:60737 to nodeTracker 0 2016-10-19 16:34:14,402 [INFO] [ContainerLauncher #0] |launcher.TezContainerLauncherImpl|: Launching container_1475850791417_0105_01_000002 2016-10-19 16:34:14,403 [INFO] [ContainerLauncher #0] |impl.ContainerManagementProtocolProxy|: Opening proxy : datanode06.bigdata.fr:60737 2016-10-19 16:34:14,466 [INFO] [ContainerLauncher #0] |history.HistoryEventHandler|: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1475850791417_0105_01_000002, launchTime=1476887654466 2016-10-19 16:34:16,348 [INFO] [AMRM Callback Handler Thread] |rm.YarnTaskSchedulerService|: Allocated container completed:container_1475850791417_0105_01_000002 last allocated to task: attempt_1475850791417_0105_1_00_000000_0 2016-10-19 16:34:16,357 [INFO] [Dispatcher thread {Central}] |container.AMContainerImpl|: Container container_1475850791417_0105_01_000002 exited with diagnostics set to Container failed, exitCode=-1000. Application application_1475850791417_0105 initialization failed (exitCode=255) with output: main : command provided 0 main : user is shfs3453 main : requested yarn user is shfs3453 2016-10-19 16:34:16,358 [INFO] [Dispatcher thread {Central}] |history.HistoryEventHandler|: [HISTORY][DAG:dag_1475850791417_0105_1][Event:CONTAINER_STOPPED]: containerId=container_1475850791417_0105_01_000002, stoppedTime=1476887656358, exitStatus=-1000 2016-10-19 16:34:16,388 [INFO] [Dispatcher thread {Central}] |history.HistoryEventHandler|: [HISTORY][DAG:dag_1475850791417_0105_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=Map 1, taskAttemptId=attempt_1475850791417_0105_1_00_000000_0, creationTime=1476887652688, allocationTime=1476887654395, startTime=0, finishTime=1476887656361, timeTaken=1476887656361, status=FAILED, taskFailureType=NON_FATAL, errorEnum=CONTAINER_EXITED, diagnostics=Container container_1475850791417_0105_01_000002 finished with diagnostics set to [Container failed, exitCode=-1000. Application application_1475850791417_0105 initialization failed (exitCode=255) with output: main : command provided 0 main : user is shfs3453 main : requested yarn user is shfs3453 ], containerId=container_1475850791417_0105_01_000002, nodeId=datanode06.bigdata.fr:60737, nodeHttpAddress=datanode06.bigdata.fr:8042, counters=Counters: 0 2016-10-19 16:34:16,389 [INFO] [Dispatcher thread {Central}] |counters.Limits|: Counter limits initialized with parameters: GROUP_NAME_MAX=256, MAX_GROUPS=500, COUNTER_NAME_MAX=64, MAX_COUNTERS=1200 2016-10-19 16:34:16,440 [INFO] [Dispatcher thread {Central}] |impl.TaskImpl|: Scheduling new attempt for task: task_1475850791417_0105_1_00_000000, currentFailedAttempts: 1, maxFailedAttempts: 4 2016-10-19 16:34:16,441 [INFO] [TaskSchedulerEventHandlerThread] |rm.YarnTaskSchedulerService|: Ignoring removal of unknown task: attempt_1475850791417_0105_1_00_000000_0 2016-10-19 16:34:16,442 [INFO] [Dispatcher thread {Central}] |util.RackResolver|: Resolved datanode03.bigdata.fr to /HZ_002A 2016-10-19 16:34:16,442 [INFO] [TaskSchedulerEventHandlerThread] |rm.TaskSchedulerManager|: Task: attempt_1475850791417_0105_1_00_000000_0 has no container assignment in the scheduler 2016-10-19 16:34:16,442 [INFO] [Dispatcher thread {Central}] |util.RackResolver|: Resolved datanode06.bigdata.fr to /HZ_002A 2016-10-19 16:34:16,442 [INFO] [Dispatcher thread {Central}] |util.RackResolver|: Resolved datanode05.bigdata.fr to /HZ_002A 2016-10-19 16:34:16,442 [ERROR] [TaskSchedulerEventHandlerThread] |rm.TaskSchedulerManager|: No container allocated to task: attempt_1475850791417_0105_1_00_000000_0 according to scheduler. Task reported container id: container_1475850791417_0105_01_000002 2016-10-19 16:34:16,443 [INFO] [Dispatcher thread {Central}] |node.AMNodeImpl|: Attempt failed on node: datanode06.bigdata.fr:60737 TA: attempt_1475850791417_0105_1_00_000000_0 failed: true container: container_1475850791417_0105_01_000002 numFailedTAs: 0 2016-10-19 16:34:16,443 [INFO] [TaskSchedulerEventHandlerThread] |rm.YarnTaskSchedulerService|: Allocation request for task: attempt_1475850791417_0105_1_00_000000_1 with request: Capability[]Priority[1] host: null rack: null 2016-10-19 16:34:18,351 [INFO] [AMRM Callback Handler Thread] |util.RackResolver|: Resolved datanode06.bigdata.fr to /HZ_002A 2016-10-19 16:34:18,352 [INFO] [DelayedContainerManager] |rm.YarnTaskSchedulerService|: Assigning container to task: containerId=container_1475850791417_0105_01_000003, task=attempt_1475850791417_0105_1_00_000000_1, containerHost=datanode06.bigdata.fr:60737, containerPriority= 1, containerResources=, localityMatchType=NodeLocal, matchedLocation=datanode06.bigdata.fr, honorLocalityFlags=true, reusedContainer=false, delayedContainers=0 2016-10-19 16:34:18,352 [INFO] [DelayedContainerManager] |util.RackResolver|: Resolved datanode06.bigdata.fr to /HZ_002A 2016-10-19 16:34:18,356 [INFO] [ContainerLauncher #1] |launcher.TezContainerLauncherImpl|: Launching container_1475850791417_0105_01_000003 2016-10-19 16:34:18,356 [INFO] [ContainerLauncher #1] |impl.ContainerManagementProtocolProxy|: Opening proxy : datanode06.bigdata.fr:60737 2016-10-19 16:34:18,375 [INFO] [ContainerLauncher #1] |history.HistoryEventHandler|: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1475850791417_0105_01_000003, launchTime=1476887658375 2016-10-19 16:34:20,356 [INFO] [AMRM Callback Handler Thread] |rm.YarnTaskSchedulerService|: Allocated container completed:container_1475850791417_0105_01_000003 last allocated to task: attempt_1475850791417_0105_1_00_000000_1 2016-10-19 16:34:20,357 [INFO] [Dispatcher thread {Central}] |container.AMContainerImpl|: Container container_1475850791417_0105_01_000003 exited with diagnostics set to Container failed, exitCode=-1000. Application application_1475850791417_0105 initialization failed (exitCode=255) with output: main : command provided 0 main : user is shfs3453 main : requested yarn user is shfs3453 2016-10-19 16:34:20,358 [INFO] [Dispatcher thread {Central}] |history.HistoryEventHandler|: [HISTORY][DAG:dag_1475850791417_0105_1][Event:CONTAINER_STOPPED]: containerId=container_1475850791417_0105_01_000003, stoppedTime=1476887660358, exitStatus=-1000 2016-10-19 16:34:20,359 [INFO] [Dispatcher thread {Central}] |history.HistoryEventHandler|: [HISTORY][DAG:dag_1475850791417_0105_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=Map 1, taskAttemptId=attempt_1475850791417_0105_1_00_000000_1, creationTime=1476887656440, allocationTime=1476887658355, startTime=0, finishTime=1476887660358, timeTaken=1476887660358, status=FAILED, taskFailureType=NON_FATAL, errorEnum=CONTAINER_EXITED, diagnostics=Container container_1475850791417_0105_01_000003 finished with diagnostics set to [Container failed, exitCode=-1000. Application application_1475850791417_0105 initialization failed (exitCode=255) with output: main : command provided 0 main : user is shfs3453 main : requested yarn user is shfs3453 ], containerId=container_1475850791417_0105_01_000003, nodeId=datanode06.bigdata.fr:60737, nodeHttpAddress=datanode06.bigdata.fr:8042, counters=Counters: 0 2016-10-19 16:34:20,359 [INFO] [Dispatcher thread {Central}] |impl.TaskImpl|: Scheduling new attempt for task: task_1475850791417_0105_1_00_000000, currentFailedAttempts: 2, maxFailedAttempts: 4 2016-10-19 16:34:20,360 [INFO] [TaskSchedulerEventHandlerThread] |rm.YarnTaskSchedulerService|: Ignoring removal of unknown task: attempt_1475850791417_0105_1_00_000000_1 2016-10-19 16:34:20,361 [INFO] [Dispatcher thread {Central}] |util.RackResolver|: Resolved datanode03.bigdata.fr to /HZ_002A 2016-10-19 16:34:20,361 [INFO] [TaskSchedulerEventHandlerThread] |rm.TaskSchedulerManager|: Task: attempt_1475850791417_0105_1_00_000000_1 has no container assignment in the scheduler 2016-10-19 16:34:20,361 [INFO] [Dispatcher thread {Central}] |util.RackResolver|: Resolved datanode06.bigdata.fr to /HZ_002A 2016-10-19 16:34:20,361 [ERROR] [TaskSchedulerEventHandlerThread] |rm.TaskSchedulerManager|: No container allocated to task: attempt_1475850791417_0105_1_00_000000_1 according to scheduler. Task reported container id: container_1475850791417_0105_01_000003 2016-10-19 16:34:20,361 [INFO] [Dispatcher thread {Central}] |util.RackResolver|: Resolved datanode05.bigdata.fr to /HZ_002A 2016-10-19 16:34:20,362 [INFO] [Dispatcher thread {Central}] |node.AMNodeImpl|: Attempt failed on node: datanode06.bigdata.fr:60737 TA: attempt_1475850791417_0105_1_00_000000_1 failed: true container: container_1475850791417_0105_01_000003 numFailedTAs: 1 2016-10-19 16:34:20,362 [INFO] [TaskSchedulerEventHandlerThread] |rm.YarnTaskSchedulerService|: Allocation request for task: attempt_1475850791417_0105_1_00_000000_2 with request: Capability[]Priority[1] host: null rack: null 2016-10-19 16:34:22,362 [INFO] [AMRM Callback Handler Thread] |util.RackResolver|: Resolved datanode06.bigdata.fr to /HZ_002A 2016-10-19 16:34:22,363 [INFO] [DelayedContainerManager] |rm.YarnTaskSchedulerService|: Assigning container to task: containerId=container_1475850791417_0105_01_000004, task=attempt_1475850791417_0105_1_00_000000_2, containerHost=datanode06.bigdata.fr:60737, containerPriority= 1, containerResources=, localityMatchType=NodeLocal, matchedLocation=datanode06.bigdata.fr, honorLocalityFlags=true, reusedContainer=false, delayedContainers=0 2016-10-19 16:34:22,363 [INFO] [DelayedContainerManager] |util.RackResolver|: Resolved datanode06.bigdata.fr to /HZ_002A 2016-10-19 16:34:22,367 [INFO] [ContainerLauncher #2] |launcher.TezContainerLauncherImpl|: Launching container_1475850791417_0105_01_000004 2016-10-19 16:34:22,367 [INFO] [ContainerLauncher #2] |impl.ContainerManagementProtocolProxy|: Opening proxy : datanode06.bigdata.fr:60737 2016-10-19 16:34:22,385 [INFO] [ContainerLauncher #2] |history.HistoryEventHandler|: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1475850791417_0105_01_000004, launchTime=1476887662385 2016-10-19 16:34:24,367 [INFO] [AMRM Callback Handler Thread] |rm.YarnTaskSchedulerService|: Allocated container completed:container_1475850791417_0105_01_000004 last allocated to task: attempt_1475850791417_0105_1_00_000000_2 2016-10-19 16:34:24,369 [INFO] [Dispatcher thread {Central}] |container.AMContainerImpl|: Container container_1475850791417_0105_01_000004 exited with diagnostics set to Container failed, exitCode=-1000. Application application_1475850791417_0105 initialization failed (exitCode=255) with output: main : command provided 0 main : user is shfs3453 main : requested yarn user is shfs3453 2016-10-19 16:34:24,369 [INFO] [Dispatcher thread {Central}] |history.HistoryEventHandler|: [HISTORY][DAG:dag_1475850791417_0105_1][Event:CONTAINER_STOPPED]: containerId=container_1475850791417_0105_01_000004, stoppedTime=1476887664369, exitStatus=-1000 2016-10-19 16:34:24,370 [INFO] [Dispatcher thread {Central}] |history.HistoryEventHandler|: [HISTORY][DAG:dag_1475850791417_0105_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=Map 1, taskAttemptId=attempt_1475850791417_0105_1_00_000000_2, creationTime=1476887660360, allocationTime=1476887662365, startTime=0, finishTime=1476887664369, timeTaken=1476887664369, status=FAILED, taskFailureType=NON_FATAL, errorEnum=CONTAINER_EXITED, diagnostics=Container container_1475850791417_0105_01_000004 finished with diagnostics set to [Container failed, exitCode=-1000. Application application_1475850791417_0105 initialization failed (exitCode=255) with output: main : command provided 0 main : user is shfs3453 main : requested yarn user is shfs3453 ], containerId=container_1475850791417_0105_01_000004, nodeId=datanode06.bigdata.fr:60737, nodeHttpAddress=datanode06.bigdata.fr:8042, counters=Counters: 0 2016-10-19 16:34:24,370 [INFO] [Dispatcher thread {Central}] |impl.TaskImpl|: Scheduling new attempt for task: task_1475850791417_0105_1_00_000000, currentFailedAttempts: 3, maxFailedAttempts: 4 2016-10-19 16:34:24,371 [INFO] [TaskSchedulerEventHandlerThread] |rm.YarnTaskSchedulerService|: Ignoring removal of unknown task: attempt_1475850791417_0105_1_00_000000_2 2016-10-19 16:34:24,372 [INFO] [Dispatcher thread {Central}] |util.RackResolver|: Resolved datanode03.bigdata.fr to /HZ_002A 2016-10-19 16:34:24,372 [INFO] [TaskSchedulerEventHandlerThread] |rm.TaskSchedulerManager|: Task: attempt_1475850791417_0105_1_00_000000_2 has no container assignment in the scheduler 2016-10-19 16:34:24,372 [INFO] [Dispatcher thread {Central}] |util.RackResolver|: Resolved datanode06.bigdata.fr to /HZ_002A 2016-10-19 16:34:24,372 [INFO] [Dispatcher thread {Central}] |util.RackResolver|: Resolved datanode05.bigdata.fr to /HZ_002A 2016-10-19 16:34:24,372 [ERROR] [TaskSchedulerEventHandlerThread] |rm.TaskSchedulerManager|: No container allocated to task: attempt_1475850791417_0105_1_00_000000_2 according to scheduler. Task reported container id: container_1475850791417_0105_01_000004 2016-10-19 16:34:24,373 [INFO] [Dispatcher thread {Central}] |node.AMNodeImpl|: Attempt failed on node: datanode06.bigdata.fr:60737 TA: attempt_1475850791417_0105_1_00_000000_2 failed: true container: container_1475850791417_0105_01_000004 numFailedTAs: 2 2016-10-19 16:34:24,373 [INFO] [TaskSchedulerEventHandlerThread] |rm.YarnTaskSchedulerService|: Allocation request for task: attempt_1475850791417_0105_1_00_000000_3 with request: Capability[]Priority[1] host: null rack: null 2016-10-19 16:34:26,376 [INFO] [AMRM Callback Handler Thread] |util.RackResolver|: Resolved datanode06.bigdata.fr to /HZ_002A 2016-10-19 16:34:26,376 [INFO] [DelayedContainerManager] |rm.YarnTaskSchedulerService|: Assigning container to task: containerId=container_1475850791417_0105_01_000005, task=attempt_1475850791417_0105_1_00_000000_3, containerHost=datanode06.bigdata.fr:60737, containerPriority= 1, containerResources=, localityMatchType=NodeLocal, matchedLocation=datanode06.bigdata.fr, honorLocalityFlags=true, reusedContainer=false, delayedContainers=0 2016-10-19 16:34:26,377 [INFO] [DelayedContainerManager] |util.RackResolver|: Resolved datanode06.bigdata.fr to /HZ_002A 2016-10-19 16:34:26,380 [INFO] [ContainerLauncher #3] |launcher.TezContainerLauncherImpl|: Launching container_1475850791417_0105_01_000005 2016-10-19 16:34:26,380 [INFO] [ContainerLauncher #3] |impl.ContainerManagementProtocolProxy|: Opening proxy : datanode06.bigdata.fr:60737 2016-10-19 16:34:26,398 [INFO] [ContainerLauncher #3] |history.HistoryEventHandler|: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1475850791417_0105_01_000005, launchTime=1476887666398 2016-10-19 16:34:28,380 [INFO] [AMRM Callback Handler Thread] |rm.YarnTaskSchedulerService|: Allocated container completed:container_1475850791417_0105_01_000005 last allocated to task: attempt_1475850791417_0105_1_00_000000_3 2016-10-19 16:34:28,382 [INFO] [Dispatcher thread {Central}] |container.AMContainerImpl|: Container container_1475850791417_0105_01_000005 exited with diagnostics set to Container failed, exitCode=-1000. Application application_1475850791417_0105 initialization failed (exitCode=255) with output: main : command provided 0 main : user is shfs3453 main : requested yarn user is shfs3453 2016-10-19 16:34:28,382 [INFO] [Dispatcher thread {Central}] |history.HistoryEventHandler|: [HISTORY][DAG:dag_1475850791417_0105_1][Event:CONTAINER_STOPPED]: containerId=container_1475850791417_0105_01_000005, stoppedTime=1476887668382, exitStatus=-1000 2016-10-19 16:34:28,383 [INFO] [Dispatcher thread {Central}] |history.HistoryEventHandler|: [HISTORY][DAG:dag_1475850791417_0105_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=Map 1, taskAttemptId=attempt_1475850791417_0105_1_00_000000_3, creationTime=1476887664371, allocationTime=1476887666378, startTime=0, finishTime=1476887668382, timeTaken=1476887668382, status=FAILED, taskFailureType=NON_FATAL, errorEnum=CONTAINER_EXITED, diagnostics=Container container_1475850791417_0105_01_000005 finished with diagnostics set to [Container failed, exitCode=-1000. Application application_1475850791417_0105 initialization failed (exitCode=255) with output: main : command provided 0 main : user is shfs3453 main : requested yarn user is shfs3453 ], containerId=container_1475850791417_0105_01_000005, nodeId=datanode06.bigdata.fr:60737, nodeHttpAddress=datanode06.bigdata.fr:8042, counters=Counters: 0 2016-10-19 16:34:28,384 [INFO] [Dispatcher thread {Central}] |impl.TaskImpl|: Failing task: task_1475850791417_0105_1_00_000000 due to too many failed attempts. currentFailedAttempts=4, maxFailedAttempts=4 2016-10-19 16:34:28,387 [INFO] [Dispatcher thread {Central}] |history.HistoryEventHandler|: [HISTORY][DAG:dag_1475850791417_0105_1][Event:TASK_FINISHED]: vertexName=Map 1, taskId=task_1475850791417_0105_1_00_000000, startTime=1476887652670, finishTime=1476887668384, timeTaken=15714, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Container container_1475850791417_0105_01_000002 finished with diagnostics set to [Container failed, exitCode=-1000. Application application_1475850791417_0105 initialization failed (exitCode=255) with output: main : command provided 0 main : user is shfs3453 main : requested yarn user is shfs3453 ]] TaskAttempt 1 failed, info=[Container container_1475850791417_0105_01_000003 finished with diagnostics set to [Container failed, exitCode=-1000. Application application_1475850791417_0105 initialization failed (exitCode=255) with output: main : command provided 0 main : user is shfs3453 main : requested yarn user is shfs3453 ]] TaskAttempt 2 failed, info=[Container container_1475850791417_0105_01_000004 finished with diagnostics set to [Container failed, exitCode=-1000. Application application_1475850791417_0105 initialization failed (exitCode=255) with output: main : command provided 0 main : user is shfs3453 main : requested yarn user is shfs3453 ]] TaskAttempt 3 failed, info=[Container container_1475850791417_0105_01_000005 finished with diagnostics set to [Container failed, exitCode=-1000. Application application_1475850791417_0105 initialization failed (exitCode=255) with output: main : command provided 0 main : user is shfs3453 main : requested yarn user is shfs3453 ]], counters=Counters: 0 2016-10-19 16:34:28,388 [INFO] [TaskSchedulerEventHandlerThread] |rm.YarnTaskSchedulerService|: Ignoring removal of unknown task: attempt_1475850791417_0105_1_00_000000_3 2016-10-19 16:34:28,388 [INFO] [TaskSchedulerEventHandlerThread] |rm.TaskSchedulerManager|: Task: attempt_1475850791417_0105_1_00_000000_3 has no container assignment in the scheduler 2016-10-19 16:34:28,389 [ERROR] [TaskSchedulerEventHandlerThread] |rm.TaskSchedulerManager|: No container allocated to task: attempt_1475850791417_0105_1_00_000000_3 according to scheduler. Task reported container id: container_1475850791417_0105_01_000005 2016-10-19 16:34:28,392 [INFO] [Dispatcher thread {Central}] |impl.VertexImpl|: Failing vertex: vertex_1475850791417_0105_1_00 [Map 1] because task failed: task_1475850791417_0105_1_00_000000 2016-10-19 16:34:28,394 [INFO] [Dispatcher thread {Central}] |impl.VertexImpl|: Killing tasks in vertex: vertex_1475850791417_0105_1_00 [Map 1] due to trigger: OWN_TASK_FAILURE 2016-10-19 16:34:28,395 [INFO] [Dispatcher thread {Central}] |impl.VertexImpl|: Task Completion: vertex_1475850791417_0105_1_00 [Map 1], tasks=1, failed=1, killed=0, success=0, completed=1, commits=0, err=OWN_TASK_FAILURE 2016-10-19 16:34:28,395 [INFO] [Dispatcher thread {Central}] |impl.VertexImpl|: Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0 2016-10-19 16:34:28,396 [INFO] [Dispatcher thread {Central}] |impl.VertexImpl|: Invoking committer abort for vertex, vertexId=vertex_1475850791417_0105_1_00 [Map 1] 2016-10-19 16:34:28,409 [INFO] [Dispatcher thread {Central}] |history.HistoryEventHandler|: [HISTORY][DAG:dag_1475850791417_0105_1][Event:VERTEX_FINISHED]: vertexName=Map 1, vertexId=vertex_1475850791417_0105_1_00, initRequestedTime=1476887652191, initedTime=1476887652657, startRequestedTime=1476887652521, startedTime=1476887652660, finishTime=1476887668398, timeTaken=15738, status=FAILED, diagnostics=Task failed, taskId=task_1475850791417_0105_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Container container_1475850791417_0105_01_000002 finished with diagnostics set to [Container failed, exitCode=-1000. Application application_1475850791417_0105 initialization failed (exitCode=255) with output: main : command provided 0 main : user is shfs3453 main : requested yarn user is shfs3453 ]], TaskAttempt 1 failed, info=[Container container_1475850791417_0105_01_000003 finished with diagnostics set to [Container failed, exitCode=-1000. Application application_1475850791417_0105 initialization failed (exitCode=255) with output: main : command provided 0 main : user is shfs3453 main : requested yarn user is shfs3453 ]], TaskAttempt 2 failed, info=[Container container_1475850791417_0105_01_000004 finished with diagnostics set to [Container failed, exitCode=-1000. Application application_1475850791417_0105 initialization failed (exitCode=255) with output: main : command provided 0 main : user is shfs3453 main : requested yarn user is shfs3453 ]], TaskAttempt 3 failed, info=[Container container_1475850791417_0105_01_000005 finished with diagnostics set to [Container failed, exitCode=-1000. Application application_1475850791417_0105 initialization failed (exitCode=255) with output: main : command provided 0 main : user is shfs3453 main : requested yarn user is shfs3453 ]]] Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0 Vertex vertex_1475850791417_0105_1_00 [Map 1] killed/failed due to:OWN_TASK_FAILURE, counters=Counters: 0, vertexStats=firstTaskStartTime=1476887652670, firstTasksToStart=[ task_1475850791417_0105_1_00_000000 ], lastTaskFinishTime=-1, lastTasksToFinish=[ ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=4, numCompletedTasks=1}, servicePluginInfo=ServicePluginInfo {containerLauncherName=TezYarn, taskSchedulerName=TezYarn, taskCommunicatorName=TezYarn, containerLauncherClassName=org.apache.tez.dag.app.launcher.TezContainerLauncherImpl, taskSchedulerClassName=org.apache.tez.dag.app.rm.YarnTaskSchedulerService, taskCommunicatorClassName=org.apache.tez.dag.app.TezTaskCommunicatorImpl } 2016-10-19 16:34:28,409 [INFO] [Dispatcher thread {Central}] |impl.VertexImpl|: vertex_1475850791417_0105_1_00 [Map 1] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED 2016-10-19 16:34:28,409 [INFO] [Dispatcher thread {Central}] |node.AMNodeImpl|: Attempt failed on node: datanode06.bigdata.fr:60737 TA: attempt_1475850791417_0105_1_00_000000_3 failed: true container: container_1475850791417_0105_01_000005 numFailedTAs: 3 2016-10-19 16:34:28,410 [INFO] [Dispatcher thread {Central}] |impl.DAGImpl|: Vertex vertex_1475850791417_0105_1_00 [Map 1] completed., numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1 2016-10-19 16:34:28,410 [INFO] [Dispatcher thread {Central}] |impl.DAGImpl|: Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=VERTEX_FAILURE 2016-10-19 16:34:28,410 [INFO] [Dispatcher thread {Central}] |impl.DAGImpl|: DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:0 2016-10-19 16:34:28,428 [INFO] [Dispatcher thread {Central}] |recovery.RecoveryService|: DAG completed, dagId=dag_1475850791417_0105_1, queueSize=0 2016-10-19 16:34:28,454 [INFO] [Dispatcher thread {Central}] |history.HistoryEventHandler|: [HISTORY][DAG:dag_1475850791417_0105_1][Event:DAG_FINISHED]: dagId=dag_1475850791417_0105_1, startTime=1476887652189, finishTime=1476887668410, timeTaken=16221, status=FAILED, diagnostics=Vertex failed, vertexName=Map 1, vertexId=vertex_1475850791417_0105_1_00, diagnostics=[Task failed, taskId=task_1475850791417_0105_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Container container_1475850791417_0105_01_000002 finished with diagnostics set to [Container failed, exitCode=-1000. Application application_1475850791417_0105 initialization failed (exitCode=255) with output: main : command provided 0 main : user is shfs3453 main : requested yarn user is shfs3453 ]], TaskAttempt 1 failed, info=[Container container_1475850791417_0105_01_000003 finished with diagnostics set to [Container failed, exitCode=-1000. Application application_1475850791417_0105 initialization failed (exitCode=255) with output: main : command provided 0 main : user is shfs3453 main : requested yarn user is shfs3453 ]], TaskAttempt 2 failed, info=[Container container_1475850791417_0105_01_000004 finished with diagnostics set to [Container failed, exitCode=-1000. Application application_1475850791417_0105 initialization failed (exitCode=255) with output: main : command provided 0 main : user is shfs3453 main : requested yarn user is shfs3453 ]], TaskAttempt 3 failed, info=[Container container_1475850791417_0105_01_000005 finished with diagnostics set to [Container failed, exitCode=-1000. Application application_1475850791417_0105 initialization failed (exitCode=255) with output: main : command provided 0 main : user is shfs3453 main : requested yarn user is shfs3453 ]]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0, Vertex vertex_1475850791417_0105_1_00 [Map 1] killed/failed due to:OWN_TASK_FAILURE] DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:0, counters=Counters: 3, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=4, AM_CPU_MILLISECONDS=1930, AM_GC_TIME_MILLIS=26 2016-10-19 16:34:28,454 [INFO] [Dispatcher thread {Central}] |impl.VertexImpl|: Ignoring multiple aborts for vertex: vertex_1475850791417_0105_1_00 [Map 1] 2016-10-19 16:34:28,455 [INFO] [Dispatcher thread {Central}] |impl.DAGImpl|: DAG: dag_1475850791417_0105_1 finished with state: FAILED 2016-10-19 16:34:28,455 [INFO] [Dispatcher thread {Central}] |impl.DAGImpl|: dag_1475850791417_0105_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED 2016-10-19 16:34:28,456 [INFO] [Dispatcher thread {Central}] |app.DAGAppMaster|: DAG completed, dagId=dag_1475850791417_0105_1, dagState=FAILED 2016-10-19 16:34:28,456 [INFO] [Dispatcher thread {Central}] |common.TezUtilsInternal|: Redirecting log file based on addend: dag_1475850791417_0105_1_post