[HIVE-9187] Hive0.14.0 query fail with Tez0.5.3 and Hadoop2.4.1

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Won't Fix
• Fix Version/s: Not Applicable

Environment

CentOS 6

• Assignee: Unassigned
• Reporter: Ethan Hu
• Created: 22/Dec/14 03:10
• Updated: 21/Dec/23 20:14
• Resolved: 21/Dec/23 20:14

Description

The Operation System，CentOS 6
Software version, Hadoop 2.4.1, Tez 0.5.3, Hive 0.14.0.

The hive table structure is
CREATE EXTERNAL TABLE IF NOT EXISTS inv_info(optDate string, code int, custno int, sku int, euno int, totalamount double, price double, type int, finalamout double) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' LOCATION '/user/hadoop/inv';

Use Hive local Cli for query,
1. Use the default engine "mr", execute the Hive QL, it's normal
hive> select count from inv_info;
Automatically selecting local only mode for query
Query ID = root_20141221181313_60e150e1-34dd-4bea-8d4f-1a703c5516d6
Total jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
set mapreduce.job.reduces=<number>
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/webapp/hadoop-2.4.1/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/webapp/apache-hive-0.14.0-bin/lib/hive-jdbc-0.14.0-standalone.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
14/12/21 18:13:50 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
14/12/21 18:13:51 WARN conf.Configuration: file:/tmp/root/2006970b-1b5e-41e7-a604-4ed7405cc7e9/hive_2014-12-21_18-13-41_747_2372484449278617092-3/-local-10003/jobconf.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
14/12/21 18:13:51 WARN conf.Configuration: file:/tmp/root/2006970b-1b5e-41e7-a604-4ed7405cc7e9/hive_2014-12-21_18-13-41_747_2372484449278617092-3/-local-10003/jobconf.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
Execution log at: /tmp/root/root_20141221181313_60e150e1-34dd-4bea-8d4f-1a703c5516d6.log
Job running in-process (local Hadoop)
Hadoop job information for null: number of mappers: 0; number of reducers: 0
2014-12-21 18:14:00,666 null map = 0%, reduce = 0%
2014-12-21 18:14:05,365 null map = 100%, reduce = 100%
Ended Job = job_local886866076_0001
Execution completed successfully
MapredLocal task succeeded
OK
2146872
Time taken: 26.281 seconds, Fetched: 1 row(s)

2. when I set hive.execution.engine=tez; the query is fail, can anyone help me to find the reason? I looked up for many reference materials, can't resolve it.
hive> select count from inv_info;
Query ID = root_20141221181515_cbf8f20b-e105-441e-b044-269266088c48
Total jobs = 1
Launching Job 1 out of 1

Status: Running (Executing on YARN cluster with App id application_1418953370736_0009)

--------------------------------------------------------------------------------
VERTICES STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED
--------------------------------------------------------------------------------
Map 1 FAILED 4 0 0 4 9 3
Reducer 2 KILLED 1 0 0 1 0 1
--------------------------------------------------------------------------------
VERTICES: 00/02 [>>--------------------------] 0% ELAPSED TIME: 29.57 s
--------------------------------------------------------------------------------
Status: Failed
Vertex failed, vertexName=Map 1, vertexId=vertex_1418953370736_0009_1_00, diagnostics=[Task failed, taskId=task_1418953370736_0009_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Container container_1418953370736_0009_01_000002 finished with diagnostics set to [Container failed. Exception from container-launch: org.apache.hadoop.util.Shell$ExitCodeException:
org.apache.hadoop.util.Shell$ExitCodeException:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:505)
at org.apache.hadoop.util.Shell.run(Shell.java:418)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:650)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:300)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:701)

Container exited with a non-zero exit code 1
]], TaskAttempt 1 failed, info=[Container container_1418953370736_0009_01_000009 finished with diagnostics set to [Container failed. Exception from container-launch: org.apache.hadoop.util.Shell$ExitCodeException:
org.apache.hadoop.util.Shell$ExitCodeException:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:505)
at org.apache.hadoop.util.Shell.run(Shell.java:418)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:650)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:300)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:701)

Container exited with a non-zero exit code 1
]], TaskAttempt 2 failed, info=[Container container_1418953370736_0009_01_000010 finished with diagnostics set to [Container failed. Exception from container-launch: org.apache.hadoop.util.Shell$ExitCodeException:
org.apache.hadoop.util.Shell$ExitCodeException:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:505)
at org.apache.hadoop.util.Shell.run(Shell.java:418)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:650)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:300)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:701)

Container exited with a non-zero exit code 1
]], TaskAttempt 3 failed, info=[Container container_1418953370736_0009_01_000013 finished with diagnostics set to [Container failed. Exception from container-launch: org.apache.hadoop.util.Shell$ExitCodeException:
org.apache.hadoop.util.Shell$ExitCodeException:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:505)
at org.apache.hadoop.util.Shell.run(Shell.java:418)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:650)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:300)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:701)

Container exited with a non-zero exit code 1
]]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1418953370736_0009_1_00 [Map 1] killed/failed due to:null]
Vertex killed, vertexName=Reducer 2, vertexId=vertex_1418953370736_0009_1_01, diagnostics=[Vertex received Kill while in RUNNING state., Vertex killed as other vertex failed. failedTasks:0, Vertex vertex_1418953370736_0009_1_01 [Reducer 2] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:1
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask